Fair competition review method based on contrast sentence vector representation framework

Abstract

The invention relates to the technical field of natural language processing, and discloses a fair competition review method based on a contrast sentence vector representation framework, which comprises the following steps: S1, collecting a large number of policy and measure files; s2, constructing a case library of fair competition review based on expert review; s3, training the semantic similarity calculation model; s4, performing semantic similarity calculation on the to-be-examined file obtained in the S1 and the case library file obtained in the S2 through the semantic similarity calculation model obtained in the S3; and S5, identifying terms of related suspected violation based on a semantic similarity result in the step S4, thereby realizing violation type identification and judgment of the to-be-examined file. According to the fair competition review method based on the contrast sentence vector representation framework, a fair competition review case library based on expert review can be constructed through policy and measure text collection, construction and training of a semantic similarity calculation model, semantic similarity calculation of a to-be-reviewed file and a case library file and violation type recognition of the to-be-reviewed file; and the intelligent level and the working efficiency of examination work are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a fair competition review method based on a comparative sentence vector representation framework. Background Art

[0002] Fair competition is the basic principle of the market economy and an important basis for the efficient operation of the market mechanism.

[0003] Fair competition review aims to promote equal treatment of various market players in the market and prevent unfair market practices such as local protectionism. The review targets the regulations, normative documents and other policy measures of the state and government in formulating market access, industrial development, investment promotion, bidding, government procurement, business conduct norms, and qualification standards involving the economic activities of market players. The review goal is to determine whether there are any issues that hinder the unified market and fair competition in accordance with the "Detailed Rules for the Implementation of the Fair Competition Review System".

[0004] The current problems in the field of fair competition review are as follows: 1) The existing policy measures are wide-ranging and large in number, and the regular review of policy measure documents is costly; 2) Manual review is currently adopted, and reviewers need to have professional knowledge of competition law and economics and be familiar with industry conditions, resulting in difficulty and low efficiency in the review. It is also very easy for the review results to be difficult to unify due to inconsistent review standards. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a fair competition review method based on a comparative sentence vector representation framework to solve the problems of low accuracy and lack of sample data in the existing fair competition review model based on the same level labels, improve the intelligence level of the review work, improve work efficiency, and effectively solve the problem of insufficient review and supervision resources.

[0007] (II) Technical solution

[0008] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a fair competition review method based on a comparative sentence vector representation framework, comprising:

[0009] S1. Collect a large number of policy and measure documents and obtain the data set of documents to be reviewed;

[0010] S2. Build a case library for fair competition review based on expert review and update it incrementally;

[0011] S3. Based on the fair competition review case library reviewed by experts obtained in step S2 and combined with the expert annotation results, the semantic similarity calculation model is trained and the parameters of the final model are saved;

[0012] S4. The document to be reviewed obtained by S1 is used to calculate the semantic similarity with the case library document in S2 through the semantic similarity calculation model obtained by S3;

[0013] S5. Based on the semantic similarity results of S4, the relevant clauses suspected of violations are identified, so as to identify and judge the violation types of the documents to be reviewed.

[0014] Preferably, collecting a large number of policy and measure documents in S1 to obtain a data set of documents to be reviewed also includes the following steps:

[0015] S1-1: Using "procurement documents" and "tender documents" as keywords, we collected relevant documents from government websites and organized them to construct a dataset of documents to be reviewed;

[0016] S1-2: Perform data preprocessing on the text of the document dataset to be reviewed, including word segmentation, stop word filtering, and word vector encoding.

[0017] Preferably, in S2, the case library of fair competition review is constructed based on expert review and updated incrementally, and further includes the following steps:

[0018] S2-1: Ask experts to manually annotate the case texts under review, including the suspected violation clauses, the suspected violation primary standards, the suspected violation secondary standards, the suspected violation tertiary standards, and the reasons for the violation;

[0019] S2-2: Clean the data in the case library, including removing noise, processing missing values, standardizing text formats, and performing text preprocessing, such as word segmentation, removing stop words, stemming, or part-of-speech tagging.

[0020] Preferably, in S3, based on the fair competition review case library reviewed by experts obtained in step S2 and combined with the expert annotation results, the semantic similarity calculation model is trained and the parameters of the final model are saved, and the following steps are also included:

[0021] S3-1: The core of the model based on contrastive sentence vector representation framework is contrastive learning. Contrastive learning aims to better learn the representation of data by shortening the distance between similar data and increasing the distance between dissimilar data, so that it can produce better results in text matching tasks. Given a sample set

[0022] where x i and For similar samples, the in-batch negatives cross entropy loss function is used to optimize the target as follows:

[0023]

[0024] Among them, hi and h i + for xi and xi + The sentence vector representation of N is the batch size during training, sim(h i ,h i + ) is the vector h i and h i + Remainder similarity, τ is the temperature hyperparameter;

[0025] S3-2: The framework of contrast sentence vector representation includes two methods: unsupervised and supervised:

[0026] Unsupervised method: Unsupervised semantic similarity calculation is given a sentence set x i Input into the encoder twice respectively, and get vector z i Positive examples and z′ i ; Since the random dropout mask mechanism exists in the fully connected layer and attention layer of the model, the same input will produce different results after passing through the model; z′ i Vector as z i The positive example of , and other vectors as negative examples, the training objective of semantic similarity calculation becomes:

[0027]

[0028] Note: z is the standard dropout mask in Transformers, and no additional dropout is added in the unsupervised semantic similarity calculation;

[0029] Supervised method: In the case library, a premise assumption text x i , with corresponding implied text and contradictory text, taking the contradictory text as a hard negative example, that is, The data pair becomes Data set, where To contain text, For contradictory texts, the training objective of supervised learning semantic similarity calculation becomes:

[0030]

[0031] After adding contradictory text as a hard negative example, the training objective of supervised learning semantic similarity calculation becomes:

[0032]

[0033] in, Respectively The embedding obtained after the encoder, α is the weighting coefficient;

[0034] S3-3: The semantic similarity calculation model is trained with different α values, and the trained model is evaluated on the case library dataset. It is found that the effect is best when α is 1, and neutral data does not bring more benefits.

[0035] Preferably, the semantic similarity calculation of the document to be reviewed obtained by S1 in S4 with the case library document of S2 through the semantic similarity calculation pre-training model obtained by S3 also includes the following steps:

[0036] S4-1: Load the case library file obtained in S2 and the to-be-examined file obtained in S1 into the data structure respectively, and encode them to obtain embedded representation;

[0037] S4-2: Use a tokenizer to encode text pairs into input that the model can understand. This step requires tokenizing, tokenizing, and adding special tags to each data point.

[0038] S4-3: For each pair of encoded texts, use the model to calculate their embedding representations and calculate the similarity score between them;

[0039] S4-4: Use similarity scores to evaluate text similarity, and use different evaluation metrics: including precision, recall, and F1 score to evaluate model performance.

[0040] Preferably, the step S5 identifies the relevant clauses suspected of violating the law based on the semantic similarity result of S4, thereby realizing the identification and judgment of the violation type of the document to be reviewed, and further includes the following steps:

[0041] S5-1: semantic similarity scores calculated using S5. These scores represent the semantic similarity between each target sentence and the sentences in the training data.

[0042] S5-2: A threshold can be set for the semantic similarity score according to the requirements. Sentences with scores above the threshold are considered to have similar meanings to the sentences in the case base file of S2 and violate the same clauses, while sentences with scores below the threshold are considered not to have similar meanings;

[0043] S5-3: Based on the results of S5-2, identify the clauses suspected of being in violation of the regulations, the first-level standards suspected of being violated, the second-level standards suspected of being violated, the third-level standards suspected of being violated, and the reasons for the violations related to the documents to be reviewed, so as to identify and judge the types of violations in the documents to be reviewed.

[0044] (III) Beneficial effects

[0045] Compared with the prior art, the present invention provides a fair competition review method based on a comparative sentence vector representation framework, which has the following beneficial effects:

[0046] The fair competition review method based on the contrastive sentence vector representation framework includes both unsupervised and supervised methods. Based on contrastive learning theory, the text is embedded in the semantic space, so that the semantic similarity between texts can be measured, and a fair competition review model for semantic similarity calculation is obtained. The specific operation method is to use the trained semantic similarity calculation model to convert the text in the pending documents and the document case library into a semantic embedding representation (vector representation), calculate the semantic similarity between the pending documents and the cases in the case library, and identify the relevant clauses suspected of violations based on the calculated similarity scores, so as to solve the problems of low accuracy and lack of sample data in the existing fair competition review model based on the same level labels, improve the intelligence level of the review work, improve work efficiency, and effectively solve the problems of insufficient review and regulatory resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of the steps of a fair competition review method based on a comparative sentence vector representation framework proposed by the present invention;

[0048] Figure 2 A schematic diagram of a first-level label for violation types that violate the principle of fair competition proposed by the present invention;

[0049] Figure 3 A schematic diagram of the secondary and tertiary labels related to the market entry and exit primary labels in the violation types that violate the fair competition principle proposed in the present invention;

[0050] Figure 4 A schematic diagram of the unsupervised method proposed in the present invention;

[0051] Figure 5 Schematic diagram of the supervised method proposed in the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] See also Figure 1-5 , a fair competition review method based on contrastive sentence vector representation framework, including:

[0054] S1. Collect a large number of policy and measure documents and obtain the data set of documents to be reviewed, which also includes the following steps:

[0055] S1-1: Using "procurement documents" and "tender documents" as keywords, we collected relevant documents from government websites and organized them to construct a dataset of documents to be reviewed;

[0056] S1-2: Perform data preprocessing on the text of the document dataset to be reviewed, including word segmentation, stop word filtering, and word vector encoding;

[0057] S2. Building a case library for fair competition review based on expert review and updating it incrementally also includes the following steps:

[0058] S2-1: Experts are invited to manually annotate the case texts under review. The annotated content includes the suspected illegal clauses, the suspected first-level standards, the suspected second-level standards, the suspected third-level standards, and the reasons for the violations. For details, please refer to the fair competition review standards in the "Fair Competition Review Implementation Rules":

[0059] Among them, the first-level labels include four categories: market entry and exit, free flow of commodity factors, impact on production and operation costs, and impact on production and operation behavior;

[0060] The secondary tags include 5 secondary tags under the primary tag (market access and exit), 5 secondary tags under the primary tag (free flow of commodity elements), 4 secondary tags under the primary tag (affecting production and operation costs), and 4 secondary tags under the primary tag (affecting production and operation behavior). Taking the primary tag (market access and exit) as an example, Figure 3 As shown, there are 5 secondary labels under the primary label (market access and exit), which are: no unreasonable and discriminatory entry and exit conditions shall be set; no franchise rights shall be granted to operators without fair competition; no restrictions on the operation, purchase, and use of goods and services provided by specific operators; no approval or prior filing procedures without legal basis shall be set; and no approval procedures shall be set for industries, fields, and businesses outside the negative list of market access;

[0061] The third-level labels include 54 third-level labels under 18 second-level labels. Taking the second-level labels (unreasonable and discriminatory entry and exit conditions shall not be set) as an example, Figure 2As shown, there are five third-level labels under the second-level label (unreasonable and discriminatory entry and exit conditions shall not be set), which are setting entry and exit conditions that are obviously unnecessary or beyond actual needs, excluding or restricting operators from participating in market competition; implementing unreasonable differentiated treatment for operators of different ownership, regions and organizational forms without the basis of laws, administrative regulations or State Council regulations, and setting unequal market entry and exit conditions; setting or disguising market entry barriers in the form of filing, registration, catalogue, annual inspection, annual report, monitoring, identification, certification, accreditation, inspection, monitoring, review, designation, numbering, re-inspection, re-examination, renewal of licenses, requiring the establishment of branches and any other forms without the basis of laws, administrative regulations or State Council regulations; setting or disguising market exit barriers for enterprise cancellation, bankruptcy, listing transfer, relocation and transfer without the basis of laws, administrative regulations or State Council regulations; forcing or disguising enterprises to transfer technology by administrative licensing, administrative inspection, administrative punishment and administrative enforcement, and setting or disguising market entry and exit barriers;

[0062] S2-2: Clean the data in the case library, including removing noise, processing missing values, standardizing text formats, and performing text preprocessing, such as word segmentation, removing stop words, stemming, or part-of-speech tagging;

[0063] S3. Based on the fair competition review case library reviewed by experts obtained in step S2 and combined with the expert annotation results, the semantic similarity calculation model is trained and the parameters of the final model are saved, which also includes the following steps:

[0064] S3-1: The core of the model based on contrastive sentence vector representation framework is contrastive learning. Contrastive learning aims to better learn the representation of data by shortening the distance between similar data and increasing the distance between dissimilar data, so that it can produce better results in text matching tasks. Given a sample set

[0065] where x i and For similar samples, the in-batch negatives (negative samples within a batch) cross entropy loss function is used to optimize the target as follows:

[0066]

[0067] Among them, h i and h i + for xi and xi + The sentence vector representation of N is the batch size during training, sim(h i ,h i + ) is the vector h iand h i + Remainder similarity, τ is the temperature hyperparameter;

[0068] τ is a hyperparameter that controls the distribution of the softmax function, usually set to 0.05 (the larger the τ, the smoother the distribution; the smaller the τ, the greater the gap between positive and negative samples), which can speed up convergence. The sim function represents the cosine similarity calculation;

[0069] S3-2: The framework of contrast sentence vector representation includes two methods: unsupervised and supervised:

[0070] Unsupervised method: The idea of ​​unsupervised semantic similarity calculation is to give a sentence set x i Input into the encoder twice respectively, and get vector z i Positive examples and z′ i ; Since the random dropout mask mechanism exists in the fully connected layer and attention layer of the model, the same input will produce different results after passing through the model; z′ i Vector as z i The positive example of , and other vectors as negative examples, the training objective of semantic similarity calculation becomes:

[0071]

[0072] Note: z is the standard dropout mask in Transformers, and no additional dropout is added in the unsupervised semantic similarity calculation; z′ i Indicates z i Enhanced samples (obtained with different dropouts);

[0073] Supervised method: In the case library, a premise assumption text x i , with corresponding implied text and contradictory text, taking the contradictory text as a hard negative example, that is, The data pair becomes Data set, where To contain text, For contradictory texts, the training objective of supervised learning semantic similarity calculation becomes:

[0074]

[0075] After adding contradictory text as a hard negative example, the effect is improved. Intuitively, it is beneficial to distinguish hard negative examples (contradictory text) from other negative examples in the batch. The training objective of supervised learning semantic similarity calculation becomes:

[0076]

[0077] in, Respectively The embedding (embedded representation / sentence vector) obtained after the encoder, α is the weighting coefficient, which is used to distinguish different types of negative samples (especially hard negative samples, i.e. hard negatives). The formula adjusts the weight of negative samples by introducing , so that the model pays more attention to these hard negative samples. Specifically, the goal of the formula is to optimize the model through the cross entropy loss function, where α is used to control the importance of hard negative samples relative to other negative samples. In other words, when the α value is large, the contribution of hard negative samples to the loss is also greater, thereby guiding the model to pay more attention to distinguishing these samples. The purpose of this method is to improve the model's discriminative ability, especially when faced with difficult-to-distinguish negative samples.

[0078] S3-3: The semantic similarity calculation model is trained with different α values, and the trained model is evaluated on the case library data set. It is found that the effect is optimal when α is 1, and neutral data does not bring more benefits;

[0079] S4. The semantic similarity calculation model obtained in S3 is used to calculate the semantic similarity between the document to be reviewed obtained in S1 and the case library document in S2, and the following steps are also included:

[0080] S4-1: Load the case library file obtained in S2 and the to-be-examined file obtained in S1 into the data structure respectively, and encode them to obtain embedded representation;

[0081] S4-2: Use a tokenizer to encode text pairs into input that the model can understand. This step requires tokenizing, tokenizing, and adding special tags to each data point.

[0082] S4-3: For each pair of encoded texts, use the model to calculate their embedding representations and calculate the similarity score between them;

[0083] S4-4: Use similarity scores to evaluate text similarity. Different evaluation indicators, such as accuracy, recall, and F1 score, can also be used to evaluate model performance.

[0084] S5. Based on the semantic similarity results of S4, the related suspected illegal clauses are identified to realize the identification and judgment of the illegal type of the document to be reviewed, and the following steps are also included:

[0085] S5-1: semantic similarity scores calculated using S5. These scores represent the semantic similarity between each target sentence and the sentences in the training data.

[0086] S5-2: A threshold can be set for the semantic similarity score according to the requirements. Sentences with scores above the threshold are considered to have similar meanings to the sentences in the case base file of S2 and violate the same clauses, while sentences with scores below the threshold are considered not to have similar meanings;

[0087] S5-3: Based on the results of S5-2, identify the clauses suspected of being in violation of the regulations, the first-level standards suspected of being violated, the second-level standards suspected of being violated, the third-level standards suspected of being violated, and the reasons for the violations related to the documents to be reviewed, so as to identify and judge the types of violations in the documents to be reviewed.

[0088] In summary, the fair competition review method based on the contrastive sentence vector representation framework includes both unsupervised and supervised methods. Based on the contrastive learning theory, the text is embedded in the semantic space, so that the semantic similarity between texts can be measured, and a fair competition review model for semantic similarity calculation is obtained. The specific operation method is to use the trained semantic similarity calculation model to convert the text in the pending documents and the document case library into a semantic embedding representation (vector representation), and calculate the semantic similarity between the pending documents and the cases in the case library.

[0089] It should be noted that the term "comprises" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article, or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0090] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A fair competition review method based on a contrastive sentence vector representation framework, characterized in that: include: S1. Collect a large number of policy and measure documents and obtain the data set of documents to be reviewed; S2. Build a case library for fair competition review based on expert review and update it incrementally; S3. Based on the fair competition review case library reviewed by experts obtained in step S2 and combined with the expert annotation results, the semantic similarity calculation model is trained and the parameters of the final model are saved; S4. The document to be reviewed obtained by S1 is used to calculate the semantic similarity with the case library document in S2 through the semantic similarity calculation model obtained by S3; S5. Based on the semantic similarity results of S4, the relevant clauses suspected of violations are identified, so as to identify and judge the violation types of the documents to be reviewed.

2. According to claim 1, a fair competition review method based on a contrastive sentence vector representation framework is characterized by: S1 collects a large number of policy and measure documents to obtain a data set of documents to be reviewed, and also includes the following steps: S1-1: Using "procurement documents" and "bidding documents" as keywords, we collected relevant documents from government websites and organized them to construct a dataset of documents to be reviewed; S1-2: Perform data preprocessing on the text of the document dataset to be reviewed, including word segmentation, stop word filtering, and word vector encoding.

3. According to claim 1, a fair competition review method based on a contrastive sentence vector representation framework is characterized by: The S2 constructs a case library for fair competition review based on expert review and updates it incrementally, and also includes the following steps: S2-1: Ask experts to manually annotate the case texts under review, including the suspected violation clauses, the suspected violation primary standards, the suspected violation secondary standards, the suspected violation tertiary standards, and the reasons for the violation; S2-2: Clean the data in the case library, including removing noise, processing missing values, standardizing text formats, and performing text preprocessing, including word segmentation, removing stop words, stemming, or part-of-speech tagging.

4. The fair competition review method based on the contrastive sentence vector representation framework according to claim 1 is characterized by: The S3 further includes the following steps: training the semantic similarity calculation model based on the fair competition review case library reviewed by experts obtained in step S2 and combining the expert annotation results, and saving the parameters of the final model: S3-1: The core of the model based on contrastive sentence vector representation framework is contrastive learning. Contrastive learning aims to better learn the representation of data by shortening the distance between similar data and increasing the distance between dissimilar data, so that it can produce better results in text matching tasks. Given a sample set where x i and For similar samples, the in-batch negatives cross entropy loss function is used to optimize the target as follows: Among them, h i and h i + for xi and xi + The sentence vector representation of N is the batch size during training, sim(h i ,h i + ) is the vector h i and h i + Remainder similarity, τ is the temperature hyperparameter; S3-2: The framework of contrast sentence vector representation includes two methods: unsupervised and supervised: Unsupervised method: Unsupervised semantic similarity calculation is given a sentence set x i Input into the encoder twice respectively, and get vector z i Positive examples and z′ i ; Since the random dropout mask mechanism exists in the fully connected layer and attention layer of the model, the same input will produce different results after passing through the model; z′ i Vector as z i The positive example of , and other vectors as negative examples, the training objective of semantic similarity calculation becomes: Note: z is the standard dropout mask in Transformers, and no additional dropout is added in the unsupervised semantic similarity calculation; Supervised method: In the case library, a premise assumption text x i , with corresponding implied text and contradictory text, taking the contradictory text as a hard negative example, that is, The data pair becomes Data set, where To contain text, For contradictory texts, the training objective of supervised learning semantic similarity calculation becomes: After adding contradictory text as a hard negative example, the training objective of supervised learning semantic similarity calculation becomes: in, Respectively The embedding obtained after the encoder, α is the weighting coefficient; S3-3: The semantic similarity calculation model is trained with different α values, and the trained model is evaluated on the case library dataset. It is found that the effect is best when α is 1, and neutral data does not bring more benefits.

5. The fair competition review method based on the contrastive sentence vector representation framework according to claim 1 is characterized by: The semantic similarity calculation of the document to be reviewed obtained by S1 in S4 is performed by using the semantic similarity calculation pre-training model obtained by S3 and the semantic similarity calculation of the case library document in S2, and further includes the following steps: S4-1: Load the case library file obtained in S2 and the to-be-examined file obtained in S1 into the data structure respectively, and encode them to obtain embedded representation; S4-2: Use a tokenizer to encode text as input for model understanding. This step requires tokenizing, tokenizing, and adding special tags to each data point. S4-3: For each pair of encoded texts, use the model to calculate their embedding representations and calculate the similarity score between them; S4-4: Use similarity scores to evaluate text similarity, and use different evaluation metrics: including precision, recall, and F1 score to evaluate model performance.

6. According to claim 1, a fair competition review method based on a contrastive sentence vector representation framework is characterized by: The S5 identifies the relevant clauses suspected of violating the law based on the semantic similarity result of S4, thereby realizing the identification and judgment of the violation type of the document to be reviewed, and further includes the following steps: S5-1: semantic similarity scores calculated using S5. These scores represent the semantic similarity between each target sentence and the sentences in the training data. S5-2: Set a threshold for the semantic similarity score according to the requirements. Sentences with scores above the threshold are considered to have similar meanings to the sentences in the case base file of S2 and violate the same clauses, while sentences with scores below the threshold are considered not to have similar meanings; S5-3: Based on the results of S5-2, identify the clauses suspected of being in violation of the regulations, the first-level standards suspected of being violated, the second-level standards suspected of being violated, the third-level standards suspected of being violated, and the reasons for the violations related to the documents to be reviewed, so as to identify and judge the types of violations in the documents to be reviewed.

Images