Complex text similarity calculation method applied to social governance scenarios

Abstract

The application discloses a complex text similarity calculation method applied to a social governance scene, and in view of the problems that data uniformly reported to a platform by grassroots governance staff are similar in format and have a high event background proportion, a method that event background and complete text are first separated and extracted and then similarity is calculated and a difficult negative sample pair is constructed is provided, so that the problem that different events with a too large event background proportion and the same event background cannot be distinguished is solved.For data autonomously reported by the public, there are problems that information is largely redundant and the description of the same event greatly deviates, and the application provides a method that a word repetition algorithm is used to construct a difficult positive sample pair, so that the problem of matching long and short similar texts is solved.After the application is used, the accuracy of the similarity algorithm is greatly improved, and in particular, the accuracy of difficult samples is significantly improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of natural language processing, and particularly relates to a complex text similarity calculation method applied to a social governance scene. BACKGROUND

[0002] With the development of social economy and the awakening of people's rights consciousness, in recent years, grassroots social contradictions have shown new trends, new directions and new characteristics; the contradiction dispute mediation work needs to start from preventing "civil to criminal" and reducing the repeat rate of high-frequency disputes, so it is necessary to build four intelligent early warning models of "key people, key events, key industries and key areas" for contradiction risk; obviously, the "key event" early warning model is difficult to build, and artificial intelligence algorithm is needed to judge whether two event descriptions are similar.

[0003] Currently, the method for calculating text similarity mainly uses pre-training models based on large-scale corpus, but the contradiction mediation data is large and the data sources are complex, part of the data is uniformly reported to the platform by grassroots governance workers, and there are characteristics such as similar format and high event background proportion; part of the data comes from the self-reporting of the public, and there are characteristics such as a large amount of redundant information and a large difference in the description of the same event. Using pre-training models based on large-scale corpus will have the following problems: slow matching between events, inability to match the same events with different text lengths, and difficulty in distinguishing different events with the same event background.

[0004] Chinese patent application with publication number CN115203365A provides a social event processing method applied to the field of comprehensive governance, which calculates the similarity of input events and events in the library through the RE2 model to assist in classifying events; but this patent technology does not take into account the complexity of input data, and when the input event format is similar and the event background proportion is high, this method is not effective.

[0005] Chinese patent application with publication number CN115982324A provides a procurement file inspection method based on improved natural language processing, which calculates the similarity of sentences in the input file and sentences in the database file through the sentence-bert model to determine whether the input file is compliant, but this method can only be used in the case of similar sentence length, and the effect is not good when facing the case of different sample pair lengths. SUMMARY

[0006] In view of the above, the application provides a complex text similarity calculation method applied to a social governance scene, which solves the problem of matching similar texts with different lengths by repeating words for length-approximate positive samples, and solves the problem of differentiating different events with the same event background and a large event background proportion by constructing event background same event pairs through an entity recognition algorithm.

[0007] A method for calculating the similarity of complex texts in a social governance scenario includes the following steps:

[0008] (1) Obtain a large amount of text similarity training data and entity recognition training data;

[0009] (2) Obtain the pre-trained model for entity recognition and the pre-trained model for text similarity;

[0010] (3) Fine-tune the entity recognition pre-trained model using entity recognition training data to obtain the entity recognition model;

[0011] (4) For positive sample pairs with similar lengths in the text similarity training data, use the word repetition algorithm to reconstruct these positive sample pairs so that their sentence pairs have different lengths.

[0012] (5) Perform data augmentation on negative sample pairs in the text similarity training data;

[0013] (6) Use the text similarity training data after steps (4) and (5) to fine-tune the text similarity pre-trained model to obtain the text similarity model;

[0014] (7) Access data through the conflict mediation platform, use the text similarity model to encode each accessed data to obtain the corresponding sentence vector and form event data to store in the corpus;

[0015] (8) Receive conflict mediation data, calculate the similarity between the conflict mediation data and the sentence vectors of event data in the corpus, and synchronize the calculation results to the event monitoring database to provide early warning for the corresponding events.

[0016] Furthermore, the text similarity training data consists of sentence pairs labeled with whether they are similar or not, and the entity recognition training data consists of sentences with sequence labels.

[0017] Furthermore, the text similarity pre-training model is paraphrase-multilingual-MiniLM-L12-v2; the entity recognition pre-training model is chinese_pretrain_mrc_roberta_wwm_ext_large.

[0018] Further, the specific implementation of step (4) is as follows: First, select positive sample pairs with approximately the same length from the text similarity training data. The positive sample pairs are sentence pairs with similar text. For each positive sample pair with approximately the same length, arbitrarily select one sentence and use the jieba word segmentation method to divide the sentence into n words {W0, W1, ..., W...}. n-1} and then use the random function to select n / 4 numbers {X1, X2, …, X n / 4} from 0 to n-1, and finally copy {W X1 ,W X2 ,…,W Xn / 4} and insert them into the original sentence after the corresponding words.

[0019] Further, the specific implementation of step (5) is as follows: first, randomly select N sentences from the text similarity training data to form a queue, and use the entity recognition model to extract the event background text and event content text of each sentence in the queue; then randomly select a sentence s from the queue, splice its event background text with the event content text of the remaining N-1 sentences in the queue to obtain N-1 new sentences, and finally combine sentence s with the N-1 new sentences to form N-1 negative sample pairs, where N is a natural number greater than 1.

[0020] Further, the specific implementation of step (7) is as follows: for any access data, first extract its event background text using the entity recognition model, the access data being text describing a contradiction dispute event, then encode the event background text and the complete text of the access data using the text similarity model to correspondingly generate two sentence vectors and form event data for storage in the corpus, each group of event data in the corpus containing event number, event complete text, event background text, complete text sentence vector, and background text sentence vector.

[0021] Further, the specific process of similarity calculation in step (8) is as follows: first, extract the event background text of the contradiction mediation data using the entity recognition model, then encode the event background text and the complete text of the contradiction mediation data using the text similarity model to correspondingly generate two sentence vectors code1 and code2; for any event data in the corpus, calculate the similarity between the background text sentence vector of the event data and code1, if the similarity reaches a set threshold, further calculate the similarity between the complete text sentence vector of the event data and code2, if the similarity also reaches the set threshold, then determine that the event data is similar to the contradiction mediation data.

[0022] Further, the event monitoring database in step (8) contains the field information of all existing events, including event number, event content, reporting time, reporting source, repetition number, and warning level; when the received contradiction mediation data is similar to the data of an existing event, the repetition number of the corresponding event is increased by one, and then all existing events are classified and warned according to the repetition number.

[0023] The present application is directed to the problem that the data reported by grassroots governance workers to the platform is similar in format and has a high proportion of event background, and proposes a method of separating and extracting event background and complete text first, then calculating similarity and constructing difficult negative sample pairs, which solves the problem of different events with the same event background and a high proportion of event background. For the data reported by the public, there is a lot of redundant information, and the description of the same event is different. The present application proposes a method of using word repetition algorithm to construct difficult positive sample pairs, which solves the problem of matching similar texts of different lengths. After using the present application, the accuracy of the similarity algorithm is greatly improved, especially the accuracy of difficult samples is significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The implementation flowchart of the model training and corpus construction of the present application.

[0025] Figure 2 The implementation flowchart of the text similarity calculation of the present application. DETAILED DESCRIPTION

[0026] In order to describe the present application more specifically, the technical solutions of the present application will be described in detail below in combination with the drawings and specific embodiments.

[0027] The complex text similarity calculation method applied to the field of social governance of the present application includes two stages:

[0028] Stage one: model training and corpus construction.

[0029] As shown in Figure 1 , the specific implementation steps of model training and corpus construction are as follows:

[0030] S1: Use the Bert-whitening model to encode the database text into sentence vectors and store them in the vector database faiss. Use the fast matching function of faiss to find the top 5 similar data of each data to form a pre-labeled positive sample pair. Take 2 data randomly for several times to form a pre-labeled negative sample pair. Then, judge whether the sample pair label is correct by manual operation. Screen out the correct sample pairs to form a text similarity training data set. Use the doccano labeling platform to pre-label the event background entities in the entity recognition data, and then correct the sequence labeling of the entity background entities by manual operation to form an entity recognition training data set.

[0031] S2: Call the python script to use the AutoModel.from_pretrained() and model.save() methods of the transformer framework to download the text similarity pre-training model and the entity recognition pre-training model from huggingface and save them to the local.

[0032] In this embodiment, the text similarity pre-training model is paraphrase-multilingual-MiniLM-L12-v2, and the entity recognition pre-training model is chinese_pretrain_mrc_roberta_wwm_ext_large.

[0033] S3: Fine-tune the entity recognition pre-training model paraphrase-multilingual-MiniLM-L12-v2 using the DeepKE framework on a single-card 4090 server, and the main hyperparameters are {batchsize=8, maxlenth=128, epoch=5, learning rate=0.0001, optimizer=Adamw}. The entity recognition training data set has a scale of 5000 data, and the best checkpoint after training is selected as the entity recognition model for subsequent use.

[0034] S4: Select a pair of positive samples with similar lengths in the text similarity training data, and use the word repetition algorithm to form a pair of positive samples with different lengths for one of the sentences.

[0035] In this embodiment, the word repetition algorithm is specifically: first, divide the sentence into n words {W0, W1…, Wn-1} using jieba segmentation, then use the random function to select n / 4 numbers {X1, X2…, Xn / 4} from 0 to n-1, and copy and fill {W n-1 ,W n / 4 …,W X1 ,W X2 …,W Xn / 4} into the original sentence; for example: abcdefg-->aabccdeefg, where a, c, and e are randomly selected words.

[0036] S5: Randomly select N sentences to form a candidate queue, extract the event context and event content of each sentence in the queue using the entity recognition model in S3, randomly select one sentence, and combine its event context with the event content of the remaining N-1 sentences to form N-1 negative sample pairs.

[0037] S6: Fine-tune the text similarity pre-training model using the text similarity training data and the data generated in S4 and S5 on a single-card 4090 server, and the main hyperparameters are {batchsize=4, maxlenth=128, epoch=3, learning rate=warmup(constant), optimizer=Adamw}.

[0038] S7: Access the contradiction mediation event data from the data interface of the contradiction mediation platform, encode each piece of contradiction mediation event data using the model trained in S6, and then store the generated sentence vector in the corpus database.

[0039] The contradiction mediation data includes data in various fields, specifically citizen hotline, grassroots governance four platforms, and police service and police situation. The generated sentence vector refers to two different sentence vectors generated by encoding the event background and complete event text respectively. The database fields used include event number, event complete text, event background text, event complete text encoding, and event background encoding.

[0040] Phase two: text similarity calculation.

[0041] As shown in Figure 2 the specific implementation steps of text similarity calculation are as follows:

[0042] Step 1: Integrate citizen hotline, grassroots governance four platforms, and police service and police situation platform information, and collect them in the contradiction mediation platform. At the same time, realize the data docking of the contradiction mediation platform and each channel, ensure the real-time update and sharing of information, call the interface of the contradiction mediation platform, and access the data to the local database.

[0043] The accessed data includes the street where the contradiction dispute event occurred, the contradiction dispute event reporting time, the contradiction dispute event type, the text describing the contradiction dispute event, and the contradiction dispute event party information, etc. Among them, the text describing the contradiction dispute event contains event background (event location, event time, etc.) and event content.

[0044] Step 2: Extract the event background in the contradiction dispute event text using the entity recognition model trained in phase one, input the event background and complete text into the text similarity model trained in phase one for encoding, respectively obtaining code1 and code2; calculate the similarity between the event background sentence vector of each data in the corpus database and code1.

[0045] Here, the similarity is cosine similarity, which can also be adjusted to other similarity calculation methods (such as Euclidean distance, Manhattan distance, geodesic distance) according to different requirements. If the similarity of the event background reaches the set threshold value, calculate the similarity between the event complete text sentence vector of the data and code2, if it also reaches the set threshold value, then the two events are considered as similar events. The threshold value is an arbitrary value between 0 and 1, which can be adjusted according to requirements.

[0046] Step 3: The results of the algorithm model are synchronized to the event monitoring database, and if the input event is similar to an existing event, the number of repetitions of the corresponding event is increased by one, wherein the event monitoring database contains the fields of event number, event content, reporting time, reporting source, repetition number, and warning level. The purpose of this step is to count the same or similar events reported by different sources in order to subsequently perform warning analysis.

[0047] Step 4: The data in the event monitoring database are input into the "key event" warning algorithm, and the repeated reported events are warned according to the set rules, wherein the rules are: events with a number of repetitions higher than 5 are set as red warning events; among the remaining events, events with a number of repetitions higher than 3 are selected, and if the initial reporting time is within the last 3 months, the events are set as red warning events, and if the number of repetitions is higher than 3 but the initial reporting time is out of the range of 3 months, the events are set as yellow warning events. The purpose of this step is to identify events that may cause significant impact or crisis and perform different levels of warning according to the urgency and severity.

[0048] The above example is the processing flow of a single data, and multiple data can be processed in parallel.

[0049] In the following, the application is compared with Sentence-Bert in the training of 10,000 training data, and the evaluation indexes are cosine Pearson coefficient and difficult sample accuracy. As shown in Table 1, the cosine Pearson coefficient of the application is improved by 10 percentage points compared with the basic method Sentence-Bert, and the performance on difficult samples is almost improved by 20 percentage points. It can be seen that the accuracy of the application is greatly improved, especially the accuracy of difficult samples is significantly improved.

[0050] Table 1

[0051] Method Training data Cosine Pearson coefficient Hard sample accuracy Sentence-Bert 10000 73.58 62.84 The invention 10000 83.79 80.31

[0052] The above description of the embodiments is for the purpose of facilitating those skilled in the art to understand and apply the application, and those skilled in the art can easily make various modifications to the above embodiments, and apply the general principles described herein to other embodiments without creative labor. Therefore, the application is not limited to the above embodiments, and the improvements and modifications of the application made by those skilled in the art according to the disclosure of the application should be within the protection scope of the application.

Claims

1. A method for calculating the similarity of complex texts in a social governance scenario, comprising the following steps: (1) Obtain a large amount of text similarity training data and entity recognition training data; (2) Obtain the pre-trained model for entity recognition and the pre-trained model for text similarity; (3) Fine-tune the entity recognition pre-trained model using entity recognition training data to obtain the entity recognition model; (4) For positive sample pairs with similar lengths in the text similarity training data, use the word repetition algorithm to reconstruct these positive sample pairs so that their sentence pairs have different lengths. (5) Perform data augmentation on negative sample pairs in the text similarity training data; (6) Use the text similarity training data after steps (4) and (5) to fine-tune the text similarity pre-trained model to obtain the text similarity model; (7) Access data through the conflict mediation platform, use the text similarity model to encode each accessed data to obtain the corresponding sentence vector and form event data to store in the corpus; For any access data, firstly, the event background text is extracted using an entity recognition model. The access data is text describing a conflict or dispute event. Then, the event background text and the complete text of the access data are encoded using a text similarity model to generate two sentence vectors and form event data, which are stored in a corpus. Each set of event data in the corpus includes an event number, the complete event text, the event background text, the complete text sentence vector, and the background text sentence vector. (8) Receive conflict mediation data, calculate the similarity between the conflict mediation data and the sentence vectors of event data in the corpus, and synchronize the calculation results to the event monitoring database to provide early warning for the corresponding events.

2. The method for calculating the similarity of complex texts according to claim 1, characterized in that: The text similarity training data consists of sentence pairs labeled with whether they are similar or not, and the entity recognition training data consists of sentences with sequence labels.

3. The method for calculating the similarity of complex texts according to claim 1, characterized in that: The text similarity pre-training model is paraphrase-multilingual-MiniLM-L12-v2; the entity recognition pre-training model is chinese_pretrain_mrc_roberta_wwm_ext_large.

4. The method for calculating the similarity of complex texts according to claim 1, characterized in that: The specific implementation of step (4) is as follows: First, select positive sample pairs with similar lengths from the text similarity training data. The positive sample pairs are sentence pairs with similar texts. For each positive sample pair with similar lengths, arbitrarily select one sentence and use the jieba word segmentation method to divide the sentence into n words {W0, W1, ..., W...}. n-1 Then, use the random function to select n / 4 numbers {X1, X2, ..., Xn} from 0 to n-1. n / 4 }, finally {W X1 W X2 ,…,W Xn / 4 Copy and insert it after the corresponding word in the original sentence.

5. The method for calculating the similarity of complex texts according to claim 1, characterized in that: The specific implementation of step (5) is as follows: First, N sentences are randomly selected from the text similarity training data to form a queue. The event background text and event content text of each sentence in the queue are extracted using the entity recognition model. Then, a sentence s is randomly selected from the queue, and its event background text is concatenated with the event content text of the remaining N-1 sentences in the queue to obtain N-1 new sentences. Finally, sentence s is combined with these N-1 new sentences to form N-1 negative sample pairs, where N is a natural number greater than 1.

6. The method for calculating the similarity of complex texts according to claim 1, characterized in that: The specific process of similarity calculation in step (8) is as follows: First, the event background text of the conflict mediation data is extracted using an entity recognition model. Then, the event background text and the complete text of the conflict mediation data are encoded using a text similarity model to generate two sentence vectors, code1 and code2. For any event data in the corpus, the similarity between the background text sentence vector of the event data and code1 is calculated. If the similarity reaches a set threshold, the similarity between the complete text sentence vector of the event data and code2 is further calculated. If the similarity also reaches the set threshold, the event data is determined to be similar to the conflict mediation data.

7. The method for calculating the similarity of complex texts according to claim 1, characterized in that: The event monitoring database in step (8) contains field information for all existing events, including event number, event content, reporting time, reporting source, number of repetitions, and warning level. When the received conflict mediation data is similar to the data of an existing event, the number of repetitions of the corresponding event is increased by one, and then all existing events are given graded warnings based on the number of repetitions.

Images