Method and device for extracting and clustering information of appeal cases based on public opinion big data

Abstract

The application provides a public opinion big data-based appeal case information extraction and clustering method and device. The method comprises steps 1 to 7. The application can extract the key information of the public appeal case in the public opinion big data, and through a matching model based on deep learning and an unsupervised clustering method, the cases can be quickly merged and clustered on a large-scale appeal data set.

Description

Technical Field

[0001] The embodiments of the present invention relate to the fields of public opinion analysis and natural language processing technology, and in particular to a method and device for extracting and clustering information on appeal cases based on public opinion big data. Background Technology

[0002] Public opinion big data is an important bottom-up social perception data source that can directly reflect public sentiment and opinions on events, possessing enormous potential application value. While increasing and refining classification categories can cover more issues, many repetitive issues within each category still require individual verification, significantly increasing unnecessary workload. Furthermore, it's impossible to know the specific targets and locations of complaints within each category, hindering comprehensive perception. On the other hand, emergencies are accompanied by a rapid stream of public demands, often sharing common locations, targets, or content. A method to efficiently identify and merge such events and process them uniformly alleviates the pressure of handling emergencies. To address the aforementioned issues in processing public opinion big data, the first step is to extract entity information from the demands. Entity information contains the constituent elements of a case, indicating characteristics that distinguish it from other cases. Case information extraction is considered a named entity recognition problem in natural language processing. Traditional machine learning-based corpora suffer from insufficient sample size when identifying and extracting text from specific domains. Currently, with the development of deep learning pre-trained language models, the strong generalization performance of these models has alleviated this problem to some extent. However, their generalization still depends on the corpus used during pre-training. Therefore, solutions include extraction methods for small sample sizes and continuous accumulation of datasets in the domain through rapid iteration. After extracting entity information from cases, the next step is to merge identical entity information from different cases into the same category, thus associating multiple cases. Generally, there are two methods to determine whether two texts are semantically related: representational matching and interactive matching. Representational models convert each text into a vector and use the vector distance as the similarity level. Their advantage is speed, but the accuracy is greatly affected by the threshold used to determine similarity. Interactive models directly determine whether two texts are similar, resulting in higher accuracy. However, their disadvantage is the high time complexity of pairwise matching with massive amounts of data. These two methods complement each other, and a combination of both methods can be used to achieve pairwise discrimination of key information. Ideally, key information would be treated as nodes, similarity as edges between nodes, and the connected graph would be extracted as the merged result. However, due to errors in model discrimination, incorrect connections can occur between nodes of different categories, leading to the accumulation and propagation of errors. Therefore, aggregating identical information from various cases without a defined number of categories remains a challenge. On the other hand, to protect complainants' privacy, the use of third-party services and interfaces should be avoided to prevent information leakage. For example, when merging identical addresses, geocoding services provided by map service providers can be used to convert addresses to latitude and longitude for aggregation, but this would leak user address information. Furthermore, geocoding methods can only determine the spatial location of an address and offer limited assistance in merging other information.Semantic-based discrimination methods can differentiate at different granularities of addresses, such as disregarding the user's specific house number, thus protecting user privacy to some extent. In summary, when processing public opinion big data, it is still not possible to effectively and securely cluster entity information extracted from cases. Therefore, developing a method and device for extracting and clustering complaint case information based on public opinion big data, which can effectively overcome the shortcomings of the aforementioned related technologies, has become a pressing technical problem for the industry. Summary of the Invention

[0003] To address the aforementioned problems in existing technologies, embodiments of the present invention provide a method and device for extracting and clustering information on public opinion cases based on big data.

[0004] In a first aspect, embodiments of the present invention provide a method for extracting and clustering information from public opinion cases based on big data, comprising: Step 1: Constructing an entity information extraction corpus, collecting data on public opinion cases to be processed, determining the entity objects and type ranges of the information to be extracted, including addresses, companies, and points of interest, obtaining a small number of samples through manual annotation to form an entity information extraction corpus, pre-extracting a portion of the results through the information extraction methods in Steps 2 and 3 to verify whether the determined entity objects are reasonable and accurate, and after verification, merging the small number of manually annotated samples with the results automatically extracted by the model as the extraction corpus; Step 2: Constructing an entity information extraction model, utilizing an ERN-based model... The pre-trained language model of IE is used as the encoder, followed by a pointer network to extract entity objects, thus constructing an entity information extraction model. The output of the pointer network is the index of the start and end positions of the extracted entity in the text. The extracted entity is extracted from the text using the index. Step 3: Train the entity information extraction model and extract entity information from the case, generating the dataset required for training the model. Based on the corpus constructed in Step 1, a training set and a validation set are randomly generated according to a certain ratio. In the training set, positive and negative samples are generated according to a certain ratio for model training. The trained model is used to extract the case text to be extracted. If there are multiple entities to be extracted, each entity is processed separately. Step 4: Construct an entity information synonym corpus. Select a portion of the text data to be matched, i.e., the entity information extracted in Step 3. Obtain a small number of synonym samples through manual annotation to form an entity information synonym corpus. Synonyms refer to samples in each category that semantically point to the same entity. If two samples are synonyms, they are said to match each other. A portion of the results are pre-generated using the information clustering method in Step 7 as a reference. Some incorrect results from the model clustering are added to the manually annotated samples. The entity information synonym corpus is generated through multiple iterations. Step 5: Construct an entity information matching model. In order to match the massive amount of case data... To quickly obtain matching results from the data, the matching model consists of a representation-based matching network and an interactive matching network. The representation-based matching network is used to quickly recall the top N results in the dataset that are similar to the sample to be matched. This matching network uses a pre-trained language model based on ERNIE as the encoder, followed by a fully connected layer for vector dimensionality reduction. The interactive matching network is responsible for determining whether the sample matches the top N recalled results. This interactive matching network uses a pre-trained language model based on ERNIE as the encoder, followed by a fully connected layer as the classifier, transforming the determination of whether two texts match into a binary classification problem and outputting the result.Step 6: Train the entity information matching model and generate the dataset required for the model. Based on the corpus constructed in Step 4, randomly divide it proportionally to generate training and validation sets. In the training set, generate positive and negative samples proportionally for model training. Since there are relatively few samples with synonym relationships in the dataset, which may cause a long-tail problem, data augmentation is used to generate positive samples as the training set. Step 7: Case entity information matching and clustering. Based on the samples in the synonym corpus, match the dataset to be matched. Successful matches are used as the entity clustering results. Use the model for matching, and use a representational matching network to extract the vector representations of all samples in the dataset. Then, build a kd-tree index for all vectors. Fast matching involves recalling the most similar samples from the top N datasets for any given text to be matched, using cosine distance as the similarity metric. An interactive matching network is then used to pairwise match the text with each of the N samples, determining if a match exists. The matching probability output by the interactive matching network is used as the distance between each pair of texts. DBSCAN clustering is then performed to obtain the merged entity information of the matched cases. This model-matched result is then merged with the results from a thesaurus. Results where the model failed to match are manually added to the final result. The frequency of occurrence of each entity in the result is counted. Finally, this result is merged with the total corpus to increase the number of matching samples in the corpus.

[0005] Based on the above method embodiments, the method for extracting and clustering information on public opinion big data provided in this embodiment of the invention includes the following steps in determining the entity objects and type range of the information to be extracted: by labeling a small number of samples, defining the information to be extracted and listing examples in different contexts for the labelers to refer to, and determining the type range of the information to be extracted by formulating labeling specifications, verifying the reasonableness of the extracted results through the model, and if the verification is incorrect, adding the incorrectly extracted samples to the corpus through verification. Among them, the type of the sample that fails to identify the entity type is corrected. If it does not belong to any type after correction, it is added to the corpus as a negative sample.

[0006] Based on the above method embodiments, the present invention provides a method for extracting and clustering information on public opinion big data for cases. In step 2, the pre-trained language model based on deep learning uses the model structure and parameters of UIE in ERNIE. The pointer network consists of two identical parts that predict the start and end positions of the entities to be extracted in the text. This includes: if the maximum length of the text is L, the text is truncated if the length exceeds L. After a piece of text passes through the encoder, an L*D dimensional vector is obtained. After passing through a fully connected layer, an L*1 dimensional vector is obtained. Finally, a sigmoid activation function is used to obtain a vector of length L, where each value is the probability of the index being the start position of the entity to be extracted. The final result is the index of the maximum probability, and the index of the end position is obtained. The information text to be extracted is obtained by truncating the text between the start and end positions.

[0007] Based on the above method embodiments, the present invention provides a method for extracting and clustering information on public opinion big data for cases. In step 3, the positive samples generated in the training set contain case text and the entity types and indices present therein, while the negative samples refer to case text and the entity types and empty indices that do not exist therein. The number of negative samples generated is related to the number of positive samples, with N being the ratio of the number of negative samples to the number of positive samples. The role of the negative samples is to indicate the entity types that do not exist in the text, which is used by the model to learn the features of different types. The case text is formatted as [CLS]Prompt[SEP]Text[SEP] and its identifier is processed, where [CLS] is used as the separator for the classification task, [SEP] is the separator to distinguish between two sentences, Prompt is the entity type, and Text is the case text. The training objective is the starting position of the corresponding entity type, and the loss function is the binary cross-entropy loss. After step 2, the vector output by the sigmoid activation function of the model predicting the starting position is X = {x1, x2, ..., x...} N}, then Y = {y1, y2, ..., y} N} represents the real label, where y exists. i =1 indicates that the text at index i is the starting position of this entity type. Therefore, the loss function includes:

[0008]

[0009] l n =y n logx n +(1-y n log(1-x) n )

[0010] Where lossBCE is the loss function at the starting position, ln is the intermediate variable, xn is the nth element of the vector X output by the sigmoid activation function, yn is the nth element of the true label Y, and N is the number of elements. After obtaining the loss function at the ending position, the loss function at the starting position and the loss function at the ending position are averaged to obtain the final loss function.

[0011] Based on the above method embodiments, the present invention provides a method for extracting and clustering information on public opinion big data-based appeal cases. In step 4, each piece of data in the corpus contains one or more semantically identical entity information, and each piece of data is semantically mutually exclusive. After obtaining the matching results through the matching model in step 7, samples that failed to match are filtered and merged with samples in the corpus. The merging method includes: checking whether there are samples with the same semantics in the corpus to be merged; if so, merging the two into the same sample; otherwise, adding a new sample to the corpus. In step 5, the pre-trained language model based on the representational matching network uses ERNIE model parameters, and the dimensionality reduction layer uses the CLSpool output by the pre-trained language model for dimensionality reduction. The loss function of the network is InfoNCE loss, including:

[0012]

[0013] Where lossq is the loss value for any entity information q, k+ is the sample belonging to the same category as q, τ is the temperature coefficient, ki is the sample belonging to a different category than q, k is the maximum quantity value, and exp is the exponential function. The loss value of the network during each training is the average of multiple lossq. The pre-trained language model based on the interactive matching network uses ERNIE model parameters, and the network's loss function is the cross-entropy loss function. Since the network task is binary classification, it is equivalent to the binary classification cross-entropy loss function, and its calculation method is the same as the loss function in step 3.

[0014] Based on the above method embodiments, the present invention provides a method for extracting and clustering information on public opinion big data-based complaint cases. In step 6, since each category contains multiple samples, when generating samples based on the representational matching model, the similarity between samples in each category is first calculated through the network, and the two samples with the highest similarity are selected as positive samples, q and k+. Other samples from the same batch during training are ki. During training, the model uses two data augmentation methods. First, before the samples are input into the matching model, the word repetition strategy is used to repeat Chinese characters at random positions twice. Second, when the samples are input into the network, the dropout method is used to randomly discard neurons as new positive samples to enhance the robustness of the model. When generating samples based on the interactive matching model, positive samples are generated by pairwise combinations in each category. Since the interactive matching model is sensitive to the order of samples, the order of two samples in each positive sample is swapped for data augmentation. Negative samples are generated by randomly selecting samples from different categories and combining them in pairs. The number of negative samples generated depends on the number of positive samples.

[0015] Based on the above method embodiments, the present invention provides a method for extracting and clustering information on appeal cases based on public opinion big data. In step 7, the method for matching samples in the synonym corpus is to match the text to be matched with the text in the corpus pairwise. If the edit distance is 0, the match is successful; otherwise, model matching is performed. The model matching method includes: first, using a representational matching model to extract the vectors of all samples to be matched; then, building a kd-tree index for all vectors of these samples; traversing each text to be matched; using a kd-tree to recall the text corresponding to the top N vectors; using an interactive matching network to match pairwise and recording the matching probability p; since the number of sample categories cannot be known in advance, after traversal, using the DBSCAN method for clustering; during clustering, the top N samples recalled for each sample are considered as the neighboring samples of that sample; the matching probability between samples is considered as the distance between samples; and the frequency is calculated as follows: each case to be processed has a case number; for a synonym, the size of the case number set among all cases in which the word appears is used as the frequency.

[0016] Secondly, embodiments of the present invention provide a clustering device for extracting information from public opinion cases based on big data, comprising: a first main module, used to implement step 1: constructing an entity information extraction corpus, collecting data of public opinion cases to be processed, determining the entity objects and type range of the information to be extracted, including addresses, companies, and points of interest, obtaining a small number of samples through manual annotation to form an entity information extraction corpus, pre-extracting a portion of the results through the information extraction methods in steps 2 and 3 to verify whether the determined entity objects are reasonable and accurate, and after verification, merging the small number of manually annotated samples with the results automatically extracted by the model as the extraction corpus; step 2: constructing an entity information extraction model, utilizing a data-driven approach... ERNIE's pre-trained language model serves as the encoder, followed by a pointer network for entity extraction, constructing an entity information extraction model. The pointer network outputs indices of the start and end positions of the extracted entities in the text, which are used to extract the entities from the text. The second main module implements step 3: training the entity information extraction model and extracting entity information from cases, generating the dataset required for model training. Based on the corpus constructed in step 1, a training set and a validation set are randomly generated according to a certain ratio. In the training set, positive and negative samples are generated according to a certain ratio for model training. The trained model is used to extract the case text to be extracted. If there are multiple entities to be extracted, for each... Step 4: Construct an entity information synonym corpus. Select a portion of the text data to be matched, i.e., the entity information extracted in Step 3. Obtain a small number of synonym samples through manual annotation to form an entity information synonym corpus. Synonyms refer to samples in each category that semantically point to the same entity. If two samples are synonyms, they are said to match each other. A portion of the results are pre-generated using the information clustering method in Step 7 as a reference. Add some incorrect results from the model clustering to the manually annotated samples. Generate an entity information synonym corpus through multiple iterations. The third main module is used to implement Step 5: Construct an entity information matching model in order to enable... To quickly obtain matching results from massive amounts of case complaint data, the matching model consists of a representation-based matching network and an interactive matching network. The representation-based matching network is used to quickly recall the top N results in the dataset that are similar to the sample to be matched. This matching network uses a pre-trained language model based on ERNIE as the encoder, followed by a fully connected layer for vector dimensionality reduction. The interactive matching network is responsible for determining whether the sample matches the top N recalled results. This interactive matching network uses a pre-trained language model based on ERNIE as the encoder, followed by a fully connected layer as the classifier, transforming the determination of whether two texts match into a binary classification problem and outputting the result.Step 6: Train the entity information matching model and generate the dataset required for the model. Based on the corpus constructed in Step 4, randomly divide it proportionally to generate training and validation sets. In the training set, generate positive and negative samples proportionally for model training. Since there are relatively few samples with synonym relationships in the dataset, it is prone to long-tail data problems. Data augmentation is used to generate positive samples as the training set. The fourth main module is used to implement Step 7: Case entity information matching and clustering. Based on the samples in the synonym corpus, match the dataset to be matched. Successful matches are taken as entity clustering results. Use the model for matching, and use a representational matching network to extract vector representations of all samples in the dataset. Then, establish kd... Tree indexing is used for fast matching. For any text to be matched, the most similar samples from the top N datasets are recalled, with similarity measured by cosine distance. Then, an interactive matching network is used to match the text to be matched with each of the N samples pairwise, determining whether they match. The matching probability output by the interactive matching network is used as the distance between the pairs of texts. The DBSCAN method is used for clustering to obtain the merged results of the case entity information matched by the model. The model matching results are merged with the results of the thesaurus matching. Results that the model failed to match are merged into the results through manual judgment. The frequency of occurrence of each entity in the results is counted. Finally, the results can be merged with the total corpus to increase the number of matching samples in the corpus.

[0017] Thirdly, embodiments of the present invention provide an electronic device, comprising:

[0018] At least one processor; and

[0019] At least one memory communicatively connected to the processor, wherein:

[0020] The memory stores program instructions that can be executed by the processor. The processor can call the program instructions to execute the clustering method for extracting and classifying complaint information based on public opinion big data provided by any of the various implementation methods of the first aspect.

[0021] Fourthly, embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer instructions that cause a computer to execute a method for extracting and clustering information on public opinion big data based on any of the various implementations of the first aspect.

[0022] The present invention provides a method and device for extracting and clustering information on public opinion cases based on big data. This method can extract key information on public opinion cases from big data. By using a deep learning-based matching model and an unsupervised clustering method, it can quickly merge and cluster cases on a large-scale dataset of public opinion cases. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating the method for extracting and clustering information on public opinion cases based on big data, as provided in this embodiment of the invention.

[0025] Figure 2 A schematic diagram of the structure of the petition case information extraction and clustering device based on public opinion big data provided in an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined with each other to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0028] This invention provides a method for extracting and clustering information on complaint cases based on public opinion big data. See [link to relevant documentation]. Figure 1The method includes: Step 1: Constructing an entity information extraction corpus, collecting data on pending cases, determining the entity objects and types to be extracted, including addresses, companies, and points of interest, obtaining a small number of samples through manual annotation to form an entity information extraction corpus, and pre-extracting a portion of the results using the information extraction methods in Steps 2 and 3 to verify whether the determined entity objects are reasonable and accurate. After verification, the small number of manually annotated samples are merged with the results automatically extracted by the model to form the extraction corpus; Step 2: Constructing an entity information extraction model, using a pre-trained language model based on ERNIE as the encoder, followed by a pointer network for extracting entity objects, constructing an entity information extraction model, wherein the output of the pointer network is the entity extraction corpus. The text is indexed at the start and end points, and entities are extracted from the text using these indices. Step 3: Train the entity information extraction model and extract entity information from the cases. Generate the dataset required for training the model. Based on the corpus constructed in Step 1, randomly divide it into training and validation sets according to a certain ratio. In the training set, generate positive and negative samples according to a certain ratio for model training. Use the trained model to extract the case text to be extracted. If there are multiple entities to be extracted, extract each entity separately to obtain the results. Step 4: Construct an entity information synonym corpus. Select a portion of the text data to be matched, i.e., the entity information extracted in Step 3, and obtain a small number of synonym samples through manual annotation to form an entity information synonym corpus. Synonyms refer to samples in each category that semantically refer to the same entity. If two samples are synonyms, they are said to match. A portion of the results are pre-generated using the information clustering method in step 7 as a reference. Some incorrect clustering results from the model are added to the manually labeled samples. An entity information synonym corpus is generated through multiple iterations. Step 5: Construct an entity information matching model. To quickly obtain matching results from massive amounts of case complaint data, the matching model consists of a representation-based matching network and an interactive matching network. The representation-based matching network is used to quickly recall the top N results in the dataset that are similar to the sample to be matched. This matching network uses a pre-trained language model based on ERNIE as the encoder. The system then uses a fully connected layer for vector dimensionality reduction. An interactive matching network is responsible for determining whether the sample matches the top N recalled results. This interactive matching network uses an ERNIE-based pre-trained language model as the encoder, followed by a fully connected layer as the classifier. This transforms the determination of whether two texts match into a binary classification problem and outputs the results. Step 6: Train the entity information matching model and generate the dataset required by the model. Based on the corpus constructed in Step 4, the training set and validation set are randomly divided proportionally. In the training set, positive and negative samples are generated proportionally for model training. Since there are few samples with synonym relationships in the dataset, it is easy to cause the long tail problem of data. Data augmentation is used to generate positive samples as the training set.Step 7: Case entity information matching and clustering. Based on samples in the synonym corpus, the dataset to be matched is matched. Successful matches are taken as entity clustering results. The model is used for matching. A representational matching network is used to extract vector representations of all samples in the dataset. Then, a kd-tree index is built for all vectors for fast matching. For any text to be matched, the most similar samples in the top N datasets are recalled. The similarity is measured by cosine distance. Then, an interactive matching network is used to match the text to be matched with each of the N samples. The matching probability output by the interactive matching network is used as the distance between the pairs of texts. The DBSCAN method is used for clustering to obtain the merged case entity information matching results. The model matching results are merged with the results of the synonym corpus matching. Results that fail to match are merged into the results by manual judgment. The frequency of occurrence of each entity in the results is counted. Finally, the results can be merged with the total corpus to increase the matching samples in the corpus.

[0029] Based on the above method embodiments, as an optional embodiment, the method for extracting and clustering information on public opinion big data provided in this embodiment of the invention includes the following steps in determining the entity objects and type range of the information to be extracted: by labeling a small number of samples, defining the information to be extracted and listing examples in different contexts for the labelers to refer to, and determining the type range of the information to be extracted by formulating labeling specifications, verifying the reasonableness of the extracted results through the model, and if the verification is incorrect, adding the incorrectly extracted samples to the corpus through verification. Among them, the type of the sample that fails to identify the entity type is corrected. If it does not belong to any type after correction, it is added to the corpus as a negative sample.

[0030] Based on the above method embodiments, as an optional embodiment, the method for extracting and clustering information on public opinion big data in this embodiment of the invention includes the following steps: In step 2, the pre-trained language model based on deep learning is the model structure and parameters of UIE in ERNIE. The pointer network consists of two identical parts, which predict the start and end positions of the entities to be extracted in the text. This includes: if the maximum length of the text is L, the text is truncated if the length exceeds L. After a piece of text passes through the encoder, an L*D dimensional vector is obtained. After passing through a fully connected layer, an L*1 dimensional vector is obtained. Finally, a sigmoid activation function is used to obtain a vector of length L, where each value is the probability of the index being the start position of the entity to be extracted. The final result is the index of the maximum probability, and the index of the end position is obtained. The information text to be extracted is obtained by truncating the text between the start and end positions.

[0031] Based on the above method embodiments, as an optional embodiment, the method for extracting and clustering information on complaint cases based on public opinion big data provided in this embodiment of the invention, in step 3, the positive samples generated in the training set include the case text and the entity types and indices present therein, while the negative samples refer to the case text and the entity types and empty indices that do not exist therein. The number of negative samples generated is related to the number of positive samples, with N being the ratio of the number of negative samples to the number of positive samples. The role of the negative samples is to indicate the entity types that do not exist in the text, which is used by the model to learn the features of different types. The case text is formatted as [CLS]Prompt[SEP]Text[SEP] and its identifier is processed, where [CLS] is used as the separator for the classification task, [SEP] is the separator to distinguish between two sentences, Prompt is the entity type, and Text is the case text. The training objective is the starting position of the corresponding entity type, and the loss function is the binary classification cross-entropy loss. After step 2, the vector output by the sigmoid activation function of the model predicting the starting position is X={x1,x2,...,x N}, then Y = {y1, y2, ..., y} N} represents the real label, where y exists. i =1 indicates that the text at index i is the starting position of this entity type. Therefore, the loss function includes:

[0032]

[0033] l n =y n logx n +(1-y n log(1-x) n )

[0034] Where lossBCE is the loss function at the starting position, ln is an intermediate variable, xn is the nth element of the vector X output by the igmoid activation function, yn is the nth element of the true label Y, and N is the number of elements. After obtaining the loss function at the ending position, the loss function at the starting position and the loss function at the ending position are averaged to obtain the final loss function.

[0035] Based on the above method embodiments, as an optional embodiment, the method for extracting and clustering information on public opinion big data for appeals provided in this embodiment of the invention includes the following steps: In step 4, each piece of data in the corpus contains one or more semantically identical entity information, and each piece of data is semantically mutually exclusive. After obtaining the matching results through the matching model in step 7, samples that failed to match are filtered and merged with samples in the corpus. The merging method includes: checking whether there are samples with the same semantics in the corpus to be merged; if so, the two are merged into the same sample; otherwise, a new sample is added to the corpus. In step 5, the pre-trained language model based on the representational matching network uses ERNIE model parameters, and the dimensionality reduction layer uses the CLSpool output by the pre-trained language model for dimensionality reduction. The loss function of the network is InfoNCE loss, which includes:

[0036]

[0037] Where lossq is the loss value for any entity information q, k+ is the sample belonging to the same category as q, τ is the temperature coefficient, ki is the sample belonging to a different category than q, k is the maximum quantity value, and exp is the exponential function. The loss value of the network during each training is the average of multiple lossq. The pre-trained language model based on the interactive matching network uses ERNIE model parameters, and the network's loss function is the cross-entropy loss function. Since the network task is binary classification, it is equivalent to the binary classification cross-entropy loss function, and its calculation method is the same as the loss function in step 3.

[0038] Based on the above method embodiments, as an optional embodiment, the clustering method for extracting and classifying complaint case information based on public opinion big data provided in this embodiment of the invention, in step 6, since each category contains multiple samples, when generating samples based on the representational matching model, firstly, the similarity between samples in each category is calculated through the network, and the two samples with the highest similarity are selected as positive samples, q and k+. Other samples in the same batch during training are ki. During model training, two data augmentation methods are adopted. First, before the samples are input into the matching model, the word repetition strategy is used to repeat Chinese characters at random positions twice. Second, when the samples are input into the network, the dropout method is used to randomly discard neurons as new positive samples to enhance the robustness of the model. When generating samples based on the interactive matching model, positive samples are generated by pairwise combinations in each category. Since the interactive matching model is sensitive to the order of samples, the order of two samples in each positive sample is swapped for data augmentation. Negative samples are generated by randomly selecting samples from different categories and combining them in pairs. The number of generated samples depends on the number of positive samples.

[0039] Based on the above method embodiments, as an optional embodiment, the method for extracting and clustering information on appeal cases based on public opinion big data provided in this embodiment of the invention, in step 7, the method of matching samples in the synonym corpus is to match the text to be matched with the text in the corpus pairwise. If the edit distance is 0, the match is successful; otherwise, model matching is performed. The model matching method includes: first, using a representational matching model to extract the vectors of all samples to be matched; then, building a kd-tree index for all vectors of these samples; traversing each text to be matched; using a kd-tree to recall the text corresponding to the first N vectors; using an interactive matching network to match pairwise and recording the matching probability p; since the number of sample categories cannot be known in advance, after traversal, using the DBSCAN method for clustering; during clustering, the first N samples recalled for each sample are considered as the neighboring samples of that sample; the matching probability between samples is considered as the distance between samples; the frequency counting method is as follows: each case to be processed has a case number; for a synonym, the size of the case number set among all cases in which the word appears is counted as the frequency.

[0040] The method for extracting and clustering information on public opinion cases based on big data provided in this invention can extract key information on public opinion cases from big data. By using a deep learning-based matching model and an unsupervised clustering method, it can quickly merge and cluster cases on a large-scale dataset of public opinion cases.

[0041] The implementation of the various embodiments of this invention is based on programmed processing through a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of this invention can be encapsulated into various modules. Based on this reality, and building upon the above embodiments, this invention provides a device for extracting and clustering information on public opinion-related cases based on big data. This device is used to execute the method for extracting and clustering information on public opinion-related cases based on big data in the above method embodiments. See also... Figure 2The device includes: a first main module, used to implement step 1: constructing an entity information extraction corpus, collecting data on pending cases, determining the entity objects and types of information to be extracted, including addresses, companies, and points of interest, obtaining a small number of samples through manual annotation to form an entity information extraction corpus, and pre-extracting a portion of the results using the information extraction methods in steps 2 and 3 to verify whether the determined entity objects are reasonable and accurate. After verification, the manually annotated small number of samples are merged with the results automatically extracted by the model to form the extraction corpus; step 2: constructing an entity information extraction model, using a pre-trained language model based on ERNIE as the encoder, followed by a pointer network for... To extract entity objects, an entity information extraction model is constructed. The output of the pointer network is an index of the start and end positions of the entity to be extracted in the text. The entity to be extracted is extracted from the text using the index. The second main module is used to implement step 3: training the entity information extraction model and extracting entity information from the case, generating the dataset required for training the model, and randomly dividing the corpus constructed in step 1 into a training set and a validation set according to a certain ratio. In the training set, positive samples and negative samples are generated according to a certain ratio for model training. The trained model is used to extract the case text to be extracted. If there are multiple entities to be extracted, each entity is extracted separately to obtain the result. Step 4 The first step involves constructing a corpus of entity information synonyms. A portion of the text data to be matched, i.e., the entity information extracted in step 3, is selected. A small number of synonym samples are obtained through manual annotation, forming the entity information synonym corpus. Synonyms refer to samples in each category that semantically refer to the same entity. If two samples are synonyms, they are said to match. A portion of the results are pre-generated using the information clustering method in step 7 as a reference. Some incorrect clustering results from the model are added to the manually annotated samples. The entity information synonym corpus is generated through multiple iterations. The third main module is used to implement step 5: constructing an entity information matching model, in order to match massive amounts of case claims. To quickly obtain matching results from the data, the matching model consists of a representation-based matching network and an interactive matching network. The representation-based matching network is used to quickly recall the top N results in the dataset that are similar to the sample to be matched. This matching network uses a pre-trained language model based on ERNIE as the encoder, followed by a fully connected layer for vector dimensionality reduction. The interactive matching network is responsible for determining whether the sample matches the top N recalled results. This interactive matching network uses a pre-trained language model based on ERNIE as the encoder, followed by a fully connected layer as the classifier, transforming the determination of whether two texts match into a binary classification problem and outputting the result.Step 6: Train the entity information matching model and generate the dataset required for the model. Based on the corpus constructed in Step 4, randomly divide it proportionally to generate training and validation sets. In the training set, generate positive and negative samples proportionally for model training. Since there are relatively few samples with synonym relationships in the dataset, it is prone to long-tail data problems. Data augmentation is used to generate positive samples as the training set. The fourth main module is used to implement Step 7: Case entity information matching and clustering. Based on the samples in the synonym corpus, match the dataset to be matched. Successful matches are taken as entity clustering results. Use the model for matching, and use a representational matching network to extract vector representations of all samples in the dataset. Then, establish kd... Tree indexing is used for fast matching. For any text to be matched, the most similar samples from the top N datasets are recalled, with similarity measured by cosine distance. Then, an interactive matching network is used to match the text to be matched with each of the N samples pairwise, determining whether they match. The matching probability output by the interactive matching network is used as the distance between the pairs of texts. The DBSCAN method is used for clustering to obtain the merged results of the case entity information matched by the model. The model matching results are merged with the results of the thesaurus matching. Results that the model failed to match are merged into the results through manual judgment. The frequency of occurrence of each entity in the results is counted. Finally, the results can be merged with the total corpus to increase the number of matching samples in the corpus.

[0042] The invention provides an embodiment of a device for extracting and clustering information on public opinion-related cases based on big data. Figure 2 Several modules within the system can extract key information from public opinion big data on cases involving public demands. Through deep learning-based matching models and unsupervised clustering methods, cases can be quickly merged and clustered on large-scale datasets of demands.

[0043] It should be noted that the apparatus in the device embodiments provided by the present invention can be used not only to implement the methods in the above method embodiments, but also to implement the methods in other method embodiments provided by the present invention. The difference lies only in the setting of corresponding functional modules. Its principle is basically the same as that of the above device embodiments provided by the present invention. As long as those skilled in the art, based on the above device embodiments and referring to the specific technical solutions in other method embodiments, obtain corresponding technical means and technical solutions composed of these technical means by combining technical features, and improve the apparatus in the above device embodiments while ensuring the practicality of the technical solutions, they can obtain corresponding device-type embodiments for implementing the methods in other method-type embodiments. For example:

[0044] Based on the above-described device embodiments, as an optional embodiment, the public opinion big data-based complaint case information extraction and clustering device provided in this embodiment of the invention further includes: a first sub-module, used to determine the entity objects and type range of the information to be extracted in step 1, including: by labeling a small number of samples, formulating the definition of the information to be extracted and listing examples in different contexts for the labelers to refer to, and by formulating labeling specifications to determine the type range of the information to be extracted, verifying the reasonableness of the extracted results through the model, and if the verification is incorrect, adding the incorrectly extracted samples to the corpus through verification, wherein the entity type recognition failure samples correct their type, and if they do not belong to any type after correction, they are added to the corpus as negative samples.

[0045] Based on the above device embodiments, as an optional embodiment, the device for extracting and clustering information on public opinion big data for appeals provided in this embodiment of the invention further includes: a second submodule, used to implement the deep learning-based pre-trained language model in step 2, which is the model structure and parameters of UIE in ERNIE. The pointer network consists of two identical parts, which predict the start and end positions of the entities to be extracted in the text, including: if the maximum length of the text is L, the text is truncated if the length exceeds L. When a piece of text passes through the encoder, an L*D dimensional vector is obtained. After passing through a fully connected layer, an L*1 dimensional vector is obtained. Finally, a sigmoid activation function is used to obtain a vector of length L, where each value is the probability of the index being the start position of the entity to be extracted. The final result is the index of the maximum probability, and the index of the end position is obtained. The information text to be extracted is obtained by truncating the text between the start and end positions.

[0046] Based on the above-described device embodiments, as an optional embodiment, the petition case information extraction and clustering device based on public opinion big data provided in this embodiment of the invention further includes: a third submodule, used to implement step 3, where the positive samples generated in the training set contain case text and entity types and indices present therein, and the negative samples refer to case text and entity types and empty indices not present therein. The number of negative samples generated is related to the number of positive samples, with N being the ratio of the number of negative samples to the number of positive samples. The role of the negative samples is to indicate entity types not present in the text, used by the model to learn features of different types. The case text is formatted as [CLS]Prompt[SEP]Text[SEP] and its identifier is processed, where [CLS] is used as the separator for the classification task, [SEP] is the separator for distinguishing between two sentences, Prompt is the entity type, and Text is the case text. The training objective is the starting position of the corresponding entity type, and the loss function is the binary cross-entropy loss. After step 2, the vector output by the sigmoid activation function of the model predicting the starting position is X = {x1, x2, ..., x N}, then Y = {y1, y2, ..., y}N} represents the real label, where y exists. i =1 indicates that the text at index i is the starting position of this entity type. Therefore, the loss function includes:

[0047]

[0048] l n =y n logx n +(1-y n log(1-x) n )

[0049] Where lossBCE is the loss function at the starting position, ln is the intermediate variable, xn is the nth element of the vector X output by the sigmoid activation function, yn is the nth element of the true label Y, and N is the number of elements. After obtaining the loss function at the ending position, the loss function at the starting position and the loss function at the ending position are averaged to obtain the final loss function.

[0050] Based on the above device embodiments, as an optional embodiment, the petition case information extraction and clustering device based on public opinion big data provided in this embodiment of the invention further includes: a fourth submodule, used to implement step 4, where each piece of data in the corpus contains one or more semantically identical entity information, and each piece of data is semantically mutually exclusive; after obtaining the matching result through the matching model in step 7, samples that failed to match the model are filtered and merged with samples in the corpus; the merging method includes: checking whether there are samples with the same semantics in the corpus to be merged; if so, merging the two into the same sample; otherwise, adding a new sample to the corpus; in step 5, the pre-trained language model based on the representational matching network uses ERNIE model parameters, and the dimensionality reduction layer uses the CLSpool output by the pre-trained language model for dimensionality reduction; the network loss function is InfoNCE loss, including:

[0051]

[0052] Where lossq is the loss value for any entity information q, k+ is the sample belonging to the same category as q, τ is the temperature coefficient, ki is the sample belonging to a different category than q, k is the maximum quantity value, and exp is the exponential function. The loss value of the network during each training is the average of multiple lossq. The pre-trained language model based on the interactive matching network uses ERNIE model parameters, and the network's loss function is the cross-entropy loss function. Since the network task is binary classification, it is equivalent to the binary classification cross-entropy loss function, and its calculation method is the same as the loss function in step 3.

[0053] Based on the above-described device embodiments, as an optional embodiment, the clustering device for extracting and classifying complaint case information based on public opinion big data provided in this embodiment of the invention further includes: a fifth submodule, used to implement step 6, where, since each category contains multiple samples, when generating samples based on the representational matching model, the similarity between samples in each category is first calculated through the network, and the two samples with the highest similarity are selected as positive samples, q and k+, and other samples from the same batch during training are ki. During model training, two data augmentation methods are used: first, before the samples are input into the matching model, the word repetition strategy is used to repeat Chinese characters at random positions twice; second, when the samples are input into the network, the dropout method is used to randomly discard neurons as new positive samples to enhance the robustness of the model. When generating samples based on the interactive matching model, positive samples are generated by pairwise combinations in each category. Since the interactive matching model is sensitive to the order of samples, the order of two samples in each positive sample is swapped for data augmentation. Negative samples are generated by randomly selecting samples from different categories and combining them in pairs, and the number of samples generated depends on the number of positive samples.

[0054] Based on the above device embodiments, as an optional embodiment, the petition case information extraction and clustering device based on public opinion big data provided in this embodiment of the invention further includes: a sixth sub-module, used to implement the method of matching samples in the synonym corpus in step 7, which is to match the text to be matched with the text in the corpus pairwise. If the edit distance is 0, the match is successful; otherwise, model matching is performed. The model matching method includes: firstly, using a representational matching model to extract the vectors of all samples to be matched, establishing a kd-tree index for all vectors of these samples, traversing each text to be matched, using a kd-tree to recall the text corresponding to the first N vectors, using an interactive matching network to match pairwise and record the matching probability p. Since the number of sample categories cannot be known in advance, after traversal, the DBSCAN method is used for clustering. During clustering, the first N samples recalled for each sample are considered as the neighboring samples of that sample, and the matching probability between samples is considered as the distance between samples. The frequency of occurrence is statistically analyzed as follows: each case to be processed has a case number, and for a synonym, the size of the case number set among all cases in which the word appears is counted as the frequency.

[0055] The method in this embodiment of the invention is implemented using an electronic device; therefore, it is necessary to introduce the relevant electronic device. For this purpose, this embodiment of the invention provides an electronic device, such as... Figure 3As shown, the electronic device includes at least one processor, a communications interface, at least one memory, and a communications bus, wherein the at least one processor, the communications interface, and the at least one memory communicate with each other via the communications bus. The at least one processor can invoke logical instructions stored in the at least one memory to execute all or part of the steps of the methods provided in the foregoing method embodiments.

[0056] Furthermore, when the logical instructions in at least one of the aforementioned memories can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various method embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0057] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0058] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0059] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Based on this understanding, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0060] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting and clustering information on public opinion cases based on big data, characterized in that, include: Step 1, Construct an entity information extraction corpus: Collect data on pending complaint cases, determine the entity objects and types of information to be extracted, including addresses, companies, and points of interest, obtain a small number of samples through manual annotation to form an entity information extraction corpus, and pre-extract a portion of the results using the information extraction methods in Steps 2 and 3 to verify whether the determined entity objects are reasonable and accurate. After verification, merge the small number of manually annotated samples with the results automatically extracted by the model to form an entity information extraction corpus. Step 2, Construct an entity information extraction model: Use a pre-trained language model based on ERNIE as an encoder, followed by a pointer network to extract entity objects, and construct an entity information extraction model. The output of the pointer network is the index of the start and end positions of the entity to be extracted in the text. The entity to be extracted in the text is extracted by using the index. Step 3, train the entity information extraction model and extract entity information from the case: generate the dataset required to train the entity information extraction model, and randomly divide the entity information extraction corpus constructed in Step 1 into a training set and a validation set according to a certain ratio. In the training set, generate positive samples and negative samples according to a certain ratio for training the entity information extraction model. Use the trained entity information extraction model to extract the case text to be extracted. If there are multiple entities to be extracted, extract each entity separately to obtain the results. Step 4, construct an entity information synonym corpus: Select a portion of the text data to be matched, namely the entity information extracted in Step 3, and obtain a small number of synonym samples through manual annotation to form an entity information synonym corpus. Synonyms refer to samples in each category that semantically point to the same entity. If two samples are synonyms, they are said to match each other. A portion of the results are pre-generated using the information clustering method in Step 7 as a reference. Some incorrect results from the model clustering are added to the manually annotated samples. The entity information synonym corpus is generated through multiple iterations. Step 5, Construct the entity information matching model: In order to quickly obtain matching results from massive amounts of case claim data, the entity information matching model consists of a representational matching network and an interactive matching network; The representational matching network is used to quickly recall the top N results in the dataset that are similar to the sample to be matched. The representational matching network uses a pre-trained language model based on ERNIE as the encoder, followed by a fully connected layer for vector dimensionality reduction. The interactive matching network is responsible for determining whether the sample matches the top N recalled results. The interactive matching network uses a pre-trained language model based on ERNIE as the encoder, followed by a fully connected layer as the classifier. It transforms the problem of determining whether two texts match into a binary classification problem and outputs the result. Step 6, train the entity information matching model: generate the dataset required to train the entity information matching model. According to the entity information synonym corpus constructed in Step 4, randomly divide it into training set and validation set according to the proportion. In the training set, generate positive samples and negative samples according to the proportion for training the entity information matching model. Since there are few samples with synonym relationships in the dataset, it is easy to cause the long tail problem of data. Use data augmentation to generate positive samples as training set. Step 7, Case Entity Information Matching and Clustering: Based on the samples in the entity information synonym corpus, match the dataset to be matched, and take the successful matches as the entity clustering results; The entity information matching model is used for matching. The representational matching network is used to extract the vector representation of all samples in the dataset. Then, a kd-tree index is built for all vectors for fast matching. For any text to be matched, the most similar samples in the top N datasets are recalled. The similarity is measured by cosine distance. Then, the interactive matching network is used to match the text to be matched with N samples one by one to determine whether they match. The matching probability output by the interactive matching network is used as the distance between the pairs of texts. The DBSCAN method is used for clustering to obtain the model matching result. The successful model matching results are merged with the entity clustering results to obtain the first merged result. The unsuccessful model matching results are merged into the first merged result through manual judgment to obtain the second merged result. The frequency of occurrence of each entity in the second merged result is counted. Finally, the second merged result is merged with the total corpus to increase the number of matching samples in the total corpus.

2. The method for extracting and clustering information on complaint cases based on public opinion big data as described in claim 1, characterized in that, Step 1, determining the entity objects and type range of the information to be extracted, includes: defining the information to be extracted and listing examples in different contexts for annotators to refer to by labeling a small number of samples; determining the type range of the information to be extracted by formulating labeling specifications; verifying the reasonableness of the extraction results through the model; if the verification is incorrect, adding the incorrectly extracted samples to the entity information extraction corpus after verification; correcting the type of samples that fail to identify the entity type; if the corrected type does not belong to any type, it is added to the entity information extraction corpus as a negative sample.

3. The method for extracting and clustering information on complaint cases based on public opinion big data as described in claim 2, characterized in that, The deep learning-based pre-trained language model in step 2 uses the model structure and parameters of UIE in ERNIE. The pointer network consists of two identical parts that predict the start and end positions of the entity to be extracted in the text. This includes: if the maximum text length is L, the text is truncated if the text length exceeds L. After a piece of text passes through the encoder, an L×D dimensional vector is obtained. After passing through a fully connected layer, an L×1 dimensional vector is obtained. Finally, a sigmoid activation function is used to obtain a vector of length L, where each value is the probability of the index being the start position of the entity to be extracted. The final result is the index of the maximum probability, and the index of the end position is obtained. The information text to be extracted is obtained by truncating the text between the start and end positions.

4. The method for extracting and clustering information on complaint cases based on public opinion big data as described in claim 3, characterized in that, In step 3, the positive samples generated in the training set contain the case text along with the entity types and indices present within it. Negative samples refer to the case text along with non-existent entity types and empty indices. The number of negative samples generated is related to the number of positive samples, with N representing the ratio of negative to positive samples. The purpose of negative samples is to indicate entity types not present in the text, which is used by the model to learn features of different types. The case text is formatted as [CLS]Prompt[SEP]Text[SEP], and its identifier is processed. [CLS] is used as the separator for the classification task, [SEP] is the separator to distinguish between two sentences, Prompt represents the entity type, and Text represents the case text. The training objective is to determine the starting position of the corresponding entity type, and the loss function is the binary cross-entropy loss. After step 2, the vector output by the sigmoid activation function predicting the starting position is... ,but For real labels, there are If the text at index i is the starting position of this entity type, then the loss function includes: in, The loss function is the starting position. As an intermediate variable, Let be the nth element of the vector X output by the sigmoid activation function. Let N be the nth element of the true label Y, and N be the number of elements. After obtaining the loss function at the termination position, the loss function at the starting position and the loss function at the termination position are averaged to obtain the final loss function.

5. The method for extracting and clustering information on complaint cases based on public opinion big data according to claim 4, characterized in that, In step 4, each piece of data in the entity information synonym corpus contains one or more semantically identical entity information. Each piece of data is semantically mutually exclusive. After obtaining the matching results through the matching model in step 7, the samples that failed to match the model are merged with the samples in the entity information synonym corpus. The merging method includes: checking whether there are samples with the same semantics as the samples to be merged in the entity information synonym corpus. If there are, the two are merged into the same sample; otherwise, a new sample is added to the corpus. In step 5, the pre-trained language model based on the representational matching network uses ERNIE model parameters, and the dimensionality reduction layer uses the CLSpool output of the pre-trained language model for dimensionality reduction. The network's loss function is InfoNCE loss, which includes: in, Let be the loss value for any entity information q. For samples belonging to the same category as q, For temperature coefficient, For samples belonging to a different class than q, k is the maximum number of samples, exp is an exponential function, and the network loss value is multiple during each training iteration. The average value is obtained by using the ERNIE model parameters in the pre-trained language model based on the interactive matching network. The network's loss function is the cross-entropy loss function. Since the network task is binary classification, it is equivalent to the binary classification cross-entropy loss function, and its calculation method is the same as the loss function in step 3.

6. The method for extracting and clustering information on complaint cases based on public opinion big data as described in claim 5, characterized in that, In step 6, since each category contains multiple samples, when generating samples based on the representational matching network, the similarity between samples in each category is first calculated through the network, and the two samples with the highest similarity are selected as positive samples. q and Other samples from the same batch during training are used as... During training, the model employs two data augmentation methods. First, before the samples are input into the matching model, a word repetition strategy is used to repeat Chinese characters at random positions twice. Second, when the samples are input into the network, the dropout method is used to randomly discard neurons as new positive samples to enhance the robustness of the model. When generating samples based on the interactive matching network, positive samples are generated by pairwise combinations within each category. Since the interactive matching network is sensitive to the order of samples, the order of two samples in each positive sample is swapped for data augmentation. Negative samples are generated by randomly selecting samples from different categories and combining them in pairs. The number of negative samples generated depends on the number of positive samples.

7. The method for extracting and clustering information on complaint cases based on public opinion big data as described in claim 6, characterized in that, In step 7, the method for matching samples from the entity information synonym corpus is to match the text to be matched with the text in the entity information synonym corpus pairwise. If the edit distance is 0, the match is successful; otherwise, model matching is performed. The model matching method includes: first, using a representational matching network to extract the vectors of all samples to be matched; then, building a kd-tree index for all vectors of these samples; traversing each text to be matched; using the kd-tree to recall the text corresponding to the top N vectors; using an interactive matching network to match pairwise and recording the matching probability p; since the number of sample categories cannot be known in advance, after traversal, using the DBSCAN method for clustering; during clustering, the top N samples recalled for each sample are considered as the neighboring samples of that sample; the matching probability between samples is considered as the distance between samples; the frequency is calculated as follows: each case to be processed has a case number; for a synonym, the size of the case number set among all cases in which the word appears is used as the frequency.

8. A device for extracting and clustering information on public opinion cases based on big data, characterized in that, include: The first main module is used to implement step 1, constructing an entity information extraction corpus: collecting data from pending cases, determining the entity objects and types to be extracted, including addresses, companies, and points of interest, obtaining a small number of samples through manual annotation to form an entity information extraction corpus, and pre-extracting a portion of the results using the information extraction methods in steps 2 and 3 to verify the reasonableness and accuracy of the determined entity objects. After verification, the small number of manually annotated samples are merged with the results automatically extracted by the model to form the entity information extraction corpus; Step 2, constructing an entity information extraction model: using a pre-trained language model based on ERNIE as an encoder, followed by a pointer network to extract entity objects, constructing an entity information extraction model, where the output of the pointer network is the index of the start and end positions of the entity to be extracted in the text, and the entity to be extracted in the text is extracted through the index; The second main module is used to implement step 3, training the entity information extraction model and extracting information from the cases. Entity Information: Generate the dataset required for training the model. Based on the entity information extraction corpus constructed in step 1, randomly divide it into training and validation sets according to a certain ratio. In the training set, generate positive and negative samples according to a certain ratio for model training. Use the trained model to extract the case text to be extracted. If there are multiple entities to be extracted, extract each entity separately to obtain the results. Step 4: Construct an entity information synonym corpus: Select a portion of the text data to be matched, i.e., the entity information extracted in step 3. Obtain a small number of synonym samples through manual annotation to form an entity information synonym corpus. Synonyms refer to samples in each category that semantically point to the same entity. If two samples are synonyms, they are said to match each other. Use the information clustering method in step 7 to pre-generate a portion of the results as a reference. Add some incorrect results from the model clustering to the manually annotated samples. Generate an entity information synonym corpus through multiple iterations. The third main module is used to implement step 5, constructing the entity information matching model: In order to quickly obtain matching results from massive amounts of case claim data, the entity information matching model consists of a representational matching network and an interactive matching network. The representational matching network is used to quickly recall the top N results in the dataset that are similar to the sample to be matched. The representational matching network uses an ERNIE-based pre-trained language model as the encoder, followed by a fully connected layer for vector dimensionality reduction. The interactive matching network is responsible for judging whether the sample matches the top N recalled results. This interactive matching network uses an ERNIE-based pre-trained language model as the encoder, followed by a fully connected layer as the classifier, transforming the judgment of whether two texts match into a binary classification problem and outputting the result. Step 6, training the entity information matching model: Generate the dataset required for training the entity information matching model. According to the entity information synonym corpus constructed in step 4, randomly divide it proportionally to generate a training set and a validation set. In the training set, generate positive and negative samples proportionally for training the entity information matching model. Since there are few samples with synonym relationships in the dataset, it is easy to cause the data long-tail problem. The first module uses data augmentation to generate positive samples as the training set. The fourth main module is used to implement step 7, case entity information matching and clustering: Based on the samples in the entity information synonym corpus, the dataset to be matched is matched, and the successfully matched samples are used as the entity clustering results. The model is used for matching, and the vector representation of all samples in the dataset is extracted using the representational matching network. Then, a kd-tree index is built for all vectors for fast matching. For any text to be matched, the most similar samples in the top N datasets are recalled. The similarity is measured by cosine distance. Then, the interactive matching network is used to match the text to be matched with N samples one by one to determine whether they match. The matching probability output by the interactive matching network is used as the distance between the pairs of texts. The DBSCAN method is used for clustering to obtain the model matching results. The model matching results are merged with the entity clustering results to obtain the first merged result. The model matching failure results are merged into the first merged result by manual judgment to obtain the second merged result. The frequency of occurrence of each entity in the second merged result is counted. Finally, the result can be merged with the total corpus to increase the matching samples in the total corpus.

9. An electronic device, characterized in that, include: At least one processor, at least one memory, and a communication interface; wherein, The processor, memory, and communication interface communicate with each other; The memory stores program instructions that can be executed by the processor, which invokes the program instructions to perform the method described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the method described in any one of claims 1 to 7.

Images