A consumer protection hotspot data analysis system and method based on clustering algorithm

Abstract

The present invention relates to the field of data analysis, and specifically to a consumer protection hot spot data analysis system and method based on a clustering algorithm. The system comprises: a data acquisition module, a data processing unit, a model iteration module, and an industry classification unit; the data acquisition module is used to collect historical data of consumer protection and rights protection complaints and corresponding industry classification results, and is also used to obtain consumer protection and rights protection data complaint texts in real time for classification and clustering model iteration; the data processing unit is used to perform text processing on the acquired consumer protection and rights protection data, and obtain word vectors of each complaint text and similarity weight data of each keyword; the model iteration module is used to construct an industry classification model according to the historical data of consumer protection and rights protection complaints and the corresponding industry classification results, and is also used to update and iterate the industry classification model according to the selected new complaint text data; the industry classification unit uses the industry classification model to classify the consumer protection and rights protection data acquired in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular to a consumer protection hotspot data analysis system and method based on a clustering algorithm. Background Art

[0002] With the improvement of public awareness of rights protection, the growth of the number of market players and the diversification of consumer activities, the number of reports and complaints received by various consumer rights protection platforms has increased day by day, and the protection of consumer rights and interests, such as acceptance, mediation, assignment, disposal and supervision of related incidents, has become particularly critical;

[0003] At present, most consumer rights protection cases on the Internet are entered by consumers themselves. There is a lack of accurate understanding of the industry classification of rights protection data and it is rather arbitrary, and there are many mistakes in the industry classification selection. At the same time, due to the low quality of data entered by users, the huge amount of data, the diversity of data and the unpredictability, the traditional fully supervised text classification technology based on manual calibration and then machine learning cannot be fully applied. In addition, the autonomous classification models formed by unsupervised clustering algorithms on many public Internet platforms have inconsistent data foundations and different results for the same problem, which cannot provide consumers with accurate and reliable classification results and rights protection directions.

[0004] Therefore, a consumer protection hotspot data analysis system and method based on clustering algorithm is needed to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a consumer protection hotspot data analysis system and method based on a clustering algorithm to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] A method for analyzing consumer protection hotspot data based on a clustering algorithm includes the following analysis steps:

[0008] Step S100: Obtain historical data of consumer protection and rights protection complaints, pre-process the historical data complaint text, obtain word vectors of the historical data complaint text, and summarize the keywords in all word vectors into an identified keyword library;

[0009] Step S200: Performing initial clustering of the pre-processed historical data according to industry affiliation based on the historical data complaint text word vectors, and analyzing and calculating the center points of the consumer protection data clusters of each industry;

[0010] Step S300: Obtain consumer protection and rights protection complaint information in real time, extract complaint text and perform keyword extraction and word weight analysis, calculate the similarity between the extracted unidentified keywords and the keywords in the identified keyword library, map the unidentified keywords in the complaint text obtained in real time to identified keywords, and transform the word vector of the complaint text obtained in real time;

[0011] Step S400: classify the complaint text obtained in real time according to the converted word vectors, and provide feedback to the corresponding processing department according to the classification results;

[0012] Step S500: monitor the number of consumer protection and rights protection complaint information in real time and set an upper limit of the number threshold. When the number of consumer protection and rights protection complaint information is greater than the threshold, mark the time unit and select new data for clustering model iteration.

[0013] According to the above technical solution, step S100 includes the following contents:

[0014] The BERT algorithm is used to process the historical data complaint text. For any historical data complaint text, all keywords in the complaint text and the similarity weight between each keyword and the current complaint text are obtained, and a similarity weight threshold is set. When the similarity weight between the keyword and the complaint text is less than the threshold, the keyword is removed from the keywords filtered by the current complaint text. Then, the keyword data of all historical data complaint texts after preprocessing are summarized into an identified keyword library, and word vectors are generated for each historical data complaint text according to the similarity weights of all keywords in the keyword library in each historical data complaint text. By vectorizing the complaint text data, the originally complex and diverse complaint texts are unified to form standard word vectors, providing standardized data standards for the initial clustering of the model.

[0015] The word vector form of each historical data complaint text is: ;in, is the word vector of the current complaint text, is the similarity weight of all keywords in the keyword library in the current complaint text, is the number of keywords in the identified keyword library; the keyword-to-text similarity weight is used as the word vector parameter to calculate the cluster center, fully considering the impact of different keywords on the complaint text classification results when they appear in the complaint text;

[0016] According to the above technical solution, step S200 includes the following contents:

[0017] Classify the complaint texts of each historical data according to the industry affiliation, and set the number of clusters according to the number of consumer protection and rights protection complaint industries. For any industry, according to the formula: Calculate the center point of each industry consumer protection data cluster; among them, is the center point of the current industry cluster, The number of complaint texts of the current industry historical data, The historical data complaint text number. Complaint text for historical data Word vectors;

[0018] According to the above technical solution, step S300 includes the following contents:

[0019] Obtain consumer protection complaint information in real time, process the complaint text using the BERT algorithm, obtain all keywords in the complaint text and the similarity weights between each keyword and the text;

[0020] Then calculate the similarity between the unidentified keywords and all the keywords in the identified keyword library, and then modify the similarity weight between each identified keyword in the complaint text word vector and the current complaint text. For any identified keyword in the complaint text word vector, the modified similarity weight calculation formula is as follows:

[0021] ;

[0022] in, is the corrected similarity weight of the currently identified keyword, is the similarity weight of the currently identified keyword before correction, Numbers for unidentified keywords in the complaint text. is the number of unidentified keywords in the complaint text. Unidentified keywords The similarity with the currently identified keywords, Unidentified keywords Similarity weight in the current complaint text;

[0023] By converting the unidentified keywords in the consumer protection and rights protection complaint information obtained in real time into identified keywords based on similarity, it ensures that the model can still be applied when the corresponding keywords are not included in the identified keyword library, thereby improving the universality of the model when used;

[0024] According to the above technical solution, step S400 includes the following contents:

[0025] Obtain the corrected word vector of the complaint text in the consumer protection and rights protection complaint information, and calculate the distance between the complaint text and the center point of the consumer protection data cluster of each industry. For the center point of any consumer protection data cluster in any industry, according to the formula:

[0026] ;

[0027] in, is the distance between the complaint text and the center point of the current industry consumer protection data cluster, Number the identified keywords. is the number of identified keywords, The keywords identified in the center of the current industry consumer protection data cluster Corresponding data, The keywords identified in the corrected complaint text word vector Corresponding data;

[0028] Set a distance threshold, select the center point of the industry consumer protection data cluster whose distance to the complaint text is less than the threshold, calculate the inverse of the distance between each center point and the complaint text and distribute it proportionally, and obtain the correlation factor between the selected industries and the complaint text. For any industry, according to the formula: ;in, is the correlation factor between the current industry and the complaint text, is the distance between the complaint text and the center point of the current industry consumer protection data cluster, The distance between the center point of the selected consumer protection data cluster in each industry and the complaint text;

[0029] According to the correlation factor, the industries are sorted from large to small, and feedback is given to the industry complaint handling department at equal time intervals until the department feedback is received, and the complaint text is classified into the industry category corresponding to the received feedback; in the multi-industry classification, the correlation factor between the complaint text and the industry is analyzed according to the distance between the complaint text and the cluster center point, reflecting the degree of correlation between the complaint text and the industry, and then multi-industry feedback is realized to ensure that even when a single text belongs to multiple industries, consumers can seek more accurate and convenient rights protection directions;

[0030] According to the above technical solution, step S500 includes the following contents:

[0031] Set the unit time and iteration threshold, count the number of consumer protection and rights protection complaints received in each unit time, and calculate the average number of consumer protection and rights protection complaints received in each unit time. and standard deviation , set the upper limit of quantity threshold When the number of consumer protection and rights protection complaint information received per unit time is greater than the upper limit of the quantity threshold, the current time unit is marked. When the number of marked time units is greater than the iteration threshold, all complaint texts and final classification results of the texts within the marked time are obtained for model iteration. The emergence of relevant rights protection hotspots is analyzed through data statistics to ensure healthy iteration and long-term optimization of the model under long-term use.

[0032] Obtain all unidentified keywords in the complaint text, count the number of times each unidentified keyword appears in the complaint text and the sum of the similarity weights between the keyword and the complaint text when it appears, set the weight and threshold, and add the unidentified keywords whose sum of similarity weights is greater than the weight and threshold to the identified keyword library;

[0033] Use the selected new complaint text data for clustering, generate complaint text word vectors based on the expanded identified keyword library, and calculate the centers of the new complaint text data clusters in each industry, and then analyze the center offsets of the consumer protection data clusters in each industry, and obtain the center points of the new consumer protection data clusters in each industry. For any industry, according to the formula:

[0034] ;

[0035] in, It is the center point of the new cluster of consumer protection data in the current industry. is the number of clustered data in the center of the original consumer protection data cluster in the current industry, It is the original consumer protection data cluster center of the current industry. The number of new data for industry complaint texts, A new data cluster center for current industry complaint texts.

[0036] According to the above technical solution, the calculation of the similarity between the extracted unidentified keywords and the keywords in the identified keyword library in step S300 includes the following contents:

[0037] When there are unlabeled keywords in the complaint text obtained in real time, set the delay analysis time and obtain all complaint text data during the delay time, retrieve the keyword existence information from the obtained complaint text data, count the number of times the keyword appears, and then calculate the similarity between the keyword and the identified keyword. With the identified keywords , according to the formula:

[0038] ;

[0039] in, Unidentified keywords appear in the complaint text obtained The number of complaint texts, is the number of complaint texts containing the identified keyword y in the obtained complaint texts, The unidentified keywords appear in the complaint text obtained With the identified keywords The number of complaint texts, For the simultaneous occurrence of unidentified keywords With the identified keywords The complaint document number, Complaint text No keywords identified The similarity weight of Complaint text Keywords identified in Similarity weights; measure the similarity between keywords based on the possibility of simultaneous occurrence of keywords and the similarity weights of their respective relative texts when they appear simultaneously, and analyze based on data to provide a more reliable data basis for the actual use and iterative optimization of the model, thereby improving the sustainability and scientificity of the model;

[0040] A consumer protection hotspot data analysis system based on a clustering algorithm using a consumer protection hotspot data analysis method based on a clustering algorithm described in the above technical solution, the system comprising: a data acquisition module, a data processing unit, a model iteration module, and an industry classification unit;

[0041] The data collection module is used to collect historical data of consumer protection and rights protection complaints and corresponding industry classification results, and is also used to obtain consumer protection and rights protection data complaint texts in real time for classification and clustering model iteration; the data processing unit is used to perform text processing on the acquired consumer protection and rights protection data, and obtain the word vectors of each complaint text and the similarity weight data of each keyword; the model iteration module is used to construct an industry classification model based on the historical data of consumer protection and rights protection complaints and the corresponding industry classification results, and is also used to update and iterate the industry classification model based on the selected new complaint text data; the industry classification unit uses the industry classification model to classify the consumer protection and rights protection data acquired in real time.

[0042] According to the above technical solution, the data acquisition module includes: a historical data acquisition unit and a real-time data acquisition unit;

[0043] The historical data collection unit is used to collect historical data of consumer protection and rights protection complaints and corresponding industry classification result data; the real-time data collection unit is used to collect consumer protection and rights protection complaint information in real time to perform industry classification of consumer protection and rights protection data.

[0044] According to the above technical solution, the model iteration module includes: a model pre-training unit and a model updating unit;

[0045] The model pre-training unit is used to perform initial clustering based on the historical data of consumer protection and rights protection complaints and the corresponding industry classification result data; the model update unit uses all complaint text data and corresponding industry classification results in the selected time unit to iteratively update the model when the number of time units in which the number of consumer protection and rights protection complaint information is greater than the quantity threshold is greater than the iteration threshold.

[0046] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0047] The present invention realizes the industry classification of consumer protection and rights protection data by pre-training the clustering model after processing the processed historical data of consumer protection and rights protection data, and accurately quantifies the complaint text by analyzing the word vector of the complaint text and the similarity weight between the keyword and the text, so as to provide a solid data foundation for the training of the subsequent industry classification model; when classifying the complaint text acquired in real time, the influence of new keywords on the industry classification results is fully considered, the similarity between keywords is introduced, and the mapping transformation between keywords is performed, so as to realize the generalization of the model; in the model iteration, the number of consumer protection and rights protection complaint information in each time unit is monitored in real time to monitor the occurrence of hot events, and then the surging complaint data of various industries is used to iterate the model, and the new keywords are absorbed into the keyword library, so as to realize the adaptive optimization and intelligent iteration of the model, so that the model can keep pace with the times and improve the universality of the model; and the constantly updated keyword library can also provide more standardized classification specifications for industry classification, and can also effectively avoid the problems of standard confusion and data randomness in the traditional model. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0049] Figure 1 It is a method flow chart of a method for analyzing consumer protection hotspot data based on a clustering algorithm of the present invention;

[0050] Figure 2 It is an organizational structure chart of a consumer protection hot spot data analysis system based on a clustering algorithm of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] See also Figure 1-Figure 2 , the present invention provides a technical solution:

[0053] like Figure 1 As shown, the present invention provides a method for analyzing consumer protection hotspot data based on a clustering algorithm, comprising the following analysis steps:

[0054] Step S100: Obtain historical data of consumer protection and rights protection complaints, pre-process the historical data complaint text, obtain word vectors of the historical data complaint text, and summarize the keywords in all word vectors into an identified keyword library;

[0055] Step S200: Performing initial clustering of the pre-processed historical data according to industry affiliation based on the historical data complaint text word vectors, and analyzing and calculating the center points of the consumer protection data clusters of each industry;

[0056] Step S300: Obtain consumer protection and rights protection complaint information in real time, extract complaint text and perform keyword extraction and word weight analysis, calculate the similarity between the extracted unidentified keywords and the keywords in the identified keyword library, map the unidentified keywords in the complaint text obtained in real time to identified keywords, and transform the word vector of the complaint text obtained in real time;

[0057] Step S400: classify the complaint text obtained in real time according to the converted word vectors, and provide feedback to the corresponding processing department according to the classification results;

[0058] Step S500: monitor the number of consumer protection and rights protection complaint information in real time and set an upper limit of the number threshold. When the number of consumer protection and rights protection complaint information is greater than the threshold, mark the time unit and select new data for clustering model iteration.

[0059] Step S100 includes the following contents:

[0060] The BERT algorithm is used to process the historical data complaint text. For any historical data complaint text, all keywords in the complaint text and the similarity weight between each keyword and the current complaint text are obtained, and a similarity weight threshold is set. When the similarity weight between the keyword and the complaint text is less than the threshold, the keyword is removed from the keywords filtered by the current complaint text. Then, the keyword data of all historical data complaint texts after preprocessing are summarized into an identified keyword library, and word vectors are generated for each historical data complaint text according to the similarity weights of all keywords in the keyword library in each historical data complaint text.

[0061] In the specific implementation, use public data sets such as WikiText-103 and BookCorpus to train the Bert model or directly use the pre-trained Bert public model to pre-process the text to generate text word vectors;

[0062] The word vector form of each historical data complaint text is: ;in, is the word vector of the current complaint text, is the similarity weight of all keywords in the keyword library in the current complaint text, is the number of keywords in the identified keyword library.

[0063] Step S200 includes the following contents:

[0064] Classify the complaint texts of each historical data according to the industry affiliation, and set the number of clusters according to the number of consumer protection and rights protection complaint industries. For any industry, according to the formula: Calculate the center point of each industry consumer protection data cluster; among them, is the center point of the current industry cluster, The number of complaint texts of the current industry historical data, The historical data complaint text number. Complaint text for historical data Word vectors;

[0065] In the specific implementation, the number of cluster center points of the clustering algorithm is set according to the preset number of industry types. Due to the multi-dimensional characteristics of word vectors and the range of similarity weight values ​​in word vectors, Therefore, there is no need to perform data normalization when performing cluster center point analysis;

[0066] Step S300 includes the following contents:

[0067] Obtain consumer protection complaint information in real time, process the complaint text using the BERT algorithm, obtain all keywords in the complaint text and the similarity weights between each keyword and the text;

[0068] Then calculate the similarity between the unidentified keywords and all the keywords in the identified keyword library, and then modify the similarity weight between each identified keyword in the complaint text word vector and the current complaint text. For any identified keyword in the complaint text word vector, the modified similarity weight calculation formula is as follows:

[0069] ;

[0070] in, is the corrected similarity weight of the currently identified keyword, is the similarity weight of the currently identified keyword before correction, Numbers for unidentified keywords in the complaint text. is the number of unidentified keywords in the complaint text. Unidentified keywords The similarity with the currently identified keywords, Unidentified keywords Similarity weight in the current complaint text;

[0071] In the specific implementation, for the unidentified keywords in the complaint text obtained in real time, the keyword is converted into an identified keyword based on the similarity data between the keyword and all the keywords in the identified keyword library. The complaint text word vector can be calculated according to the following formula:

[0072] ;

[0073] in, is the corrected complaint text word vector, is the word vector of the complaint text before correction, Numbers for unidentified keywords in the complaint text. is the number of unidentified keywords in the complaint text. Unidentified keywords Similarity weight in the current complaint text, Unidentified keywords A vector consisting of similarity data with all keywords in the identified keyword library; and The form of is consistent with the dimension of the complaint text word vector, where each parameter represents the similarity between the unidentified keyword and each identified keyword;

[0074] Step S400 includes the following contents:

[0075] Obtain the corrected word vector of the complaint text in the consumer protection and rights protection complaint information, and calculate the distance between the complaint text and the center point of the consumer protection data cluster of each industry. For the center point of any consumer protection data cluster in any industry, according to the formula:

[0076] ;

[0077] in, is the distance between the complaint text and the center point of the current industry consumer protection data cluster, Number the identified keywords. is the number of identified keywords, The keywords identified in the center of the current industry consumer protection data cluster Corresponding data, The keywords identified in the corrected complaint text word vector Corresponding data;

[0078] Set a distance threshold, select the center point of the industry consumer protection data cluster whose distance to the complaint text is less than the threshold, calculate the inverse of the distance between each center point and the complaint text and distribute it proportionally, and obtain the correlation factor between the selected industries and the complaint text. For any industry, according to the formula: ;in, is the correlation factor between the current industry and the complaint text, is the distance between the complaint text and the center point of the current industry consumer protection data cluster, The distance between the center point of the selected consumer protection data cluster in each industry and the complaint text;

[0079] Sort the industries according to the correlation factors from large to small, and report to the industry complaint handling department at regular intervals until receiving feedback from the department. Then classify the complaint text into the industry category corresponding to the feedback received;

[0080] In specific implementation, since the complaint text may involve multiple industries, the correlation factor data after the proportion distribution between the complaint text obtained in real time and each industry is calculated, and the data is distributed in order from large to small according to the correlation factor, to ensure that the complaint text can continue to be classified again when the initial judgment error occurs, to avoid the problem of consumer rights protection being delayed due to department allocation errors;

[0081] Step S500 includes the following contents:

[0082] Set the unit time and iteration threshold, count the number of consumer protection and rights protection complaints received in each unit time, and calculate the average number of consumer protection and rights protection complaints received in each unit time. and standard deviation , set the upper limit of quantity threshold When the number of consumer protection and rights protection complaint information received within a unit time is greater than the upper limit of the quantity threshold, the current time unit is marked. When the number of marked time units is greater than the iteration threshold, all complaint texts within the marked time and the final classification results of the text are obtained to perform model iteration;

[0083] Obtain all unidentified keywords in the complaint text, count the number of times each unidentified keyword appears in the complaint text and the sum of the similarity weights between the keyword and the complaint text when it appears, set the weight and threshold, and add the unidentified keywords whose sum of similarity weights is greater than the weight and threshold to the identified keyword library;

[0084] Use the selected new complaint text data for clustering, generate complaint text word vectors based on the expanded identified keyword library, and calculate the centers of the new complaint text data clusters in each industry, and then analyze the center offsets of the consumer protection data clusters in each industry, and obtain the center points of the new consumer protection data clusters in each industry. For any industry, according to the formula:

[0085] ;

[0086] in, It is the center point of the new cluster of consumer protection data in the current industry. is the number of clustered data in the center of the original consumer protection data cluster in the current industry, It is the original consumer protection data cluster center of the current industry. The number of new data for industry complaint texts, A new data cluster center for current industry complaint texts;

[0087] In the specific implementation, due to the sudden occurrence of consumer protection and rights protection events in various industries, the number of rights protection complaints surges in a short period of time, and hot events are usually accompanied by new hot keywords; therefore, the number of consumer protection and rights protection complaints received every day is monitored and counted in real time, and hot event analysis is conducted. When the number of consumer protection and rights protection complaints is greater than the upper limit of the standard deviation fluctuation range, it is determined that there is a hot event on that day that has caused a surge in the number of complaints, and all current complaint data and processing results are recorded; the iteration threshold is further set to days, when the number of complaints exceeds the upper limit of the standard deviation fluctuation threshold Days, use the recorded All complaint text data and processing results of the day are used to iterate the model;

[0088] The calculation of the similarity between the extracted unidentified keywords and the keywords in the identified keyword library in step S300 includes the following contents:

[0089] When there are unlabeled keywords in the complaint text obtained in real time, set the delay analysis time and obtain all complaint text data during the delay time, retrieve the keyword existence information from the obtained complaint text data, count the number of times the keyword appears, and then calculate the similarity between the keyword and the identified keyword. With the identified keywords , according to the formula:

[0090] ;

[0091] in, Unidentified keywords appear in the complaint text obtained The number of complaint texts, is the number of complaint texts containing the identified keyword y in the obtained complaint texts, The unidentified keywords appear in the complaint text obtained With the identified keywords The number of complaint texts, For the simultaneous occurrence of unidentified keywords With the identified keywords The complaint document number, Complaint text No keywords identified The similarity weight of Complaint text Keywords identified in Similarity weight of ;

[0092] In the specific implementation, when unidentified keywords appear, there are two situations; one is that new unidentified keywords are associated with hot events. By delaying the processing of complaint information, the complaint text data of related events is accumulated, and the number of occurrences of unidentified keywords and identified keywords in the complaint texts are analyzed. Then, the Bert model is used to analyze the similarity weight data of the two keywords in the complaint texts where the unidentified keywords and the identified keywords in the vocabulary appear at the same time, and then the similarity data between the unidentified keywords and the keywords in the vocabulary are calculated.

[0093] like Figure 2 As shown, the present invention also provides a consumer protection hot spot data analysis system based on a clustering algorithm, the system comprising: a data acquisition module, a data processing unit, a model iteration module, and an industry classification unit;

[0094] The data collection module is used to collect historical data of consumer protection and rights protection complaints and corresponding industry classification results, and is also used to obtain consumer protection and rights protection data complaint texts in real time for classification and clustering model iteration; the data processing unit is used to perform text processing on the acquired consumer protection and rights protection data, and obtain the word vectors of each complaint text and the similarity weight data of each keyword; the model iteration module is used to construct an industry classification model based on the historical data of consumer protection and rights protection complaints and the corresponding industry classification results, and is also used to update and iterate the industry classification model based on the selected new complaint text data; the industry classification unit uses the industry classification model to classify the consumer protection and rights protection data acquired in real time.

[0095] The data acquisition module includes: a historical data acquisition unit and a real-time data acquisition unit;

[0096] The historical data collection unit is used to collect historical data of consumer protection and rights protection complaints and corresponding industry classification result data; the real-time data collection unit is used to collect consumer protection and rights protection complaint information in real time to perform industry classification of consumer protection and rights protection data.

[0097] The model iteration module includes: a model pre-training unit and a model updating unit;

[0098] The model pre-training unit is used to perform initial clustering based on the historical data of consumer protection and rights protection complaints and the corresponding industry classification result data; the model update unit uses all complaint text data and corresponding industry classification results in the selected time unit to iteratively update the model when the number of time units in which the number of consumer protection and rights protection complaint information is greater than the quantity threshold is greater than the iteration threshold.

[0099] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0100] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for analyzing consumer protection hotspot data based on clustering algorithm, characterized in that: The method comprises the following analysis steps: Step S100: Obtain historical data of consumer protection and rights protection complaints, pre-process the historical data complaint text, obtain word vectors of the historical data complaint text, and summarize the keywords in all word vectors into an identified keyword library; Step S200: Performing initial clustering of the pre-processed historical data according to industry affiliation based on the historical data complaint text word vectors, and analyzing and calculating the center points of the consumer protection data clusters of each industry; Step S300: Obtain consumer protection and rights protection complaint information in real time, extract complaint text and perform keyword extraction and word weight analysis, calculate the similarity between the extracted unidentified keywords and the keywords in the identified keyword library, map the unidentified keywords in the complaint text obtained in real time to identified keywords, and transform the word vector of the complaint text obtained in real time; Step S400: classify the complaint text obtained in real time according to the converted word vectors, and provide feedback to the corresponding processing department according to the classification results; Step S500: monitor the number of consumer protection and rights protection complaint information in real time and set an upper limit of the number threshold. When the number of consumer protection and rights protection complaint information is greater than the threshold, mark the time unit and select new data for clustering model iteration; Wherein, step S300 includes the following contents: Obtain consumer protection complaint information in real time, process the complaint text using the BERT algorithm, obtain all keywords in the complaint text and the similarity weights between each keyword and the text; Then calculate the similarity between the unidentified keywords and all the keywords in the identified keyword library, and then modify the similarity weight between each identified keyword in the complaint text word vector and the current complaint text. For any identified keyword in the complaint text word vector, the modified similarity weight calculation formula is as follows: ； in, is the corrected similarity weight of the currently identified keyword, is the similarity weight of the currently identified keyword before correction, Numbers for unidentified keywords in the complaint text. is the number of unidentified keywords in the complaint text. Unidentified keywords The similarity with the currently identified keywords, Unidentified keywords Similarity weight in the current complaint text.

2. According to claim 1, a method for analyzing consumer protection hotspot data based on clustering algorithm is characterized in that: Step S100 includes the following contents: The BERT algorithm is used to process the historical data complaint text. For any historical data complaint text, all keywords in the complaint text and the similarity weight between each keyword and the current complaint text are obtained, and a similarity weight threshold is set. When the similarity weight between the keyword and the complaint text is less than the threshold, the keyword is removed from the keywords filtered by the current complaint text. Then, the keyword data of all historical data complaint texts after preprocessing are summarized into an identified keyword library, and word vectors are generated for each historical data complaint text according to the similarity weights of all keywords in the keyword library in each historical data complaint text. The word vector form of each historical data complaint text is: ;in, is the word vector of the current complaint text, is the similarity weight of all keywords in the keyword library in the current complaint text, is the number of keywords in the identified keyword library.

3. The method for analyzing consumer protection hotspot data based on clustering algorithm according to claim 1, characterized in that: Step S200 includes the following contents: Classify the complaint texts of each historical data according to the industry affiliation, and set the number of clusters according to the number of consumer protection and rights protection complaint industries. For any industry, according to the formula: Calculate the center point of each industry consumer protection data cluster; among them, is the center point of the current industry cluster, The number of complaint texts of the current industry historical data, The historical data complaint text number. Complaint text for historical data Word vectors.

4. The method for analyzing consumer protection hotspot data based on clustering algorithm according to claim 1, characterized in that: Step S400 includes the following contents: Obtain the corrected word vector of the complaint text in the consumer protection and rights protection complaint information, and calculate the distance between the complaint text and the center point of the consumer protection data cluster of each industry. For the center point of any consumer protection data cluster in any industry, according to the formula: ； in, is the distance between the complaint text and the center point of the current industry consumer protection data cluster, Number the identified keywords. is the number of identified keywords, The keywords identified in the center of the current industry consumer protection data cluster Corresponding data, The keywords identified in the corrected complaint text word vector Corresponding data; Set a distance threshold, select the center point of the industry consumer protection data cluster whose distance to the complaint text is less than the threshold, calculate the inverse of the distance between each center point and the complaint text and distribute it proportionally, and obtain the correlation factor between the selected industries and the complaint text. For any industry, according to the formula: ;in, is the correlation factor between the current industry and the complaint text, is the distance between the complaint text and the center point of the current industry consumer protection data cluster, The distance between the center point of the selected consumer protection data cluster in each industry and the complaint text; The industries are sorted from large to small according to the correlation factors, and feedback is given to the industry complaint handling department at regular intervals until feedback is received from the department. The complaint text is then classified into the industry category corresponding to the feedback received.

5. The method for analyzing consumer protection hotspot data based on clustering algorithm according to claim 2 is characterized in that: Step S500 includes the following contents: Set the unit time and iteration threshold, count the number of consumer protection and rights protection complaints received in each unit time, and calculate the average number of consumer protection and rights protection complaints received in each unit time. and standard deviation , set the upper limit of quantity threshold When the number of consumer protection and rights protection complaint information received within a unit time is greater than the upper limit of the quantity threshold, the current time unit is marked. When the number of marked time units is greater than the iteration threshold, all complaint texts within the marked time and the final classification results of the text are obtained to perform model iteration; Obtain all unidentified keywords in the complaint text, count the number of times each unidentified keyword appears in the complaint text and the sum of the similarity weights between the keyword and the complaint text when it appears, set the weight and threshold, and add the unidentified keywords whose sum of similarity weights is greater than the weight and threshold to the identified keyword library; Use the selected new complaint text data for clustering, generate complaint text word vectors based on the expanded identified keyword library, and calculate the centers of the new complaint text data clusters in each industry, and then analyze the center offsets of the consumer protection data clusters in each industry, and obtain the center points of the new consumer protection data clusters in each industry. For any industry, according to the formula: ； in, It is the center point of the new cluster of consumer protection data in the current industry. is the number of clustered data in the center of the original consumer protection data cluster in the current industry, It is the original consumer protection data cluster center of the current industry. The number of new data for industry complaint texts, A new data cluster center for current industry complaint texts.

6. The method for analyzing consumer protection hotspot data based on clustering algorithm according to claim 1, characterized in that: The calculation of the similarity between the extracted unidentified keywords and the keywords in the identified keyword library in step S300 includes the following contents: When there are unlabeled keywords in the complaint text obtained in real time, set the delay analysis time and obtain all complaint text data during the delay time, retrieve the keyword existence information from the obtained complaint text data, count the number of times the keyword appears, and then calculate the similarity between the keyword and the identified keyword. With the identified keywords , according to the formula: ； in, Unidentified keywords appear in the complaint text obtained The number of complaint texts, is the number of complaint texts containing the identified keyword y in the obtained complaint texts, The unidentified keywords appear in the complaint text obtained With the identified keywords The number of complaint texts, For the simultaneous occurrence of unidentified keywords With the identified keywords The complaint document number, Complaint text No keywords identified The similarity weight of Complaint text Keywords identified in The similarity weight of .

7. A consumer protection hotspot data analysis system based on a clustering algorithm using a consumer protection hotspot data analysis method based on a clustering algorithm as claimed in any one of claims 1 to 6, characterized in that: The system includes: a data acquisition module, a data processing unit, a model iteration module, and an industry classification unit; The data collection module is used to collect historical data of consumer protection and rights protection complaints and corresponding industry classification results, and is also used to obtain consumer protection and rights protection data complaint texts in real time for classification and clustering model iteration; the data processing unit is used to perform text processing on the acquired consumer protection and rights protection data, and obtain the word vectors of each complaint text and the similarity weight data of each keyword; the model iteration module is used to construct an industry classification model based on the historical data of consumer protection and rights protection complaints and the corresponding industry classification results, and is also used to update and iterate the industry classification model based on the selected new complaint text data; the industry classification unit uses the industry classification model to classify the consumer protection and rights protection data acquired in real time.

8. The consumer protection hotspot data analysis system based on clustering algorithm according to claim 7 is characterized in that: The data acquisition module includes: a historical data acquisition unit and a real-time data acquisition unit; The historical data collection unit is used to collect historical data of consumer protection and rights protection complaints and corresponding industry classification result data; the real-time data collection unit is used to collect consumer protection and rights protection complaint information in real time to perform industry classification of consumer protection and rights protection data.

9. The consumer protection hotspot data analysis system based on clustering algorithm according to claim 7 is characterized in that: The model iteration module includes: a model pre-training unit and a model updating unit; The model pre-training unit is used to perform initial clustering based on the historical data of consumer protection and rights protection complaints and the corresponding industry classification result data; the model update unit uses all complaint text data and corresponding industry classification results in the selected time unit to iteratively update the model when the number of time units in which the number of consumer protection and rights protection complaint information is greater than the quantity threshold is greater than the iteration threshold.

Images