A method and system for automatic data clustering management of public opinion events

Abstract

The present invention relates to the field of data management technology, and specifically to a method and system for automatic data clustering management of public opinion events, the management method comprising the following steps: capturing public opinion event storage records; analyzing the event content of each public opinion event storage record to obtain the correlation between each public opinion event storage record; analyzing the timeliness feedback value of each public opinion storage record; constructing a timeliness feedback model and training the timeliness feedback model; analyzing user feedback on target storage records to calculate a real-time timeliness feedback value; extracting the event content of the target storage record to obtain the probability of subsequent generation of similar public opinion event storage records, and calculating the expected feedback value of the target storage record after a period of time; analyzing the actual timeliness feedback value of the target storage record to determine whether to issue an abnormal reminder for the expected feedback value of the target storage record.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to a method and system for automatic data clustering management of public opinion events. Background Art

[0002] Public opinion events refer to events or topics that have attracted widespread attention in social public opinion. The development of public opinion events is often affected by factors such as media coverage and social media dissemination. They can spread rapidly and trigger heated discussions and controversies. Automatic data clustering refers to the process of automatically classifying and grouping large amounts of data using computer algorithms and technologies. Through automatic data clustering, the system can classify data points into different categories based on the characteristics and similarities of the data, thereby helping users better understand the structure and characteristics of the data set.

[0003] In the digital age, the speed and scope of the spread of public opinion events have been greatly improved. Similarly, people learn about new public opinion events at all times. Therefore, it becomes more important to control the timeliness of public opinion events. It is necessary to monitor the feedback of public opinion events among the masses. If the dynamic development of public opinion events cannot be grasped in time, it will lead to distortion of public opinion events, affect the misunderstanding of public opinion events and reduce trust, and also affect the decision-making judgment of the masses based on public opinion events. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for automatic data clustering management of public opinion events to solve the problems raised in the above-mentioned background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a data automatic clustering management method for public opinion events, the management method comprising the following steps:

[0006] Step S100: Establish a public opinion event storage system on a public opinion social platform, capture the public opinion event storage records generated in the public opinion event storage system; analyze the event content of each public opinion event storage record, obtain the correlation between each public opinion event storage record, and divide several public opinion event storage records and integrate the data; the public opinion event storage system stores the event content, record generation time, user comment volume and user forwarding volume of the public opinion event;

[0007] Step S200: obtaining user feedback on each public opinion event storage record, analyzing the timeliness feedback value of each public opinion storage record; constructing a timeliness feedback model, and using the timeliness feedback value of each public opinion event storage record to train the timeliness feedback model;

[0008] Step S300: Obtain a public opinion event storage record with the shortest record generation time from the current moment and set it as a target storage record, analyze the user feedback of the target storage record, and calculate the real-time feedback value; extract the event content of the target storage record, obtain the user feedback of the same type of public opinion event storage record, obtain the probability that the target storage record will subsequently generate the same type of public opinion event storage record, and calculate the expected feedback value of the target storage record after a period of time;

[0009] Step S400: After the period of time, the record generation status of the public opinion event storage record in the public opinion event storage system is obtained, and the actual timeliness feedback value of the target storage record is obtained by analysis. By comparing the difference between the actual timeliness feedback value of the target storage record and the expected feedback value, it is determined whether to issue an abnormal reminder for the expected feedback value of the target storage record.

[0010] Furthermore, step S100 includes the following steps:

[0011] Step S101: Count the number of occurrences of each word on a public opinion social platform, establish a word frequency database, and store each word and the corresponding number of occurrences;

[0012] Step S102: Obtain the event content stored in the i-th public opinion event storage record, divide the event content into information, and obtain a number of information blocks; arbitrarily select a certain information block, extract a number of words, and compare them with the words stored in the word frequency database; if there is a word in the several words that is the same as the words stored in the word frequency database, then select the word with the largest number of occurrences as the characteristic word of the certain information block, if there is no identical word, then set the certain information block as characteristic information; generate the feature set T of the i-th public opinion event storage record i =(W i ,M i ), where W i is the feature vocabulary set for storing records of the i-th public opinion event, M i A set of characteristic information stored for the i-th public opinion event;

[0013] Feature words are keywords extracted from the event content, which are used to determine the type of public opinion event, such as entertainment-related public opinion events, case-related public opinion events, etc. Feature information is detailed information corresponding to the public opinion event, such as the time and location of occurrence;

[0014] Step S103: Obtain a feature set of each public opinion event storage record in the public opinion event storage system, arbitrarily select two sets of feature sets, respectively obtain feature word sets in the two sets of feature sets, extract a feature word from each of the two sets of feature word sets for similarity comparison, and if the similarity exceeds a set similarity threshold, set the two compared feature words as the same word; by obtaining the number of the same words, determine whether the two public opinion events are of the same type, which is conducive to the subsequent calculation of the correlation between the public opinions;

[0015] Step S104: Obtain the feature information set in the two feature sets, and set the i-th feature information set M to be selected. i The ath feature information in the jth feature information set M j The similarity between the two feature information is obtained by comparing the bth feature information in (S ij ) ab , set the similarity threshold to β, if (S ij ) ab <β, then (S ij ) ab =0; for the case where the feature information has low similarity, it is directly regarded as unrelated, reducing the interference caused by the occasional similarity of some information, which is conducive to the accuracy of the subsequent calculation of the timeliness feedback value; according to the formula:

[0016]

[0017] Among them, Max() is the maximum value function, N i is the feature information set M i The number of feature information in, N j is the feature information set M j The number of feature information in; calculate the feature information set M i and feature information set M j The similarity S between ij ;

[0018] Step S105: Obtain the characteristic vocabulary set W of the storage record of the i-th public opinion event i and the characteristic vocabulary set W of the jth public opinion event storage record j , the number of words that have the same words between the two feature word sets is H ij ; Get the feature vocabulary set W i The number of words is H i and feature vocabulary set W j The vocabulary size is H j , the proportion of the same words is X ij =H ij / Max(H i ,H j );Set the quantity ratio threshold γ, if X ij <γ, then we get the correlation G between the i-th public opinion event storage record and the j-th public opinion event storage record ij =0; if X ij ≥γ, then we get the correlation G ij =X ij ×(1+S ij );

[0019] To determine whether there is a correlation between two public opinion events, we first need to confirm whether the feature words are the same, so as to determine whether they belong to the same type. After determining that they are public opinion events of the same type, we continue to combine the similarity between the feature information to obtain the final correlation.

[0020] Step S106: When G ij ≠0, the i-th public opinion event storage record and the j-th public opinion event storage record are classified as similar public opinion event storage records, and a similar public opinion event storage record set is generated.

[0021] Further, step S200 includes the following steps:

[0022] Step S201: Starting from the generation of the i-th public opinion event storage record, the number of user comments and user forwardings of the i-th public opinion event storage record is recorded every unit time; it is set that at time t, the number of user comments of the i-th public opinion event storage record is P t And the user forwarding amount is Q t ;

[0023] Step S202: Obtain the correlation between each public opinion event storage record and the i-th public opinion event storage record after the record generation time of the i-th public opinion event storage record; construct a timeliness feedback model:

[0024]

[0025] Where y is a constant coefficient, c is the total number of public opinion event storage records currently stored in the public opinion event storage system, k is an integer and k<i, G ik is the correlation between the i-th public opinion event storage record and the k-th public opinion event storage record; the timeliness feedback value (F i ) t ; The timeliness feedback value is affected by intuitive factors such as the number of user comments, forwarding volume and duration, as well as other related public opinion events before;

[0026] Step S203: obtaining the user comment amount and user forwarding amount of the i-th public opinion event storage record at different times, inputting the user comment amount and user forwarding amount at different times into the timeliness feedback model, and obtaining the timeliness feedback value of the i-th public opinion event storage record at different times; obtaining the difference in the total amount of data between any two adjacent times, the total amount of data being the sum of the user comment amount and the user forwarding amount, judging whether the change trend of the difference in the total amount of data is consistent with the change trend of the timeliness feedback value, and obtaining several values ​​of the constant coefficient y;

[0027] Step S204: Selecting a number of public opinion storage events from the public opinion event storage system, training the timeliness feedback model through the timeliness feedback values ​​and the total amount of data at different times in the number of public opinion storage events, and confirming the value of the constant coefficient y;

[0028] In the timeliness feedback model, in the case of an uncertain constant coefficient y, the obtained timeliness feedback value is a value with y. In the same public opinion event storage record, because the correlation of other public opinion events is the same, the timeliness feedback value is only affected by user feedback data and duration. When the user's feedback data does not increase much over time, it means that the popularity of the public opinion event has decreased, so the timeliness feedback value should also decrease. Therefore, the constant coefficient y is needed to match the changes between the feedback data and the timeliness feedback value.

[0029] Further, step S300 includes the following steps:

[0030] Step S301: Set the public opinion event storage record with the shortest record generation time from the current moment as the target storage record, obtain the record generation time of the target storage record, and obtain the duration of the current target storage record as t now ; Get the number of user comments recorded in the current target storage record as P now And the user forwarding amount is Q now , according to the formula:

[0031]

[0032] Where y is a constant coefficient, k is an integer and k∈[1,c-1], c is the total number of public opinion event storage records currently stored in the public opinion event storage system, G k,now is the correlation between the kth public opinion event storage record and the target storage record; the real-time feedback value of the target storage record (F now ) tnow ;

[0033] Step S302: extract the event content stored in the target storage record, and extract the feature set T of the target storage record. now=(W now ,M now ), where W now A feature vocabulary set M is recorded for the target storage. now The feature information set recorded for the target storage; the feature vocabulary set W now Compare the feature word set with any other public opinion event storage record. If the two sets of feature word sets are the same, set the public opinion event storage record to be compared as the reference storage record of the target storage record;

[0034] Step S303: Obtain all reference storage records of the target storage record, and classify all reference storage records according to whether they belong to the same type of public opinion event storage record set to obtain several types of reference storage record sets, and set the number of records in the rth type of reference storage record set as U r ; Get the number of records in the public opinion event storage system whose correlation with the target storage record is not 0, which is U now , if 1+U now <U r , then set the rth type of reference storage record set as a valid set; count the number of valid sets in the several types of reference storage record sets, calculate the proportion of the number of valid sets, and obtain the probability that the target storage record will subsequently generate a storage record of the same type of public opinion event;

[0035] Step S304: Set the characteristic information set M of the target storage record now Perform similarity comparison with the feature information set of any public opinion event storage record in a valid set, select a public opinion event storage record with the greatest similarity, obtain the next public opinion event storage record after the public opinion event storage record with the greatest similarity in the valid set, obtain the record generation time interval of the two public opinion event storage records, and use it as the expected time interval for the subsequent generation of similar public opinion event storage records by the target storage record;

[0036] Step S305: Set the expected time interval to Δt now The probability of generating a similar public opinion event storage record subsequently with the target storage record is η now , according to the formula:

[0037]

[0038] The target storage record is calculated to be after the expected time interval Δt now The expected feedback value after now ) ex; The expected feedback value is a reflection of the subsequent changes in the target storage record, not for calculating an accurate time-effectiveness feedback value; it is conducive to the subsequent judgment of the accuracy of the prediction results.

[0039] Further, step S400 includes the following steps:

[0040] Step S401: When the expected time interval Δt has passed now After that, the real-time user comment volume and real-time user forwarding volume of the target storage record are obtained, and the target storage record is calculated to obtain the expected time interval Δt now The actual time feedback value after (F now ) ac ;

[0041] Step S402: Determine whether a new public opinion event storage record is generated after the target storage record in the public opinion event storage system; if a new public opinion event storage record exists, and [(F now ) tnow -(F now ) ac ]×[(F now ) tnow -(F now ) ex ]<0, where (F now ) ex The target storage records the expected time interval Δt. now If the expected feedback value after the target storage record is reached, an abnormal reminder is given for the expected feedback value of the target storage record;

[0042] Comparing the expected feedback value with the actual timeliness feedback value is a reflection of the relevant data of subsequent users of the target storage record. The expected feedback value is not for calculating an accurate feedback value, but a prediction of the subsequent development of the public opinion event. Taking public opinion event A as an example, if the subsequently generated public opinion events have no correlation with public opinion event A, then the actual timeliness feedback value of public opinion event A will be lower than the original timeliness feedback value. If the expected feedback value is higher than the original timeliness feedback value, it can be said that the prediction is wrong.

[0043] In order to better implement the above method, a data automatic clustering management system for public opinion events is also proposed. The management system includes a historical information analysis module, a timeliness feedback evaluation module, a real-time information analysis module, and an abnormal feedback module;

[0044] The historical information analysis module is used to establish a public opinion event storage system on a public opinion social platform, capture the public opinion event storage records generated in the public opinion event storage system; analyze the event content of each public opinion event storage record, obtain the correlation between each public opinion event storage record, and divide several public opinion event storage records and integrate the data;

[0045] The timeliness feedback evaluation module is used to obtain user feedback on each public opinion event storage record, analyze the timeliness feedback value of each public opinion storage record; build a timeliness feedback model, and use the timeliness feedback value of each public opinion event storage record to train the timeliness feedback model;

[0046] The real-time information analysis module is used to obtain the public opinion event storage record with the shortest record generation time from the current moment and set it as the target storage record, analyze the user feedback of the target storage record, and calculate the real-time feedback value; extract the event content of the target storage record, obtain the user feedback of the same type of public opinion event storage record, obtain the probability that the target storage record will subsequently generate the same type of public opinion event storage record, and calculate the expected feedback value of the target storage record after a period of time;

[0047] The abnormal feedback module is used to obtain the record generation status of the public opinion event storage record in the public opinion event storage system after a period of time, analyze and obtain the actual timeliness feedback value of the target storage record, and judge whether to issue an abnormal reminder for the expected feedback value of the target storage record by comparing the difference between the actual timeliness feedback value of the target storage record and the expected feedback value.

[0048] Further, the historical information analysis module includes a record capture unit and a record association analysis unit;

[0049] A record capture unit is used to establish a public opinion event storage system on a public opinion social platform and capture the public opinion event storage records generated in the public opinion event storage system; a record association analysis unit is used to analyze the event content of each public opinion event storage record, obtain the association between each public opinion event storage record, and divide several public opinion event storage records and integrate the data.

[0050] Furthermore, the timeliness feedback evaluation module includes a timeliness feedback calculation unit and a model building unit;

[0051] The timeliness feedback calculation unit is used to obtain user feedback on each public opinion event storage record and analyze the timeliness feedback value of each public opinion storage record; the model construction unit is used to construct a timeliness feedback model and train the timeliness feedback model using the timeliness feedback value of each public opinion event storage record.

[0052] Further, the real-time information analysis module includes a real-time feedback analysis unit and an expected feedback prediction unit;

[0053] The real-time feedback analysis unit is used to obtain the public opinion event storage record with the shortest record generation time from the current moment and set it as the target storage record, analyze the user feedback of the target storage record, and calculate the real-time feedback value; the expected feedback prediction unit is used to extract the event content of the target storage record, obtain the user feedback of the same type of public opinion event storage records, obtain the probability that the target storage record will subsequently generate the same type of public opinion event storage record, and calculate the expected feedback value of the target storage record after a period of time.

[0054] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: (1) The present invention analyzes the timeliness of public opinion events by constructing a timeliness feedback model, thereby avoiding adverse effects caused by the distortion of public opinion events, ensuring that users can grasp the latest public opinion events in a timely manner, and improving the accuracy of users' grasp of public opinion events; (2) The present invention takes into account the continuity of the development of public opinion events and analyzes the correlation between public opinion events, which is conducive to improving the accuracy of calculating the timeliness feedback value of public opinion events; (3) The present invention calculates the timeliness feedback value of public opinion events more accurately by analyzing the correlation between each public opinion event, and can judge the possibility of subsequent development of public opinion events based on the correlation, and can grasp the dynamic development of public opinion earlier, which is conducive to improving users' trust in public opinion events and facilitating users to obtain the latest situation and make relevant decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0056] Figure 1 It is a schematic diagram of the steps of a method for automatic clustering management of data for public opinion events;

[0057] Figure 2 The present invention is a structural diagram of an automatic data clustering management system for public opinion events. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] See also Figure 1 to Figure 2 The present invention provides a technical solution: a data automatic clustering management method for public opinion events, the management method comprising the following steps:

[0060] Step S100: establishing a public opinion event storage system on a public opinion social platform, capturing the public opinion event storage records generated in the public opinion event storage system; analyzing the event content of each public opinion event storage record, obtaining the correlation between each public opinion event storage record, dividing several public opinion event storage records and integrating the data;

[0061] Wherein, step S100 includes the following steps:

[0062] Step S101: Count the number of occurrences of each word on a public opinion social platform, establish a word frequency database, and store each word and the corresponding number of occurrences;

[0063] Step S102: Obtain the event content stored in the i-th public opinion event storage record, divide the event content into information, and obtain a number of information blocks; arbitrarily select a certain information block, extract a number of words, and compare them with the words stored in the word frequency database; if there is a word in the several words that is the same as the words stored in the word frequency database, then select the word with the largest number of occurrences as the characteristic word of the certain information block, if there is no identical word, then set the certain information block as characteristic information; generate the feature set T of the i-th public opinion event storage record i =(W i ,M i ), where W i is the feature vocabulary set for storing records of the i-th public opinion event, M i A set of characteristic information stored for the i-th public opinion event;

[0064] For example, information is extracted from a public opinion event about a traffic accident, and the characteristic vocabulary set obtained includes "traffic accident", "private car", and "injury", and the characteristic information set includes "xx month xx day", "in xx street", and "number of injured people x";

[0065] Step S103: obtaining a feature set of each public opinion event storage record in the public opinion event storage system, arbitrarily selecting two sets of feature sets, respectively obtaining feature word sets in the two sets of feature sets, respectively extracting a feature word from the two sets of feature word sets for similarity comparison, and if the similarity exceeds a set similarity threshold, setting the compared two feature words as the same word;

[0066] For example, the first set of feature vocabulary contains the vocabulary "traffic accident", and the second set of feature vocabulary contains the vocabulary "traffic accident". The two words are compared for similarity. Because the similarity threshold is exceeded, "traffic accident" and "traffic accident" can be set as the same vocabulary.

[0067] Step S104: Obtain the feature information set in the two feature sets, and set the i-th feature information set M to be selected. i The ath feature information in the jth feature information set M j The similarity between the two feature information is obtained by comparing the bth feature information in (S ij ) ab , set the similarity threshold to β, if (S ij ) ab <β, then (S ij ) ab =0; According to the formula:

[0068]

[0069] Among them, Max() is the maximum value function, N i is the feature information set M i The number of feature information in, N j is the feature information set M j The number of feature information in; calculate the feature information set M i and feature information set M j The similarity S between ij ;

[0070] For example, obtain the feature information set of another public opinion event storage record, which includes all the feature information in the above example and the cause of the accident; therefore, the similarity between the three pairs of feature information is 100%, the similarity of the cause of the accident is 0, and the similarity between the two sets of feature information is calculated to be 300% / 4=75%;

[0071] Step S105: Obtain the characteristic vocabulary set W of the storage record of the i-th public opinion event i and the characteristic vocabulary set W of the jth public opinion event storage record j , the number of words that have the same words between the two feature word sets is H ij ; Get the feature vocabulary set W i The number of words is H i and feature vocabulary set W j The vocabulary size is H j , the proportion of the same words is X ij =H ij / Max(H i ,H j );Set the quantity ratio threshold γ, if X ij <γ, then we get the correlation G between the i-th public opinion event storage record and the j-th public opinion event storage record ij =0; if X ij ≥γ, then we get the correlation G ij =X ij ×(1+S ij );

[0072] For example, in the above example, the characteristic words of the two public opinion event storage records are the same and the number is equal, so the proportion of the number of the same words is 100%; therefore, the correlation between the two public opinion event storage records is 1×(1+0.75)=1.75;

[0073] Step S106: When G ij ≠0, the i-th public opinion event storage record and the j-th public opinion event storage record are classified as similar public opinion event storage records, and a similar public opinion event storage record set is generated.

[0074] Step S200: obtaining user feedback on each public opinion event storage record, analyzing the timeliness feedback value of each public opinion storage record; constructing a timeliness feedback model, and using the timeliness feedback value of each public opinion event storage record to train the timeliness feedback model;

[0075] Wherein, step S200 includes the following steps:

[0076] Step S201: Starting from the generation of the i-th public opinion event storage record, the number of user comments and user forwardings of the i-th public opinion event storage record is recorded every unit time; it is set that at time t, the number of user comments of the i-th public opinion event storage record is P t And the user forwarding amount is Q t ;

[0077] Step S202: Obtain the correlation between each public opinion event storage record and the i-th public opinion event storage record after the record generation time of the i-th public opinion event storage record; construct a timeliness feedback model:

[0078]

[0079] Where y is a constant coefficient, c is the total number of public opinion event storage records currently stored in the public opinion event storage system, k is an integer and k<i, G ik is the correlation between the i-th public opinion event storage record and the k-th public opinion event storage record; the timeliness feedback value (F i )t ;

[0080] Example: Get the time duration of the distance record generation event to be 10 seconds, the number of user comments is 100, the number of user forwarding is 100, and there is no correlation with the previous public opinion event storage record; determine y = 2, therefore, the timeliness feedback value of the i-th public opinion event storage record is 10;

[0081] Step S203: obtaining the user comment amount and user forwarding amount of the i-th public opinion event storage record at different times, inputting the user comment amount and user forwarding amount at different times into the timeliness feedback model, and obtaining the timeliness feedback value of the i-th public opinion event storage record at different times; obtaining the difference in the total amount of data between any two adjacent times, the total amount of data being the sum of the user comment amount and the user forwarding amount, judging whether the change trend of the difference in the total amount of data is consistent with the change trend of the timeliness feedback value, and obtaining several values ​​of the constant coefficient y;

[0082] Step S204: Select several public opinion storage events from the public opinion event storage system, train the timeliness feedback model through the timeliness feedback values ​​and total data of the several public opinion storage events at different times, and confirm the value of the constant coefficient y.

[0083] Step S300: Obtain a public opinion event storage record with the shortest record generation time from the current moment and set it as a target storage record, analyze the user feedback of the target storage record, and calculate the real-time feedback value; extract the event content of the target storage record, obtain the user feedback of the same type of public opinion event storage record, obtain the probability that the target storage record will subsequently generate the same type of public opinion event storage record, and calculate the expected feedback value of the target storage record after a period of time;

[0084] Wherein, step S300 includes the following steps:

[0085] Step S301: Set the public opinion event storage record with the shortest record generation time from the current moment as the target storage record, obtain the record generation time of the target storage record, and obtain the duration of the current target storage record as t now ; Get the number of user comments recorded in the current target storage record as P now and the user forwarding amount is Q now , according to the formula:

[0086]

[0087] Where y is a constant coefficient, k is an integer and k∈[1,c-1], c is the total number of public opinion event storage records currently stored in the public opinion event storage system, G k,nowis the correlation between the kth public opinion event storage record and the target storage record; the real-time feedback value of the target storage record (F now ) tnow ;

[0088] Step S302: extract the event content stored in the target storage record, and extract the feature set T of the target storage record. now =(W now ,M now ), where W now A feature vocabulary set M is recorded for the target storage. now The feature information set recorded for the target storage; the feature vocabulary set W now Compare the feature word set with any other public opinion event storage record. If the two sets of feature word sets are the same, set the public opinion event storage record to be compared as the reference storage record of the target storage record;

[0089] Step S303: Obtain all reference storage records of the target storage record, and classify all reference storage records according to whether they belong to the same type of public opinion event storage record set to obtain several types of reference storage record sets, and set the number of records in the rth type of reference storage record set as U r ; Get the number of records in the public opinion event storage system whose correlation with the target storage record is not 0, which is U now , if 1+U now <U r , then set the rth type of reference storage record set as a valid set; count the number of valid sets in the several types of reference storage record sets, calculate the proportion of the number of valid sets, and obtain the probability that the target storage record will subsequently generate a storage record of the same type of public opinion event;

[0090] Step S304: Set the characteristic information set M of the target storage record now Perform similarity comparison with the feature information set of any public opinion event storage record in a valid set, select a public opinion event storage record with the greatest similarity, obtain the next public opinion event storage record after the public opinion event storage record with the greatest similarity in the valid set, obtain the record generation time interval of the two public opinion event storage records, and use it as the expected time interval for the subsequent generation of similar public opinion event storage records by the target storage record;

[0091] Step S305: Set the expected time interval to Δt now The probability of generating a similar public opinion event storage record subsequently with the target storage record is η now , according to the formula:

[0092]

[0093] The target storage record is calculated to be after the expected time interval Δt now The expected feedback value after now ) ex ;

[0094] For example: suppose the real-time feedback value of the current public opinion event storage record is 10; according to the record generation situation in the storage record of the same type of public opinion event, the probability of subsequent generation of the storage record of the same type of public opinion event is 70%; after the expected interval of 10s, the expected feedback value is 10+(0.4×5)=12;

[0095] Step S400: after the period of time, obtaining the record generation status of the public opinion event storage record in the public opinion event storage system, analyzing and obtaining the actual timeliness feedback value of the target storage record, and determining whether to issue an abnormal reminder for the expected feedback value of the target storage record by comparing the difference between the actual timeliness feedback value of the target storage record and the expected feedback value;

[0096] Wherein, step S400 includes the following steps:

[0097] Step S401: When the expected time interval Δt has passed now After that, the real-time user comment volume and real-time user forwarding volume of the target storage record are obtained, and the target storage record is calculated to obtain the expected time interval Δt now The actual time feedback value after (F now ) ac ;

[0098] Step S402: Determine whether a new public opinion event storage record is generated after the target storage record in the public opinion event storage system; if a new public opinion event storage record exists, and Among them, (F now ) ex The target storage records the expected time interval Δt. now If the expected feedback value after the target storage record is reached, an abnormal reminder is given for the expected feedback value of the target storage record;

[0099] Example: According to the above example, the expected feedback value is 12, and the real-time feedback value is 10; after 10 seconds, the actual feedback value is 15, because (12-10)×(15-10)>0, which means that the prediction of the expected feedback value is correct.

[0100] A data automatic clustering management system for public opinion events, the management system includes a historical information analysis module, a timeliness feedback evaluation module, a real-time information analysis module, and an abnormal feedback module;

[0101] The historical information analysis module is used to establish a public opinion event storage system on a public opinion social platform, capture the public opinion event storage records generated in the public opinion event storage system; analyze the event content of each public opinion event storage record, obtain the correlation between each public opinion event storage record, and divide several public opinion event storage records and integrate the data;

[0102] The timeliness feedback evaluation module is used to obtain user feedback on each public opinion event storage record, analyze the timeliness feedback value of each public opinion storage record; build a timeliness feedback model, and use the timeliness feedback value of each public opinion event storage record to train the timeliness feedback model;

[0103] The real-time information analysis module is used to obtain the public opinion event storage record with the shortest record generation time from the current moment and set it as the target storage record, analyze the user feedback of the target storage record, and calculate the real-time feedback value; extract the event content of the target storage record, obtain the user feedback of the same type of public opinion event storage record, obtain the probability that the target storage record will subsequently generate the same type of public opinion event storage record, and calculate the expected feedback value of the target storage record after a period of time;

[0104] The abnormal feedback module is used to obtain the record generation status of the public opinion event storage record in the public opinion event storage system after a period of time, analyze and obtain the actual timeliness feedback value of the target storage record, and judge whether to issue an abnormal reminder for the expected feedback value of the target storage record by comparing the difference between the actual timeliness feedback value of the target storage record and the expected feedback value.

[0105] Wherein, the historical information analysis module includes a record capture unit and a record association analysis unit;

[0106] A record capture unit is used to establish a public opinion event storage system on a public opinion social platform and capture the public opinion event storage records generated in the public opinion event storage system; a record association analysis unit is used to analyze the event content of each public opinion event storage record, obtain the association between each public opinion event storage record, and divide several public opinion event storage records and integrate the data.

[0107] Among them, the timeliness feedback evaluation module includes a timeliness feedback calculation unit and a model building unit;

[0108] The timeliness feedback calculation unit is used to obtain user feedback on each public opinion event storage record and analyze the timeliness feedback value of each public opinion storage record; the model construction unit is used to construct a timeliness feedback model and train the timeliness feedback model using the timeliness feedback value of each public opinion event storage record.

[0109] Wherein, the real-time information analysis module includes a real-time feedback analysis unit and an expected feedback prediction unit;

[0110] The real-time feedback analysis unit is used to obtain the public opinion event storage record with the shortest record generation time from the current moment and set it as the target storage record, analyze the user feedback of the target storage record, and calculate the real-time feedback value; the expected feedback prediction unit is used to extract the event content of the target storage record, obtain the user feedback of the same type of public opinion event storage records, obtain the probability that the target storage record will subsequently generate the same type of public opinion event storage record, and calculate the expected feedback value of the target storage record after a period of time.

[0111] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0112] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for automatic data clustering management of public opinion events, characterized in that: The management method comprises the following steps: Step S100: establishing a public opinion event storage system on a public opinion social platform, capturing the public opinion event storage records generated in the public opinion event storage system; analyzing the event content of each public opinion event storage record, obtaining the correlation between each public opinion event storage record, dividing several public opinion event storage records and integrating the data; Step S200: obtaining user feedback on each public opinion event storage record, analyzing the timeliness feedback value of each public opinion storage record; constructing a timeliness feedback model, and using the timeliness feedback value of each public opinion event storage record to train the timeliness feedback model; Step S300: Obtain a public opinion event storage record with the shortest record generation time from the current moment and set it as a target storage record, analyze the user feedback of the target storage record, and calculate the real-time feedback value; extract the event content of the target storage record, obtain the user feedback of the same type of public opinion event storage record, obtain the probability that the target storage record will subsequently generate the same type of public opinion event storage record, and calculate the expected feedback value of the target storage record after a period of time; Step S400: After the period of time, the record generation status of the public opinion event storage record in the public opinion event storage system is obtained, and the actual timeliness feedback value of the target storage record is obtained by analysis. By comparing the difference between the actual timeliness feedback value of the target storage record and the expected feedback value, it is determined whether to issue an abnormal reminder for the expected feedback value of the target storage record.

2. According to claim 1, a method for automatic clustering management of public opinion events, characterized in that: The step S100 includes the following steps: Step S101: Count the number of occurrences of each word on a public opinion social platform, establish a word frequency database, and store each word and the corresponding number of occurrences; Step S102: Obtain the event content stored in the i-th public opinion event storage record, divide the event content into information, and obtain a number of information blocks; arbitrarily select a certain information block, extract a number of words, and compare them with the words stored in the word frequency database; if there is a word in the several words that is the same as the words stored in the word frequency database, then select the word with the largest number of occurrences as the characteristic word of the certain information block, if there is no identical word, then set the certain information block as characteristic information; generate the feature set T of the i-th public opinion event storage record i =(W i ,M i ), where W i is the feature vocabulary set for storing records of the ith public opinion event, M i A set of characteristic information stored for the i-th public opinion event; Step S103: obtaining a feature set of each public opinion event storage record in the public opinion event storage system, arbitrarily selecting two sets of feature sets, respectively obtaining feature word sets in the two sets of feature sets, respectively extracting a feature word from the two sets of feature word sets for similarity comparison, and if the similarity exceeds a set similarity threshold, setting the compared two feature words as the same word; Step S104: Obtain the feature information set in the two feature sets, and set the i-th feature information set M to be selected. i The ath feature information in the jth feature information set M j The similarity between the two feature information is obtained by comparing the bth feature information in (S ij ) ab , set the similarity threshold to β, if (S ij ) ab <β, then (S ij ) ab =0; According to the formula: Among them, Max() is the maximum value function, N i is the feature information set M i The number of feature information in j is the feature information set M j The number of feature information in; calculate the feature information set M i and feature information set M j The similarity S between ij ; Step S105: Obtain the characteristic vocabulary set W of the storage record of the i-th public opinion event i and the characteristic vocabulary set W of the jth public opinion event storage record j , the number of words that have the same words between the two feature word sets is H ij ; Get the feature vocabulary set W i The number of words is H i and feature vocabulary set W j The vocabulary size is H j , the proportion of the same words is X ij =H ij / Max(H i ,H j );Set the quantity ratio threshold γ, if X ij <γ, then we get the correlation G between the i-th public opinion event storage record and the j-th public opinion event storage record ij =0; if X ij ≥γ, then we get the correlation G ij =X ij ×(1+S ij ); Step S106: When G ij ≠0, the i-th public opinion event storage record and the j-th public opinion event storage record are classified as similar public opinion event storage records, and a similar public opinion event storage record set is generated.

3. According to claim 2, a method for automatic clustering management of public opinion events, characterized in that: The step S200 includes the following steps: Step S201: Starting from the generation of the i-th public opinion event storage record, the number of user comments and user forwardings of the i-th public opinion event storage record is recorded every unit time; it is set that at time t, the number of user comments of the i-th public opinion event storage record is P t And the user forwarding amount is Q t ; Step S202: Obtain the correlation between each public opinion event storage record and the i-th public opinion event storage record after the record generation time of the i-th public opinion event storage record; construct a timeliness feedback model: Where y is a constant coefficient, c is the total number of public opinion event storage records currently stored in the public opinion event storage system, k is an integer and k<i, G ik is the correlation between the i-th public opinion event storage record and the k-th public opinion event storage record; the timeliness feedback value (F i ) t ; Step S203: obtaining the user comment amount and user forwarding amount of the i-th public opinion event storage record at different times, inputting the user comment amount and user forwarding amount at different times into the timeliness feedback model, and obtaining the timeliness feedback value of the i-th public opinion event storage record at different times; obtaining the difference in the total amount of data between any two adjacent times, the total amount of data being the sum of the user comment amount and the user forwarding amount, judging whether the change trend of the difference in the total amount of data is consistent with the change trend of the timeliness feedback value, and obtaining several values ​​of the constant coefficient y; Step S204: Select several public opinion storage events from the public opinion event storage system, train the timeliness feedback model through the timeliness feedback values ​​and total data of the several public opinion storage events at different times, and confirm the value of the constant coefficient y.

4. The method for automatic data clustering management of public opinion events according to claim 3 is characterized by: The step S300 includes the following steps: Step S301: Set the public opinion event storage record with the shortest record generation time from the current moment as the target storage record, obtain the record generation time of the target storage record, and obtain the duration of the current target storage record as t now ; Get the number of user comments recorded in the current target storage record as P now And the user forwarding amount is Q now , according to the formula: Where y is a constant coefficient, k is an integer and k∈[1,c-1], c is the total number of public opinion event storage records currently stored in the public opinion event storage system, G k,now is the correlation between the kth public opinion event storage record and the target storage record; the real-time feedback value of the target storage record is calculated Step S302: extract the event content stored in the target storage record, and extract the feature set T of the target storage record. now =(W now ,M now ), where W now A feature vocabulary set M is recorded for the target storage. now The feature information set recorded for the target storage; the feature vocabulary set W now Compare the feature word set with any other public opinion event storage record. If the two sets of feature word sets are the same, set the public opinion event storage record to be compared as the reference storage record of the target storage record; Step S303: Obtain all reference storage records of the target storage record, and classify all reference storage records according to whether they belong to the same type of public opinion event storage record set to obtain several types of reference storage record sets, and set the number of records in the rth type of reference storage record set as U r ; Get the number of records in the public opinion event storage system whose correlation with the target storage record is not 0, which is U now , if 1+U now <U r , then set the rth type of reference storage record set as a valid set; count the number of valid sets in the several types of reference storage record sets, calculate the proportion of the number of valid sets, and obtain the probability that the target storage record will subsequently generate a storage record of the same type of public opinion event; Step S304: Set the characteristic information set M of the target storage record now Perform similarity comparison with the feature information set of any public opinion event storage record in a valid set, select a public opinion event storage record with the greatest similarity, obtain the next public opinion event storage record after the public opinion event storage record with the greatest similarity in the valid set, obtain the record generation time interval of the two public opinion event storage records, and use it as the expected time interval for the subsequent generation of similar public opinion event storage records by the target storage record; Step S305: Set the expected time interval to Δt now The probability of generating a similar public opinion event storage record subsequently with the target storage record is η now , according to the formula: The target storage record is calculated to be after the expected time interval Δt now The expected feedback value after now ) ex .

5. According to claim 4, a method for automatic clustering management of public opinion events, characterized in that: The step S400 includes the following steps: Step S401: When the expected time interval Δt has passed now After that, the real-time user comment volume and real-time user forwarding volume of the target storage record are obtained, and the target storage record is calculated to obtain the expected time interval Δt now The actual time feedback value after (F now ) ac ; Step S402: Determine whether a new public opinion event storage record is generated after the target storage record in the public opinion event storage system; if a new public opinion event storage record exists, and Among them, (F now ) ex The target storage records the expected time interval Δt. now If the expected feedback value after the target storage record is obtained, an abnormal reminder will be given for the expected feedback value of the target storage record.

6. A data automatic clustering management system, used to execute a data automatic clustering management method for public opinion events as described in any one of claims 1 to 5, characterized in that: The management system includes a historical information analysis module, a timeliness feedback evaluation module, a real-time information analysis module, and an abnormality feedback module; The historical information analysis module is used to establish a public opinion event storage system on a public opinion social platform and capture the public opinion event storage records generated in the public opinion event storage system; Analyze the event content of each public opinion event storage record, obtain the correlation between each public opinion event storage record, and divide several public opinion event storage records and integrate the data; The timeliness feedback evaluation module is used to obtain user feedback on each public opinion event storage record and analyze the timeliness feedback value of each public opinion storage record; Construct a time-effect feedback model and train it using the time-effect feedback values ​​stored in each public opinion event record; The real-time information analysis module is used to obtain a public opinion event storage record with the shortest record generation time from the current moment and set it as a target storage record, analyze the user feedback of the target storage record, and calculate the real-time feedback value; extract the event content of the target storage record, obtain the user feedback of the same type of public opinion event storage record, obtain the probability that the target storage record will subsequently generate the same type of public opinion event storage record, and calculate the expected feedback value of the target storage record after a period of time; The abnormal feedback module is used to obtain the record generation status of the public opinion event storage record in the public opinion event storage system after the period of time, analyze and obtain the actual timeliness feedback value of the target storage record, and judge whether to issue an abnormal reminder for the expected feedback value of the target storage record by comparing the difference between the actual timeliness feedback value of the target storage record and the expected feedback value.

7. The data automatic clustering management system according to claim 6, characterized in that: The historical information analysis module includes a record capture unit and a record association analysis unit; The record capture unit is used to establish a public opinion event storage system on a public opinion social platform, and capture the public opinion event storage records generated in the public opinion event storage system; The record association analysis unit is used to analyze the event content of each public opinion event storage record, obtain the association between each public opinion event storage record, and divide several public opinion event storage records and integrate the data.

8. The data automatic clustering management system according to claim 6, characterized in that: The timeliness feedback evaluation module includes a timeliness feedback calculation unit and a model building unit; The timeliness feedback calculation unit is used to obtain user feedback on each public opinion event storage record and analyze the timeliness feedback value of each public opinion storage record; the model construction unit is used to construct a timeliness feedback model and train the timeliness feedback model using the timeliness feedback value of each public opinion event storage record.

9. The data automatic clustering management system according to claim 6, characterized in that: The real-time information analysis module includes a real-time feedback analysis unit and an expected feedback prediction unit; The real-time feedback analysis unit is used to obtain the public opinion event storage record with the shortest record generation time from the current moment and set it as the target storage record, analyze the user feedback of the target storage record, and calculate the real-time feedback value; the expected feedback prediction unit is used to extract the event content of the target storage record, obtain the user feedback of the same type of public opinion event storage records, obtain the probability that the target storage record will subsequently generate the same type of public opinion event storage record, and calculate the expected feedback value of the target storage record after a period of time.

Images