Large Model-Based Event Automation Evaluation System and Method

Abstract

The present invention discloses an event automated evaluation system and method based on a large model, which relates to the technical field of big data analysis. The present invention determines credible events, determines characteristic events and predicts the severity of the events through a characteristic event automated evaluation model, obtains information parameters of the characteristic events, inputs them into an information prediction system, determines the information index caused by the characteristic events, and determines the characteristic information corresponding to each characteristic record; draws a graph of the actual information index change of the evaluation records, determines abnormal records, and calculates the weight of the characteristic records according to the abnormality rate of the characteristic records and the number of evaluation records of the characteristic records; collects key information of real-time characteristic events, determines the characteristic records corresponding to the real-time characteristic events, calculates the real-time sampling period, and determines the sampling status according to the real-time sampling period, so as to improve the self-adaptability of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of big data analysis, and specifically to an event automated evaluation system and method based on a large model. Background Art

[0002] In the current highly developed information era, social networks, news media, and public platforms continuously generate a large amount of multi-modal event data such as text, images, and videos, resulting in a significant increase in the complexity, propagation speed, and cross-modal relevance of events. Traditional event evaluation methods rely on manual experience or rule systems based on keyword matching, and have defects such as low efficiency, strong subjectivity, and limited coverage. The dynamic evolution characteristics of emergencies make it difficult for models based on fixed-period sampling to respond in a timely manner, resulting in lagging classification and grading, and affecting the timeliness of emergency decision-making;

[0003] Therefore, it is necessary to find an appropriate time to collect event information, improve the intelligent level of event monitoring and response, and enhance network information management capabilities. Summary of the Invention

[0004] The purpose of the present invention is to provide an event automated evaluation system and method based on a large model to solve the problems raised in the prior art.

[0005] To solve the above technical problems, the present invention provides the following technical solution: An event automated evaluation method based on a large model, the method comprising:

[0006] Step S100: Legally monitor each social platform, obtain the information content published during the monitoring period, generate an event for the published information content through natural language processing, summarize relevant events, obtain the information content with videos in the relevant event set, collect the EXIF metadata of the information content and perform similarity matching with key information to determine whether the information content is credible information, calculate the credibility score of the event based on the number of information content without videos, the number of credible information, and the similarity matching score of the credible information in the relevant events, and determine credible events;

[0007] Step S200: Collect the feature data of the credible events, determine the feature events and predict the severity of the feature events through a feature event automated evaluation model, obtain the information parameters of the feature events, input them into an information prediction system, judge the information index caused by the feature events, establish an information index change graph, and generate an evaluation record at the same time;

[0008] Step S300: Set an information index threshold, plot the information index threshold as a horizontal threshold line, and insert it into the information index change graph. Determine the warning evaluation record. Set the time point corresponding to the first intersection point of the information index change graph and the horizontal threshold line as the first time point, summarize to obtain the first record set. Set the time point corresponding to the last intersection point of the information index change graph and the horizontal threshold line as the second time point, summarize to obtain the second record set, and calculate to obtain the third record set, where the third record set is the intersection of the first record set and the second record set;

[0009] Step S400: Calculate the sets of the first and second feature records according to the first record set, the second record set, and the third record set. Collect the key information of the warning evaluation record. Set the key information that exceeds the occurrence frequency threshold as the feature information, and determine the feature information corresponding to each feature record;

[0010] Step S500: Obtain the evaluation record of the information prediction system in a certain historical sampling period, plot the actual information index change graph of the evaluation record, determine the abnormal record, and calculate the weight of the feature record according to the abnormal rate of the feature record and the number of evaluation records of the feature record;

[0011] Step S600: Collect the key information of the real-time feature event, determine the feature record corresponding to the real-time feature event, preset a sampling time period, where the sampling time period is less than the sampling period. Count the number of each feature record in the sampling time period, calculate the real-time sampling period according to the number of feature records and the weight of the feature record, and judge the sampling status according to the real-time sampling period.

[0012] Further, step S100 includes:

[0013] Step S101: Legally monitor each social platform, obtain the information content published within a certain monitoring period, extract the key information of a piece of information content through natural language processing, and generate an event. Summarize the information content of related events according to the content of the event;

[0014] Step S102: In the information content set of a certain related event, obtain the information content with videos, and collect the EXIF metadata of the information content. Perform a similarity match between the key information of the information content and the EXIF metadata, calculate the similarity match score, set a similarity match score threshold. If the similarity match score of the information content exceeds the similarity match score threshold, set the information content as trustworthy information;

[0015] Step S103: Count the number of information content without videos and trustworthy information in a certain related event within the monitoring period, and calculate the trust score of the event according to the following formula:

[0016] ;

[0017] Among them, A represents the credibility score of an event, B represents the number of information contents without videos, and D b represents the similarity matching score of the b-th piece of credible information, and a B represents the weight of the information content without videos, and a D represents the weight of the credible information, and C represents the number of credible information;

[0018] Step S104: Set a credibility score threshold. When the credibility score of an event is greater than the credibility score threshold, mark the event as a credible event;

[0019] Among them, the key information includes the location, time, core elements of the event occurrence, etc., and the EXIF metadata includes the shooting time information, geographical location information, shooting device information, shooting parameter information, etc.;

[0020] By cross-verifying the text information through videos and EXIF data, false contents are effectively eliminated, and the credibility is quantified using mathematical formulas, realizing the automated evaluation of information credibility;

[0021] Integrating social platforms, multimedia content, and metadata enhances the comprehensiveness of the analysis. The automated process reduces subjective judgment errors, making the system more objective and stable.

[0022] Further, step S200 includes:

[0023] Step S201: Collect the feature data of the credible event and input it into the feature event automated evaluation model to determine the credible event as a feature event and predict the severity;

[0024] Step S202: Obtain the information parameters of the feature event and input them together with the predicted severity into the information prediction system to judge the information index caused by the feature event, establish an information index change graph, and generate an evaluation record at the same time;

[0025] Through the feature data of the credible event and the automated evaluation model, it is possible to quickly identify whether an event belongs to a feature event and predict its severity. This data-driven analysis method significantly improves the accuracy and efficiency of event classification and early warning, and reduces the delay and misjudgment caused by human intervention;

[0026] By introducing information parameters and combining with the predicted severity, further generate an information index change graph through the information prediction system to visualize the information fluctuation trend caused by the feature event. This not only enables managers to clearly grasp the information development trend but also can be used to formulate more targeted countermeasures;

[0027] A complete evaluation record will be generated in the method, including key data such as event characteristics, prediction results, and information indices, providing a basis for subsequent event analysis and system iterative optimization, and also facilitating the regulatory agency to conduct post-event review and policy evaluation.

[0028] Further, step S300 includes:

[0029] Step S301: Through the information prediction system, obtain historical evaluation records, set an information index threshold, collect the information index change graph of a certain historical evaluation record, draw the information index threshold as a horizontal threshold line, and insert it into the information index change graph. When there is an intersection point between the information index change graph and the horizontal threshold line, set the historical evaluation record as a warning evaluation record;

[0030] Step S302: Obtain the warning evaluation record, set the time point corresponding to the first intersection point of the information index change graph and the horizontal threshold line as the first time point, collect the first time point of the warning evaluation record, preset the characteristic first time point. If the first time point of a certain warning evaluation record is less than the characteristic first time point for comparison, set the warning evaluation record as the first record, summarize the first records, and obtain the set of the first records as P1;

[0031] Step S303: Set the time point corresponding to the last intersection point of the information index change graph and the horizontal threshold line as the second time point, collect the second time point of the warning evaluation record, preset the characteristic second time point. If the second time point of a certain warning evaluation record is greater than the characteristic second time point for comparison, set the warning evaluation record as the second record, summarize the second records, and obtain the set of the second records as P2. Calculate the set of the third characteristic records as P3 = P1 ∩ P2;

[0032] By introducing the information index threshold line and analyzing the intersection points of the information index change graph and the threshold, the identification of potential high-risk events in historical evaluation records is effectively realized. This method can quickly lock the information anomaly inflection points, form a dynamic warning mechanism for similar events, and significantly improve the scientificity and forward-looking of the warning;

[0033] The comparison logics of the first time point and the second time point are respectively introduced, and further combined with the preset time characteristics to accurately capture the key nodes in the early and late stages of the event development. By calculating the intersection to obtain the set P3, the events that trigger warnings from the early stage and whose subsequent risks still persist can be screened out, providing a more valuable reference for management decisions;

[0034] This method not only relies on a single indicator for judgment, but also integrates a triple mechanism of threshold judgment + time point determination + historical data analysis to construct a more comprehensive, three-dimensional and intelligent system. This is of great significance for the system to handle complex events and improves the robustness and adaptability of the overall evaluation system.

[0035] Further, step S400 includes:

[0036] Step S401: Subtract the set of the third feature record from the sets of the first and second records respectively to obtain the sets of the first and second feature records;

[0037] Step S402: In a set of a certain feature record, collect a certain early warning evaluation record, extract the key information of the event in the early warning evaluation record, summarize the key information of the feature record, count the number of times a certain key information appears in the feature record, and calculate the appearance frequency of the key information as Z = Z1 / Z2, where Z1 represents the number of times the key information appears, and Z2 represents the total number of times the key information appears;

[0038] Step S403: Preset an appearance frequency threshold, set the key information exceeding the appearance frequency threshold as feature information, and summarize the feature information corresponding to a certain feature record;

[0039] By calculating the sets of the first and second feature records, the early warning records that only appear in the early stage or only in the later stage of the event are effectively separated. This differential analysis method helps to form a portrait of the phased characteristics of the event and improves the ability to understand and describe the event life cycle;

[0040] Introducing the extraction and frequency statistics of key information constructs a high-frequency feature recognition model based on statistical methods. By calculating the appearance frequency, it is possible to identify the truly representative high-frequency feature information from a large amount of historical data, providing key variables for subsequent event recognition and model optimization;

[0041] Setting the key information with an appearance frequency exceeding the threshold as "feature information" forms a set of representative and reusable information sets, realizing the continuous self-learning and intelligent evolution of the system.

[0042] Further, step S500 includes:

[0043] Step S501: Obtain the evaluation records of the information prediction system in a certain historical sampling period, collect the actual information index of a certain evaluation record, draw a change graph of the actual information index of the evaluation record, obtain the change graph of the information index of the evaluation record, combine the actual information index change graph and the information index change graph, calculate the error rate of the information index, preset an error rate threshold, and set the evaluation records exceeding the error rate threshold as abnormal records;

[0044] Step S502: Collect the key information of the evaluation records, match it with the feature information corresponding to each feature record, and determine the feature record corresponding to the evaluation record;

[0045] Step S503: Count the number of evaluation records H1 and the number of abnormal records H2 of a certain feature record in a certain historical sampling period, calculate the abnormal rate of the feature record as H2 / H1, summarize the abnormal rates of the feature record in all historical sampling periods, and calculate the average abnormal rate of the feature record;

[0046] Step S504: Calculate the weight of the feature record according to the following formula:

[0047] ;

[0048] where W represents the weight of the feature record, H represents the number of evaluation records of the feature record in all historical sampling periods, H' represents the total number of evaluation records in all historical sampling periods, K represents the average abnormal rate of the feature record, p1 represents the weight of the number of evaluation records, and p2 represents the weight of the average abnormal rate;

[0049] By comparing the actual information index change graph with the system-predicted information index change graph, calculating the error rate and setting the error rate threshold, abnormal evaluation records with large prediction deviations can be effectively screened out. This mechanism can not only improve the system's recognition ability for unstable evaluation samples, but also serve as an important basis for subsequent model optimization;

[0050] The system statistically calculates and averages the abnormal rates of feature records in multiple historical sampling periods to form an abnormal rate index of long-term performance, which has very practical significance for measuring the "stability" and "reliability" of a feature record;

[0051] Evaluating the two dimensions of activity and abnormal stability enables the feature record weight to have the characteristics of being quantifiable, adjustable, and interpretable, which helps to construct a feedback mechanism for risk control and model tuning.

[0052] Further, step S600 includes:

[0053] Step S601: Obtain a real-time event, calculate the real-time credibility score of the real-time event. When the real-time credibility score is greater than the credibility score threshold, mark the real-time event as a real-time credible event;

[0054] Step S602: Collect the feature data of real-time credible events and input them into the feature event automated evaluation model to determine that the real-time credible events are real-time feature events and predict the severity. Collect the key information of the real-time feature events and determine the feature records corresponding to the real-time feature events;

[0055] Step S603: Preset a sampling time period, where the sampling time period is less than the sampling period. Count the number of each feature record in the sampling time period and calculate the real-time sampling period according to the following formula:

[0056] ;

[0057] where, T2 u represents the real-time sampling period at the u-th sampling time, T represents the sampling period, T1 represents the real-time sampling period obtained in the previous calculation, S n represents the number of the n-th feature record, W n represents the weight of the n-th feature record, and m represents the total number of feature records and m = 3;

[0058] Step S604: When the real-time sampling period is greater than the sampling time period, then execute Step S603. When the real-time sampling period is less than or equal to the sampling time period, remind the staff to resample and analyze the event at the real-time sampling period;

[0059] The system calculates the real-time credibility score and compares it with the set threshold, and only processes the events with a score higher than the threshold. This design ensures that the system resources are mainly concentrated on the key events with high credibility and possible information impacts, improving the accuracy and processing efficiency of event recognition from the source;

[0060] After identifying the "real-time credible events", immediately extract their features and input them into the feature event evaluation model, which realizes the rapid identification, classification and impact prediction of events. At the same time, extract their key information for subsequent feature matching and analysis. This process realizes data-driven rapid judgment + intelligent evaluation, significantly compressing the response cycle;

[0061] Through the real-time sampling period calculation formula, the system dynamically adjusts the current sampling period according to the number and weight of different feature records in the sampling time period. This mechanism enables the system to automatically adjust the sampling frequency according to the event occurrence frequency and risk intensity, realizing the rhythm self-adaptability of the evaluation system and the optimization of data collection sensitivity;

[0062] Based on the comparison results of the sampling cycle and time period, it is intelligently judged whether it is necessary to immediately start re-sampling analysis. This mechanism ensures that the system does not miss the development of events at critical time points, thereby realizing full-process, continuous and dynamic monitoring and feedback of events, forming an intelligent closed-loop system of evaluation-sampling-optimization-response, and significantly improving the resilience and decision-making support efficiency of the entire social governance system.

[0063] In order to better implement the above method, an event automatic evaluation system based on a large model is also proposed. The system includes a trusted event module, an evaluation record module, a record classification module, a feature information module, a feature record weight module and a real-time calculation module;

[0064] Trusted event module: Through legal monitoring of various social platforms, obtain the information content released during the monitoring period, generate an event for the released information content through natural language processing, summarize related events, obtain the information content with videos in the related event set, collect the EXIF metadata of the information content and perform similarity matching with key information, determine whether the information content is trustworthy information, calculate the trustworthiness score of the event according to the number of information content without videos in the related events, the number of trustworthy information, and the similarity matching score of the trustworthy information, and determine the trustworthy event;

[0065] Evaluation record module: collects characteristic data of credible events, determines characteristic events and predicts the severity of characteristic events through the characteristic event automatic evaluation model, obtains information parameters of characteristic events, inputs them into the information prediction system, determines the information index caused by characteristic events, establishes an information index change graph, and generates an evaluation record at the same time;

[0066] Record classification module: setting an information index threshold, and drawing the information index threshold as a horizontal threshold line, and inserting it into the information index change graph, determining the early warning assessment record, setting the time point corresponding to the first intersection point of the information index change graph and the horizontal threshold line as the first time point, summarizing to obtain a first record set, setting the time point corresponding to the last intersection point of the information index change graph and the horizontal threshold line as the second time point, summarizing to obtain a second record set, and calculating to obtain a third record set, the third record set being the intersection of the first record set and the second record set;

[0067] Feature information module: calculates the first and second feature record sets according to the first record set, the second record set, and the third record set, collects key information of the early warning assessment record, sets the key information exceeding the frequency threshold as feature information, and determines the feature information corresponding to each feature record;

[0068] Feature Record Weight Module: Obtain the evaluation records of the information prediction system in a certain historical sampling period, draw a graph of the actual information index change of the evaluation records, determine the abnormal records, and calculate the weight of the feature records according to the abnormal rate of the feature records and the number of evaluation records of the feature records;

[0069] Real-time Calculation Module: Collect the key information of real-time feature events, determine the feature records corresponding to the real-time feature events, preset a sampling time period, where the sampling time period is less than the sampling period, count the number of each feature record in the sampling time period, calculate the real-time sampling period according to the number of feature records and the weight of the feature records, and judge the sampling status according to the real-time sampling period.

[0070] Furthermore, the Feature Record Weight Module includes an Abnormal Record Determination Unit and a Feature Record Weight Calculation Unit:

[0071] Abnormal Record Determination Unit: Obtain the evaluation records of the information prediction system in a certain historical sampling period, collect the actual information index of a certain evaluation record, draw a graph of the actual information index change of the evaluation record, obtain the information index change graph of the evaluation record, combine the actual information index change graph and the information index change graph, calculate the error rate of the information index, preset an error rate threshold, and set the evaluation records exceeding the error rate threshold as abnormal records;

[0072] Feature Record Weight Calculation Unit: Collect the key information of the evaluation records, match it with the feature information corresponding to each feature record, determine the feature records corresponding to the evaluation records, count the number of evaluation records and the number of abnormal records of a certain feature record in a certain historical sampling period, calculate the abnormal rate of the feature record, summarize the abnormal rates of the feature record in all historical sampling periods, and calculate the average abnormal rate of the feature record, and calculate the weight of the feature record.

[0073] Furthermore, the Real-time Calculation Module includes a Real-time Feature Record Determination Unit and a Real-time Sampling Period Calculation Unit:

[0074] Real-time Feature Record Determination Unit: Obtain a real-time event, calculate the real-time credibility score of the real-time event, when the real-time credibility score is greater than the credibility score threshold, mark the real-time event as a real-time credible event, collect the feature data of the real-time credible event, and input it into the feature event automated evaluation model to determine the real-time credible event as a real-time feature event and predict the severity, collect the key information of the real-time feature event, and determine the feature record corresponding to the real-time feature event

[0075] Calculation of real-time sampling period unit: preset a sampling time period, where the sampling time period is less than the sampling period. Count the number of each feature record in the sampling time period, calculate the real-time sampling period. When the real-time sampling period is less than or equal to the sampling time period, remind the staff to resample and analyze the event at the real-time sampling period.

[0076] Compared with the prior art, the beneficial effects of the present invention are as follows: By introducing mechanisms such as EXIF metadata comparison, information similarity matching, and video content judgment, a credible scoring model is constructed, and a credible threshold is set. Only credible events are retained as the objects for further processing, greatly improving the accuracy of information screening, reducing the misjudgment rate, and effectively coping with the risks brought by information overflow and false propagation.

[0077] The system designs an information index change graph + threshold line intersection recognition mechanism, which can automatically identify the warning inflection point of the event and generate a warning evaluation record, realizing the graphical and dynamic monitoring of the information development trend, and being more automated, real-time, and objective than the traditional manual judgment.

[0078] The system can perform difference set analysis on the first record, the second record, and the intersection record, and combine the statistics of the occurrence frequency of key information to screen out a set of high-impact feature information, thereby realizing the identification of the essential factors of the event and providing a data basis for model training and risk traceability.

[0079] Combining the event impact frequency and stability performance, calculate the feature weight value, realizing the self-feedback, self-evolution, and adaptive optimization ability of the evaluation system, and significantly improving the robustness and generalization ability of the system during long-term operation.

[0080] By dynamically calculating the sampling period, the adaptive adjustment of the system evaluation frequency is realized, enabling it to automatically speed up the response rhythm during the outbreak of high-frequency events, while maintaining resource optimization during the stable period of the event, demonstrating a high level of intelligent control ability. Brief Description of the Drawings

[0081] Figure 1 It is a schematic flowchart of the event automatic evaluation method based on the large model of the present invention;

[0082] Figure 2 It is a schematic structural diagram of the event automatic evaluation system based on the large model of the present invention. Detailed Embodiments

[0083] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0084] Please refer to Figure 1 , the present invention provides a technical solution: an event automated evaluation method based on a large model, the method comprising:

[0085] Step S100: By legally monitoring each social platform, obtain the information content published within a monitoring period, through natural language processing, generate an event for the published information content, summarize relevant events, obtain the information content with videos in the relevant event set, collect the EXIF metadata of the information content and perform similarity matching with the key information, determine whether the information content is credible information, and calculate the credibility score of the event according to the number of information content without videos, the number of credible information, and the similarity matching score of the credible information in the relevant events, and determine credible events;

[0086] Among them, step S100 includes:

[0087] Step S101: Legally monitor each social platform, obtain the information content published within a certain monitoring period, through natural language processing, extract the key information of a piece of information content, and generate an event, and summarize the information content of relevant events according to the content of the event;

[0088] Step S102: In the information content set of a certain relevant event, obtain the information content with videos, and collect the EXIF metadata of the information content, perform similarity matching between the key information of the information content and the EXIF metadata, calculate the similarity matching score, set a similarity matching score threshold, and if the similarity matching score of the information content exceeds the similarity matching score threshold, set the information content as credible information;

[0089] Step S103: Count the number of information content without videos and credible information in a certain relevant event within the monitoring period, and calculate the credibility score of the event according to the following formula:

[0090] ;

[0091] Among them, A represents the credibility score of the event, B represents the number of information content without videos, D b represents the similarity matching score of the b-th credible information, a B represents the weight of the information content without videos, a D represents the weight of the credible information, and C represents the number of credible information;

[0092] Step S104: Set a credibility score threshold, and when the credibility score of an event is greater than the credibility score threshold, mark the event as a credible event;

[0093] For example, the system legally monitors social platforms; obtains multiple pieces of information: "Two cars collided at the mall entrance", "The driver was injured", "There were many people at the scene", etc.; the system integrates keywords and identifies it as "car accident at the mall entrance";

[0094] Among the relevant information, a video is captured. The time and location recorded in the video EXIF metadata are at the mall entrance; the system performs image and semantic matching on the description of the "car accident" and the video content, and the similarity score is 85; if the set credible similarity threshold of the system is 80 and the score exceeds it, this piece of information is considered "credible information";

[0095] Suppose B = 5, a B = 0.3, C = 2, D1 = 85, D2 = 90, a D = 0.7, and it is calculated that A is 62.75. If the set credible event threshold of the system is 60; since A > 60, this event is automatically marked as a "credible event" by the system.

[0096] Step S200: Collect the characteristic data of the credible event, determine the characteristic event through the characteristic event automated evaluation model, predict the severity of the characteristic event, obtain the information parameters of the characteristic event, input them into the information prediction system, judge the information index caused by the characteristic event, establish an information index change graph, and generate an evaluation record at the same time;

[0097] Among them, step S200 includes:

[0098] Step S201: Collect the characteristic data of the credible event and input it into the characteristic event automated evaluation model to determine the credible event as the characteristic event and predict the severity;

[0099] Step S202: Obtain the information parameters of the characteristic event, input them into the information prediction system together with the predicted severity, judge the information index caused by the characteristic event, establish an information index change graph, and generate an evaluation record at the same time;

[0100] For example, in the invention application with the Chinese patent publication number CN115187148A, a method, system, device and readable storage medium for analyzing the situation of emergencies are disclosed. The characteristic data of the credible event is input into the invention application to determine the credible event as the characteristic event and predict the severity;

[0101] The characteristic data of the credible event includes characteristics and locations. The characteristics refer to the on-site situation, which is mainly used to judge the urgency and severity of the event. The characteristics are as described in Table 1:

[0102] Table 1

[0103] Feature Example or description Number of people on site Small / medium / large Whether there is physical conflict Whether there is pushing behavior Whether there is injury Obvious signs of injury, falling to the ground, bleeding, ambulance, etc. Emotional state Collective emotional excitement, emotional stability, screaming or chaotic scenes Stability and duration of the picture Whether it is a clear video, whether it is continuously recorded, viewing angle range, etc.

[0104] Location refers to the spatial environment and social sensitivity where an event occurs, which helps to evaluate the potential impact scope. The locations are described in Table 2 as follows:

[0105] Table 2

[0106] Location category Example Special places such as hospitals Sensitive institutions such as hospitals Transportation hubs Railway stations, airports, subway stations, main roads Commercially prosperous areas Crowded areas such as shopping malls, commercial streets, large supermarkets Residential communities Inside the community, apartments, corridors, etc. Remote areas Wild areas, etc.

[0107] In the invention application with Chinese patent publication number CN101763401A, a method for predicting and analyzing hotspots of network information is disclosed. The information parameters of a characteristic event are input into the invention application to judge the information index caused by the characteristic event.

[0108] Step S300: Set an information index threshold, draw the information index threshold as a horizontal threshold line, and insert it into the information index change graph. Determine the early warning evaluation record. Set the time point corresponding to the first intersection point of the information index change graph and the horizontal threshold line as the first time point, summarize to obtain the first record set. Set the time point corresponding to the last intersection point of the information index change graph and the horizontal threshold line as the second time point, summarize to obtain the second record set, and calculate to obtain the third record set, where the third record set is the intersection of the first record set and the second record set;

[0109] Among them, step S300 includes:

[0110] Step S301: Through the information prediction system, obtain historical evaluation records, set an information index threshold, collect the information index change graph of a certain historical evaluation record, draw the information index threshold as a horizontal threshold line, and insert it into the information index change graph. When there is an intersection point between the information index change graph and the horizontal threshold line, set the historical evaluation record as the early warning evaluation record;

[0111] Step S302: Obtain the early warning evaluation record, set the time point corresponding to the first intersection point of the information index change graph and the horizontal threshold line as the first time point, collect the first time point of the early warning evaluation record, preset the characteristic first time point. If the first time point of a certain early warning evaluation record is less than the characteristic first time point for comparison, set the early warning evaluation record as the first record, summarize the first records, and obtain the set of the first records as P1;

[0112] Step S303: Set the time point corresponding to the last intersection point of the information index change graph and the horizontal threshold line as the second time point, collect the second time point of the early warning evaluation record, preset the characteristic second time point. If the second time point of a certain early warning evaluation record is greater than the characteristic second time point for comparison, set the early warning evaluation record as the second record, summarize the second records, and obtain the set of the second records as P2. Calculate the set of the third characteristic records as P3 = P1 ∩ P2;

[0113] For example, obtain the time points corresponding to the intersections in the early warning evaluation records. For each early warning record, take the first intersection time point as the first time point T1. The system sets the characteristic first time point as Tf = 12:00. For each early warning evaluation record, if its first time point satisfies T1 < Tf, then include this record in the first record set P1;

[0114] Obtain the last intersection as the second time point T2. Set the characteristic second time point Ts = 20:00. If T2 > Ts for a certain early warning record, it indicates that the event continues to ferment until evening and has the risk of delayed spread, and record it in the second record set P2.

[0115] Step S400: Calculate the sets of the first and second characteristic records according to the first record set, the second record set, and the third record set. Collect the key information of the early warning evaluation records. Set the key information that exceeds the occurrence frequency threshold as the characteristic information, and determine the characteristic information corresponding to each characteristic record;

[0116] Among them, step S400 includes:

[0117] Step S401: Subtract the set of the third characteristic record from the sets of the first and second records respectively to obtain the sets of the first and second characteristic records;

[0118] Step S402: In a set of a certain characteristic record, collect an early warning evaluation record, extract the key information of the event in the early warning evaluation record, summarize the key information of the characteristic record, count the number of times a certain key information appears in the characteristic record, and calculate the occurrence frequency of the key information as Z = Z1 / Z2, where Z1 represents the number of occurrences of the key information, and Z2 represents the total number of occurrences of the key information;

[0119] Step S403: Preset the occurrence frequency threshold, set the key information that exceeds the occurrence frequency threshold as the characteristic information, and summarize the characteristic information corresponding to a certain characteristic record;

[0120] For example, taking the first characteristic record set P1 as an example, extract the key information fields in the early warning evaluation record. Each record contains the following key information: information keywords (such as "power outage", "explosion", "loss of contact", etc.), geographical labels (such as "xx City", "xx Town"), involved parties (such as "enterprise name"), information types (such as "complaint", "false information", "accident");

[0121] Suppose the keyword "power outage" appears 12 times in P1 in total, and the total number of occurrences of all key information is 100, then the occurrence frequency of "power outage" is 12%; the system sets the threshold as 10%, then "power outage" is regarded as the characteristic information.

[0122] Step S500: Obtain the evaluation records of the information prediction system in a certain historical sampling period, draw the actual information index change graph of the evaluation records, determine the abnormal records, and calculate the weight of the feature records according to the abnormal rate of the feature records and the number of evaluation records of the feature records;

[0123] Among them, step S500 includes:

[0124] Step S501: Obtain the evaluation records of the information prediction system in a certain historical sampling period, collect the actual information index of a certain evaluation record, draw the actual information index change graph of the evaluation record, obtain the information index change graph of the evaluation record, combine the actual information index change graph and the information index change graph, calculate the error rate of the information index, preset the error rate threshold, and set the evaluation records exceeding the error rate threshold as abnormal records;

[0125] Step S502: Collect the key information of the evaluation records, match it with the feature information corresponding to each feature record, and determine the feature record corresponding to the evaluation record;

[0126] Step S503: Count the number of evaluation records H1 and the number of abnormal records H2 of a certain feature record in a certain historical sampling period, calculate the abnormal rate of the feature record as H2 / H1, summarize the abnormal rates of the feature record in all historical sampling periods, and calculate the average abnormal rate of the feature record;

[0127] Step S504: Calculate the weight of the feature record according to the following formula:

[0128] ;

[0129] Among them, W represents the weight of the feature record, H represents the number of evaluation records of the feature record in all historical sampling periods, H' represents the total number of evaluation records in all historical sampling periods, K represents the average abnormal rate of the feature record, p1 represents the weight of the number of evaluation records, and p2 represents the weight of the average abnormal rate;

[0130] For example, the total number of evaluation records of all feature records is 1000, the number of a certain feature record is 50, and the average abnormal rate is 0.24. The calculated weight of the feature record is 0.334.

[0131] Step S600: Collect the key information of the real-time feature event, determine the feature record corresponding to the real-time feature event, preset the sampling time period, the sampling time period is less than the sampling period, count the number of each feature record in the sampling time period, calculate the real-time sampling period according to the number of feature records and the weight of the feature record, and judge the sampling state according to the real-time sampling period;

[0132] Among them, step S600 includes:

[0133] Step S601: Obtain a real-time event, calculate the real-time credibility score of the real-time event. When the real-time credibility score is greater than the credibility score threshold, mark the real-time event as a real-time credible event;

[0134] Step S602: Collect the feature data of the real-time credible event, and input it into the feature event automated evaluation model to determine that the real-time credible event is a real-time feature event and predict the severity. Collect the key information of the real-time feature event to determine the feature record corresponding to the real-time feature event;

[0135] Step S603: Preset a sampling time period, where the sampling time period is less than the sampling period. Count the number of each feature record in the sampling time period, and calculate the real-time sampling period according to the following formula:

[0136] ;

[0137] where, T2 u represents the real-time sampling period at the u-th sampling time, T represents the sampling period, T1 represents the real-time sampling period obtained in the previous calculation, S n represents the number of the n-th feature record, W n represents the weight of the n-th feature record, and m represents the total number of feature records and m = 3;

[0138] Step S604: When the real-time sampling period is greater than the sampling time period, execute step S603. When the real-time sampling period is less than or equal to the sampling time period, remind the staff to re-sample and analyze the event at the real-time sampling period;

[0139] For example, set the sampling period to 60 minutes, set the sampling time period to 10 minutes, the previous real-time sampling period to 45 minutes, the number of the first feature record to 5, the weight to 0.3, the number of the second feature record to 3, the weight to 0.25, the number of the third feature record to 7, and the weight to 0.45. Calculate the real-time sampling period to be 4.05 minutes, then remind the staff to re-sample and analyze after 3.8 minutes.

[0140] In order to better implement the above method, an event automated evaluation system based on a large model is also proposed. The system includes a credible event module, an evaluation record module, a record classification module, a feature information module, a feature record weight module, and a real-time calculation module;

[0141] Trusted Event Module: By legally monitoring each social platform, obtaining the information content published during the monitoring period, generating an event from the published information content through natural language processing, aggregating relevant events, obtaining the information content with videos in the relevant event set, collecting the EXIF metadata of the information content and performing similarity matching with the key information to determine whether the information content is trusted information, calculating the trust score of the event based on the number of information content without videos in the relevant event, the number of trusted information, and the similarity matching score of the trusted information, and determining the trusted event;

[0142] Evaluation Record Module: Collecting the characteristic data of the trusted event, determining the characteristic event and predicting the severity of the characteristic event through the characteristic event automatic evaluation model, obtaining the information parameters of the characteristic event, inputting them into the information prediction system, judging the information index caused by the characteristic event, establishing an information index change graph, and generating an evaluation record at the same time;

[0143] Record Classification Module: Setting an information index threshold, drawing the information index threshold as a horizontal threshold line, and inserting it into the information index change graph to determine the warning evaluation record. Setting the time point corresponding to the first intersection point of the information index change graph and the horizontal threshold line as the first time point, aggregating to obtain the first record set, setting the time point corresponding to the last intersection point of the information index change graph and the horizontal threshold line as the second time point, aggregating to obtain the second record set, and calculating to obtain the third record set, where the third record set is the intersection of the first record set and the second record set;

[0144] Characteristic Information Module: Calculating the sets of the first and second characteristic records based on the first record set, the second record set, and the third record set, collecting the key information of the warning evaluation record, setting the key information exceeding the occurrence frequency threshold as the characteristic information, and determining the characteristic information corresponding to each characteristic record;

[0145] Characteristic Record Weight Module: Obtaining the evaluation records of the information prediction system in a certain historical sampling period, drawing the actual information index change graph of the evaluation records, determining the abnormal records, and calculating the weights of the characteristic records according to the abnormality rate of the characteristic records and the number of evaluation records of the characteristic records;

[0146] Among them, the characteristic record weight module includes an abnormal record determination unit and a characteristic record weight calculation unit:

[0147] Determine abnormal record unit: Obtain the evaluation records of the information prediction system in a certain historical sampling period, collect the actual information index of a certain evaluation record, draw the change graph of the actual information index of the evaluation record, obtain the change graph of the information index of the evaluation record, combine the actual information index change graph and the information index change graph, calculate the error rate of the information index, preset the error rate threshold, and set the evaluation records exceeding the error rate threshold as abnormal records;

[0148] Calculate feature record weight unit: Collect the key information of the evaluation record, match it with the feature information corresponding to each feature record, determine the feature record corresponding to the evaluation record, count the number of evaluation records and the number of abnormal records of a certain feature record in a certain historical sampling period, calculate the abnormal rate of the feature record, summarize the abnormal rates of the feature record in all historical sampling periods, and calculate the average abnormal rate of the feature record, and calculate the weight of the feature record.

[0149] Real-time calculation module: Collect the key information of real-time feature events, determine the feature records corresponding to the real-time feature events, preset the sampling time period, where the sampling time period is less than the sampling period, count the number of each feature record in the sampling time period, calculate the real-time sampling period according to the number of feature records and the weight of the feature records, and judge the sampling status according to the real-time sampling period;

[0150] Among them, the real-time calculation module includes a real-time feature record determination unit and a real-time sampling period calculation unit:

[0151] Real-time feature record determination unit: Obtain real-time events, calculate the real-time credibility score of the real-time events. When the real-time credibility score is greater than the credibility score threshold, mark the real-time events as real-time credible events, collect the feature data of the real-time credible events, and input them into the feature event automatic evaluation model to determine the real-time credible events as real-time feature events and predict the severity, collect the key information of the real-time feature events, and determine the feature records corresponding to the real-time feature events

[0152] Real-time sampling period calculation unit: Preset the sampling time period, where the sampling time period is less than the sampling period, count the number of each feature record in the sampling time period, calculate the real-time sampling period. When the real-time sampling period is less than or equal to the sampling time period, remind the staff to resample and analyze the events at the real-time sampling period.

[0153] It is obvious to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, in all respects, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Thus, all changes that fall within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.

Claims

1. An event automated evaluation method based on a large model, characterized in that, The method includes: Step S100: By legally monitoring each social platform, obtain the information content published during the monitoring period. Through natural language processing, generate an event for the published information content, summarize relevant events, obtain the information content with videos in the relevant event set, collect the EXIF metadata of the information content and perform similarity matching with the key information to determine whether the information content is credible information. Calculate the credibility score of the event based on the number of information content without videos in the relevant events, the number of credible information, and the similarity matching score of the credible information, and determine the credible events; Step S200: Collect the characteristic data of the credible events, determine the characteristic events through the characteristic event automatic evaluation model, predict the severity of the characteristic events, and obtain the information parameters of the characteristic events. Input them into the information prediction system to judge the information index caused by the characteristic events, establish an information index change graph, and generate an evaluation record at the same time; Step S300: Set an information index threshold, draw the information index threshold as a horizontal threshold line, and insert it into the information index change graph to determine the early warning evaluation record. Set the time point corresponding to the first intersection point of the information index change graph and the horizontal threshold line as the first time point, summarize to obtain the first record set, set the time point corresponding to the last intersection point of the information index change graph and the horizontal threshold line as the second time point, summarize to obtain the second record set, and calculate to obtain the third record set. The third record set is the intersection of the first record set and the second record set; Step S400: Calculate the sets of the first and second characteristic records according to the first record set, the second record set, and the third record set. Collect the key information of the early warning evaluation record, set the key information exceeding the occurrence frequency threshold as the characteristic information, and determine the characteristic information corresponding to each characteristic record; Step S400 includes the following steps: Step S401: Subtract the set of the third characteristic record from the sets of the first and second records respectively to obtain the sets of the first and second characteristic records; Step S402: In a set of a certain characteristic record, collect a certain early warning evaluation record, extract the key information of the event in the early warning evaluation record, summarize the key information of the characteristic record, count the number of times a certain key information appears in the characteristic record, and calculate the occurrence frequency of the key information as Z = Z1 / Z2, where Z1 represents the number of occurrences of the key information and Z2 represents the total number of occurrences of the key information; Step S403: Preset an occurrence frequency threshold, set the key information exceeding the occurrence frequency threshold as the characteristic information, and summarize the characteristic information corresponding to a certain characteristic record; Step S500: Obtain the evaluation record of the information prediction system in a certain historical sampling period, draw the actual information index change graph of the evaluation record, determine the abnormal records, and calculate the weight of the characteristic record according to the abnormality rate of the characteristic record and the number of evaluation records of the characteristic record; Step S600: Collect the key information of real-time feature events, determine the feature records corresponding to the real-time feature events, preset a sampling time period, where the sampling time period is less than the sampling period, count the number of each feature record in the sampling time period, calculate the real-time sampling period according to the number of feature records and the weights of the feature records, and determine the sampling status according to the real-time sampling period; The step S600 includes the following steps: Step S601: Obtain real-time events, calculate the real-time credibility score of the real-time events. When the real-time credibility score is greater than the credibility score threshold, mark the real-time events as real-time credible events; Step S602: Collect the feature data of real-time credible events and input them into the feature event automatic evaluation model to determine that the real-time credible events are real-time feature events and predict the severity, collect the key information of the real-time feature events, and determine the feature records corresponding to the real-time feature events; Step S603: Preset a sampling time period, where the sampling time period is less than the sampling period, count the number of each feature record in the sampling time period, and calculate the real-time sampling period according to the following formula: ； Among them, T2 u represents the real-time sampling period at the u-th sampling time, T represents the sampling period, T1 represents the real-time sampling period obtained from the previous calculation, S n represents the number of the n-th feature records, W n represents the weight of the n-th feature record, m represents the total number of feature records and m = 3; Step S604: When the real-time sampling period is greater than the sampling time period, execute step S603. When the real-time sampling period is less than or equal to the sampling time period, remind the staff to resample and analyze the events at the real-time sampling period.

2. The event automation evaluation method based on a large model according to claim 1, wherein The step S100 includes the following steps: Step S101: Legally monitor each social platform, obtain the information content published within a certain monitoring period, extract the key information of a certain piece of information content through natural language processing, and generate an event. Summarize the information content of related events according to the content of the event; Step S102: In the information content set of a certain related event, obtain the information content with videos, collect the EXIF metadata of the information content, perform similarity matching between the key information of the information content and the EXIF metadata, calculate the similarity matching score, set the similarity matching score threshold. If the similarity matching score of the information content exceeds the similarity matching score threshold, set the information content as credible information; Step S103: Count the number of information content without videos and credible information in a certain related event within the monitoring period, and calculate the credibility score of the event according to the following formula: ； Among them, A represents the credibility score of the event, B represents the number of information contents without videos, D b represents the similarity matching score of the b-th piece of credible information, a B represents the weight of the information content without videos, a D represents the weight of the credible information, and C represents the number of credible information; Step S104: Set the credibility score threshold. When the credibility score of a certain event is greater than the credibility score threshold, mark the event as a credible event.

3. The event automation evaluation method based on a large model according to claim 2, wherein The step S200 includes the following steps: Step S201: Collect the feature data of credible events and input them into the feature event automatic evaluation model to determine that the credible events are feature events and predict the severity; Step S202: Obtain the information parameters of the feature events, input them into the information prediction system together with the predicted severity, judge the information index caused by the feature events, establish an information index change graph, and generate an evaluation record at the same time.

4. The method for automatically evaluating events based on a large model according to claim 3, wherein The step S300 includes the following steps: Step S301: Obtain historical evaluation records through the information prediction system, set an information index threshold, collect the information index change graph of a certain historical evaluation record, draw the information index threshold as a horizontal threshold line, and insert it into the information index change graph. When there is an intersection point between the information index change graph and the horizontal threshold line, set the historical evaluation record as a warning evaluation record; Step S302: Obtain the warning evaluation record, set the time point corresponding to the first intersection point of the information index change graph and the horizontal threshold line as the first time point, collect the first time point of the warning evaluation record, preset the characteristic first time point. If the first time point of a certain warning evaluation record is less than the characteristic first time point for comparison, set the warning evaluation record as the first record, summarize the first records, and obtain the set of the first records as P1; Step S303: Set the time point corresponding to the last intersection point of the information index change graph and the horizontal threshold line as the second time point, collect the second time point of the warning evaluation record, preset the characteristic second time point. If the second time point of a certain warning evaluation record is greater than the characteristic second time point for comparison, set the warning evaluation record as the second record, summarize the second records, and obtain the set of the second records as P2. Calculate the set of the third characteristic records as P3 = P1 ∩ P2.

5. The event automation evaluation method based on a large model according to claim 1, characterized in that The said step S500 includes the following steps: Step S501: Obtain the evaluation records of the information prediction system in a certain historical sampling period, collect the actual information index of a certain evaluation record, draw the actual information index change graph of the evaluation record, obtain the information index change graph of the evaluation record, combine the actual information index change graph and the information index change graph, calculate the error rate of the information index, preset the error rate threshold, and set the evaluation records exceeding the error rate threshold as abnormal records; Step S502: Collect the key information of the evaluation record, and match it with the characteristic information corresponding to each characteristic record to determine the characteristic record corresponding to the evaluation record; Step S503: Count the number of evaluation records H1 and the number of abnormal records H2 of a certain characteristic record in a certain historical sampling period, calculate the abnormal rate of the characteristic record as H2 / H1, summarize the abnormal rates of the characteristic record in all historical sampling periods, and calculate the average abnormal rate of the characteristic record; Step S504: Calculate the weight of the characteristic record according to the following formula: ； Where, W represents the weight of the characteristic record, H represents the number of evaluation records of the characteristic record in all historical sampling periods, H’ represents the total number of evaluation records in all historical sampling periods, K represents the average abnormal rate of the characteristic record, p1 represents the weight of the number of evaluation records, and p2 represents the weight of the average abnormal rate.

6. An event automated evaluation system based on a large model, which is used to implement the event automated evaluation method based on a large model described in any one of claims 1-5, and is characterized in that, The said system includes a trusted event module, an evaluation record module, a record classification module, a characteristic information module, a characteristic record weight module, and a real-time calculation module; The trusted event module: By legally monitoring each social platform, obtaining the information content published within the monitoring period, generating an event for the published information content through natural language processing, summarizing relevant events, obtaining the information content with videos in the relevant event set, collecting the EXIF metadata of the information content and performing similarity matching with the key information to determine whether the information content is trusted information, calculating the trust score of the event based on the number of information content without videos, the number of trusted information, and the similarity matching score of the trusted information in the relevant events, and determining the trusted event; The evaluation record module: Collecting the feature data of the trusted event, determining the feature event and predicting the severity of the feature event through the feature event automated evaluation model, obtaining the information parameters of the feature event, inputting them into the information prediction system, judging the information index caused by the feature event, establishing an information index change graph, and generating an evaluation record at the same time; The record classification module: Setting an information index threshold, drawing the information index threshold as a horizontal threshold line, and inserting it into the information index change graph to determine the warning evaluation record. Setting the time point corresponding to the first intersection point of the information index change graph and the horizontal threshold line as the first time point, summarizing to obtain the first record set, setting the time point corresponding to the last intersection point of the information index change graph and the horizontal threshold line as the second time point, summarizing to obtain the second record set, and calculating to obtain the third record set, where the third record set is the intersection of the first record set and the second record set; The feature information module: Calculating the sets of the first and second feature records according to the first record set, the second record set, and the third record set, collecting the key information of the warning evaluation record, setting the key information exceeding the occurrence frequency threshold as the feature information, and determining the feature information corresponding to each feature record; The feature record weight module: Obtaining the evaluation records of the information prediction system in a certain historical sampling period, drawing the actual information index change graph of the evaluation records, determining the abnormal records, and calculating the weight of the feature records according to the abnormality rate of the feature records and the number of evaluation records of the feature records; The real-time calculation module: Collecting the key information of the real-time feature event, determining the feature record corresponding to the real-time feature event, presetting a sampling time period, where the sampling time period is less than the sampling period, counting the number of each feature record in the sampling time period, calculating the real-time sampling period according to the number of feature records and the weight of the feature records, and judging the sampling state according to the real-time sampling period; 7. The event automation evaluation system based on a large model according to claim 6, wherein, The feature record weight module includes an abnormal record determination unit and a feature record weight calculation unit: The abnormal record determination unit: obtains the evaluation records of the information prediction system in a certain historical sampling period, collects the actual information index of a certain evaluation record, draws the change graph of the actual information index of the evaluation record, obtains the change graph of the information index of the evaluation record, combines the actual information index change graph and the information index change graph, calculates the error rate of the information index, presets an error rate threshold, and sets the evaluation records exceeding the error rate threshold as abnormal records; The feature record weight calculation unit: collects the key information of the evaluation record, matches it with the feature information corresponding to each feature record, determines the feature record corresponding to the evaluation record, counts the number of evaluation records and the number of abnormal records of a certain feature record in a certain historical sampling period, calculates the abnormal rate of the feature record, summarizes the abnormal rates of the feature record in all historical sampling periods, and calculates the average abnormal rate of the feature record, and calculates the weight of the feature record.

8. The event automation evaluation system based on a large model according to claim 6, wherein The real-time calculation module includes a real-time feature record determination unit and a real-time sampling period calculation unit: The real-time feature record determination unit: obtains a real-time event, calculates the real-time credibility score of the real-time event, when the real-time credibility score is greater than the credibility score threshold, marks the real-time event as a real-time credible event, collects the feature data of the real-time credible event, and inputs it into the feature event automated evaluation model to determine the real-time credible event as a real-time feature event and predict the severity, collects the key information of the real-time feature event, and determines the feature record corresponding to the real-time feature event The real-time sampling period calculation unit: presets a sampling time period, the sampling time period is less than the sampling period, counts the number of each feature record in the sampling time period, calculates the real-time sampling period, and when the real-time sampling period is less than or equal to the sampling time period, reminds the staff to resample and analyze the event at the real-time sampling period.

Images