Social event identification and interpretation system based on large model

Abstract

The invention discloses a social event identification and interpretation system based on a large model, and relates to the technical field of data analysis, and the system comprises a plurality of modules: a data collection module collects social news data in real time; the preprocessing module preprocesses the data; the event recognition agent automatically recognizes, classifies and sorts the importance of the social events by using the pre-trained large model; the event interpretation agent generates a detailed event interpretation report; the knowledge base construction module is used for collecting and extracting periodic event features and constructing a knowledge base; the monitoring and processing module judges the event type through similarity retrieval and updates the knowledge base; the risk assessment module assesses investment risks and provides early warning by using statistics and machine learning methods in combination with the data and the interpretation result; and the user interaction interface provides customized query and result display for analysts to obtain information. According to the method, the social data can be automatically captured, classified, interpreted and analyzed, and the workload of a data analyst is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and specifically relates to a social event recognition and interpretation system based on a large model. Background Art

[0002] In this ever-changing world, countless events are intertwined and occurring every moment. In relevant departments, various events reflect certain social conditions and influence scopes. Behind these events and data, the problems they reflect determine the adjustment of the development direction of relevant departments for the future.

[0003] For relevant departments, every minor fluctuation in society may imply an immeasurable influence scope. These social dynamics, like invisible forces, quietly affect the decisions of relevant personnel. Data analysts, as the unsung heroes behind relevant departments, their job is to deeply explore these complex events, carefully analyze the impacts they bring, evaluate the risks and opportunities, and thus provide accurate and powerful suggestions for relevant personnel.

[0004] However, in this era of information explosion, data analysis work faces unprecedented challenges. Traditionally, data analysts need to spend a large amount of time and energy to perceive, organize, interpret, and analyze this information. But human resources are ultimately limited. Facing a vast amount of data, how to more effectively utilize limited resources has become a difficult problem for data analysts. Summary of the Invention

[0005] In view of the requirements and deficiencies in the current technological development, the present invention provides a social event recognition and interpretation system based on a large model.

[0006] The technical solution adopted by the social event recognition and interpretation system based on a large model of the present invention to solve the above technical problems is as follows:

[0007] A social event recognition and interpretation system based on a large model, which includes:

[0008] A data collection module, responsible for collecting social news data in real time from a target platform,

[0009] A preprocessing module, responsible for performing preprocessing operations on the collected data;

[0010] An event recognition agent, responsible for automatically recognizing, classifying, and ranking the importance of social events contained in the preprocessed data by using a pre-trained large model;

[0011] An event interpretation agent, responsible for interpreting and analyzing the ranked social events and generating corresponding event interpretation reports;

[0012] The knowledge base construction module is responsible for collecting social events classified by the event recognition agent, extracting features related to periodic events from them, and constructing a knowledge base for periodic events;

[0013] The monitoring and processing module is responsible for collecting social events classified by the event recognition agent, and judging whether it is a known periodic event in the knowledge base or a new periodic pattern through similarity retrieval technology. If it is a known periodic event, it will send the actual risks of the retrieved corresponding historical similar events to the risk assessment module. If it is a new periodic pattern, it will extract features related to periodic events from the social event and update the knowledge base;

[0014] The risk assessment module is responsible for combining social news data and event interpretation results, referring to the actual risks of historical similar events, and using statistical and machine learning methods to evaluate the potential risks of social events to investors and provide early warnings;

[0015] The user interaction interface is responsible for providing customized query condition input and result display functions for data analysts to obtain the classification results, interpretation results and risk assessment results of social events.

[0016] Optionally, the data collection module involved includes:

[0017] The data source docking unit is used to dock the target news platform and government affairs data platform;

[0018] The data collection unit is used to simultaneously collect social news data from the docked target platform by using web crawler technology in combination with multi-threading and asynchronous IO mechanisms;

[0019] The data parsing unit is used to convert unstructured data in the collected data into structured data.

[0020] Further optionally, the preprocessing module involved specifically includes:

[0021] The data cleaning unit is used to clean the collected data, including removing duplicate records, correcting errors and filling missing values;

[0022] The data formatting unit is used to perform standardization and normalization operations on the cleaned data so that data from different sources and with different dimensions can be analyzed in the same framework.

[0023] Optionally, the event recognition agent involved specifically includes:

[0024] A historical event retrospective unit, which is used to train a Transformer model with event recognition and classification using historical data that is manually annotated and covers different types of events and their impact situations. It is also used to deeply understand and classify historical social events using the Transformer model, and combine time series analysis and clustering algorithms to identify periodic events;

[0025] A real-time event monitoring unit, which is used to instantly analyze newly incoming social information to identify newly occurring social events by means of a real-time stream processing framework and the online inference ability of the Transformer model, and introduce an attention mechanism to enhance the ability to capture key information;

[0026] An event importance ranking unit, which is used to design a multi-dimensional scoring system covering the scope of the event, the impact degree of historical similar events, and social media sentiment analysis, and use a machine learning model to score and rank events.

[0027] Optionally, the involved event interpretation agent specifically includes:

[0028] A causal relationship analysis unit, which is used to apply a graph neural network to construct a social entity relationship graph and analyze the causal connections between events, social policies, and social emotions;

[0029] A prediction unit, which is used to use a time series prediction model combined with event characteristics to predict the impact scope and duration of events on different social fields, and introduce an attention mechanism to focus on key variables to improve prediction accuracy;

[0030] An argument generation unit, which is used to extract similar event cases from historical data, compare and analyze the similarities and differences between the current event and historical events, automatically generate an argument report that can reflect the impact of the event, and then use natural language generation technology to automatically generate an event interpretation report covering influencing factors, expected impact scope, and time axis based on the data analysis results.

[0031] Optionally, the involved risk assessment module specifically includes:

[0032] A data association unit, which is responsible for receiving the collected data from the collection and preprocessing module and the event interpretation report generated by the event interpretation agent, and finding the internal connections between these data according to the set rules and logic, so as to associate different data information of the same event or related events and integrate them into a data set about social events;

[0033] A feature engineering unit, which is responsible for performing feature engineering processing on the integrated data set and screening out the features most relevant to risk assessment;

[0034] A model construction unit, which is responsible for selecting at least one algorithm from decision trees, random forests, and gradient boosting machines to construct a risk assessment model;

[0035] A model training unit, which is responsible for training a risk assessment model using the selected features. During the training process, the risk assessment model continuously adjusts its own parameters and structure to minimize the difference between the predicted risk level and the actual risk level, thereby gradually learning the mapping rule between different event feature combinations and the corresponding risk levels;

[0036] A risk assessment model, which is responsible for quantitatively evaluating the risk levels of different events directly or by referring to the actual risks of historical similar events;

[0037] A risk warning unit, which is responsible for issuing a risk warning for events that reach the set risk level according to the risk assessment results.

[0038] Optionally, the involved user interface has the following functions:

[0039] Adopts a responsive design to ensure a good user experience on different devices;

[0040] Provides a variety of personalized setting options, allowing data analysts to customize the interface according to their own needs, including event type filtering, time range selection, and risk level display;

[0041] Displays data in a branched manner through charts and maps;

[0042] Supports interactive queries, so that data analysts can input query conditions and obtain results through click and drag operations;

[0043] Supports real-time data update to ensure that data analysts always have the latest information;

[0044] Has a user feedback mechanism to encourage data analysts to put forward improvement suggestions.

[0045] Optionally, the involved social event recognition and interpretation system further includes:

[0046] A training and optimization module, which is used to continuously collect new data collected by the collection and preprocessing module, and combines automatic recognition of common problems with manual fine-tuning of rare problems to train and optimize the event recognition agent and the event interpretation agent. At the same time, online learning and transfer learning technologies are adopted to enable the model to adapt to social changes and improve the accuracy of recognition and interpretation

[0047] A performance evaluation module, which is used to regularly evaluate the performance of the event recognition agent and the event interpretation agent based on preset rules, and adjust the agent parameters according to the evaluation results.

[0048] Optionally, the involved system backend adopts big data processing technology to process and analyze massive social event data;

[0049] The system uses the Apache Hadoop distributed file system for data storage and also uses a MongoDB knowledge base to provide flexible storage and query capabilities for unstructured data;

[0050] The system integrates Apache Spark to support complex data processing tasks. It uses Spark Streaming for real-time data stream processing, processes real-time social data, captures social dynamics, and provides instant data support for event recognition and risk assessment.

[0051] A social event recognition and interpretation system based on a large model according to the present invention has the beneficial effects compared with the prior art as follows:

[0052] The present invention can achieve automatic scraping, classification, and preliminary analysis of social data, thus greatly reducing the workload of data analysts. At the same time, it can more accurately predict development trends, provide more accurate and timely suggestions for relevant personnel, significantly improve the work efficiency of data analysts, reduce human errors, and provide more professional and in-depth social event analysis services. Description of the Drawings

[0053] Attached Figure 1 is a block diagram of module connections in Embodiment 1 of the present invention. Detailed Embodiments

[0054] To make the technical solutions, problems to be solved, and technical effects of the present invention clearer and more understandable, the following describes the technical solutions of the present invention clearly and completely in conjunction with specific embodiments.

[0055] Embodiment 1:

[0056] Combined with the attached Figure 1 , this embodiment proposes a social event recognition and interpretation system based on a large model, which includes:

[0057] A data collection module, responsible for collecting social news data in real time from the target platform,

[0058] A preprocessing module, responsible for performing preprocessing operations on the collected data;

[0059] An event recognition agent, responsible for automatically recognizing, classifying, and ranking the importance of social events contained in the preprocessed data using a pre-trained large model;

[0060] An event interpretation agent, responsible for interpreting and analyzing the ranked social events and generating corresponding event interpretation reports;

[0061] The knowledge base construction module is responsible for collecting social events classified by the event recognition agent, extracting features related to periodic events from them, and constructing a knowledge base of periodic events;

[0062] The monitoring and processing module is responsible for collecting social events classified by the event recognition agent, and judging whether it is a known periodic event in the knowledge base or a new periodic pattern through similarity retrieval technology. If it is a known periodic event, it will send the actual risks of the retrieved corresponding historical similar events to the risk assessment module. If it is a new periodic pattern, it will extract features related to periodic events from the social event and update the knowledge base;

[0063] The risk assessment module is responsible for combining social news data and event interpretation results, referring to the actual risks of historical similar events, and using statistical and machine learning methods to evaluate the potential risks of social events to investors and provide early warnings;

[0064] The user interaction interface is responsible for providing customized query condition input and result display functions for data analysts to obtain the classification results, interpretation results and risk assessment results of social events.

[0065] In this embodiment, the data acquisition module involved includes:

[0066] The data source docking unit is used to dock with the target news platform and government affairs data platform;

[0067] The data acquisition unit is used to simultaneously collect social news data from the docked target platform by using web crawler technology in combination with multi-threading and asynchronous IO mechanisms;

[0068] The data parsing unit is used to convert unstructured data in the collected data into structured data.

[0069] In this embodiment, the preprocessing module specifically includes:

[0070] The data cleaning unit is used to clean the collected data, including removing duplicate records, correcting errors and filling in missing values;

[0071] The data formatting unit is used to perform standardization and normalization operations on the cleaned data so that data from different sources and with different dimensions can be analyzed in the same framework.

[0072] In this embodiment, the event recognition agent specifically includes:

[0073] A historical event retrospective unit, which is used to train a Transformer model with event recognition and classification using historical data that is manually annotated and covers different types of events and their impact situations. It is also used to deeply understand and classify historical social events using the Transformer model, and combine time series analysis and clustering algorithms to identify periodic events;

[0074] A real-time event monitoring unit, which is used to instantly analyze newly incoming social information to identify newly occurred social events by means of a real-time stream processing framework and the online inference ability of the Transformer model, and introduce an attention mechanism to enhance the ability to capture key information;

[0075] An event importance ranking unit, which is used to design a multi-dimensional scoring system covering the scope of events, the impact degree of historical similar events, and social media sentiment analysis, and use a machine learning model to score and rank events.

[0076] In this embodiment, the involved event interpretation agent specifically includes:

[0077] A causal relationship analysis unit, which is used to apply a graph neural network to construct a social entity relationship graph and analyze the causal connections between events, social policies, and social emotions;

[0078] A prediction unit, which is used to use a time series prediction model combined with event characteristics to predict the impact scope and duration of events on different social fields, and introduce an attention mechanism to focus on key variables to improve prediction accuracy;

[0079] An argument generation unit, which is used to extract similar event cases from historical data, compare and analyze the similarities and differences between the current event and historical events, automatically generate an argument report that can reflect the impact of the event, and then use natural language generation technology to automatically generate an event interpretation report covering influencing factors, expected impact scope, and time axis according to the data analysis results.

[0080] In this embodiment, the involved risk assessment module specifically includes:

[0081] A data association unit, which is responsible for receiving the collected data from the collection and preprocessing module and the event interpretation report generated by the event interpretation agent, and finding the internal connections between these data according to the set rules and logics, so as to associate different data information of the same event or related events and integrate them into a dataset about social events;

[0082] A feature engineering unit, which is responsible for performing feature engineering processing on the integrated dataset and screening out the features most relevant to risk assessment;

[0083] The model construction unit is responsible for selecting at least one algorithm from decision trees, random forests, and gradient boosting machines to construct a risk assessment model;

[0084] The model training unit is responsible for training the risk assessment model using the selected features. During the training process, the risk assessment model continuously adjusts its own parameters and structure to minimize the difference between the predicted risk level and the actual risk level, thereby gradually learning the mapping rules between different event feature combinations and corresponding risk levels;

[0085] The risk assessment model is responsible for quantitatively evaluating the risk levels of different events directly or by referring to the actual risks of historical similar events;

[0086] The risk warning unit is responsible for issuing a risk warning for events that reach the set risk level according to the risk assessment results.

[0087] In this embodiment, the involved user interface has the following functions:

[0088] Adopt responsive design to ensure a good user experience on different devices;

[0089] Provide a variety of personalized setting options, allowing data analysts to customize the interface according to their own needs, including event type filtering, time range selection, and risk level display;

[0090] Display data through charts and maps;

[0091] Support interactive queries, so that data analysts can input query conditions and obtain results through click and drag operations;

[0092] Support real-time data update to ensure that data analysts always have the latest information;

[0093] Have a user feedback mechanism to encourage data analysts to put forward improvement suggestions.

[0094] In this embodiment, the involved social event recognition and interpretation system further includes:

[0095] The training and optimization module is used to continuously collect and preprocess the new data collected by the acquisition and preprocessing module, and combine the automatic recognition of common problem models with the manual fine-tuning of rare problems to train and optimize the event recognition agent and the event interpretation agent. At the same time, adopt online learning and transfer learning technologies to enable the model to adapt to social changes and improve the accuracy of recognition and interpretation

[0096] The performance evaluation module is used to regularly evaluate the performance of the event recognition agent and the event interpretation agent based on preset rules, and adjust the agent parameters according to the evaluation results.

[0097] In this embodiment, the back-end of the system adopts big data processing technology to process and analyze a large amount of social event data; the system uses the Apache Hadoop distributed file system as a data storage solution, providing high-throughput data access to handle large-scale data sets; the system also adopts a MongoDB knowledge base to provide flexible storage and query capabilities for unstructured data; the system integrates Apache Spark to support complex data processing tasks such as SQL queries, machine learning, and graph processing; the system uses Spark Streaming for real-time data stream processing, which can efficiently process real-time social data, capture social dynamics, and provide instant data support for event recognition and risk assessment. The machine learning library (MLlib) of Spark can also be used to build and train machine learning models to improve the accuracy of social event recognition.

[0098] It should be added that: when designing the system, full consideration has been given to data security and user privacy protection. Encryption storage and transmission technologies are adopted to ensure the security of data during the processes of collection, processing, and storage. At the same time, the system follows relevant data protection regulations to anonymize user data and protect the privacy rights and interests of users.

[0099] It should be added that: the system adopts a modular design, which is easy to expand and maintain. The back-end services are deployed on a cloud platform, taking advantage of the elasticity and scalability of cloud computing to meet the needs of users of different scales. The front-end interface is implemented through Web technology, supporting cross-platform access, and users can access the system through a browser anytime and anywhere.

[0100] In summary, by using the social event recognition and interpretation system based on a large model of the present invention, automatic capture, classification, and preliminary analysis of social data can be achieved, thus greatly reducing the workload of data analysts. At the same time, it can more accurately predict development trends, provide more accurate and timely suggestions for relevant personnel, significantly improve the work efficiency of data analysts, reduce human errors, and provide more professional and in-depth social event analysis services.

[0101] The above specific application examples have elaborated in detail the principles and implementation methods of the present invention. These embodiments are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made by those skilled in the art of this technology field without departing from the principles of the present invention shall fall within the scope of patent protection of the present invention.

Claims

1. A social event recognition and interpretation system based on a large model, characterized in that: It includes: The data collection module is responsible for collecting social news data from the target platform in real time. The preprocessing module is responsible for preprocessing the collected data; The event recognition agent is responsible for automatically identifying, classifying, and ranking the importance of social events contained in the pre-processed data using the pre-trained large model; The event interpretation agent is responsible for interpreting and analyzing the sorted social events and generating corresponding event interpretation reports; The knowledge base construction module is responsible for collecting social events classified by the event recognition agent, extracting features related to periodic events, and building a knowledge base of periodic events; The monitoring and processing module is responsible for collecting social events classified by the event recognition agent, and judging whether it is a known periodic event in the knowledge base or a new periodic pattern through similarity retrieval technology. If it is a known periodic event, the actual risk of the corresponding historical similar event will be retrieved and sent to the risk assessment module. If it is a new periodic pattern, the features related to the periodic event will be extracted from the social event and the knowledge base will be updated. The risk assessment module is responsible for combining social news data and event interpretation results, referring to the actual risks of similar historical events, using statistical and machine learning methods to assess the potential risks of social events to investors and provide early warnings; The user interaction interface is responsible for providing customized query condition input and result display functions so that data analysts can obtain classification results, interpretation results and risk assessment results of social events.

2. According to the large model-based social event recognition and interpretation system of claim 1, it is characterized in that: The data acquisition module comprises: Data source docking unit, used to dock with target news platforms and government data platforms; The data collection unit is used to collect social news data from the docked target platform simultaneously by using web crawler technology combined with multi-threading and asynchronous IO mechanism; The data parsing unit is used to convert unstructured data in the collected data into structured data.

3. A social event recognition and interpretation system based on a large model according to claim 2, characterized in that: The preprocessing module specifically includes: The data cleaning unit is used to perform cleaning operations on the collected data, including removing duplicate records, correcting errors and filling missing values; The data formatting unit is used to standardize and normalize the cleaned data so that data from different sources and different dimensions can be analyzed within the same framework.

4. According to the large model-based social event recognition and interpretation system of claim 1, it is characterized in that: The event recognition agent specifically includes: The historical event retrospective unit is used to train a Transformer model with event recognition and classification using manually annotated historical data covering different types of events and their impacts. It is also used to use the Transformer model to deeply understand and classify historical social events, and to identify periodic events by combining time series analysis and clustering algorithms; The real-time event monitoring unit is used to analyze the newly-incoming social information in real time to identify new social events by leveraging the real-time stream processing framework and the online reasoning capability of the Transformer model, and introduces an attention mechanism to enhance the ability to capture key information; The event importance ranking unit is used to design a multi-dimensional scoring system that covers the coverage of events, the impact of similar historical events, and social media sentiment analysis, and uses a machine learning model to score and rank events.

5. According to the large model-based social event recognition and interpretation system of claim 1, it is characterized in that: The event interpretation agent specifically includes: The causal relationship analysis unit is used to apply graph neural networks to construct social entity relationship maps and analyze the causal relationship between events and social policies and social emotions; The prediction unit is used to use the time series prediction model combined with event characteristics to predict the scope and duration of the impact of events on different social fields, and introduce an attention mechanism to focus on key variables to improve prediction accuracy; The argument generation unit is used to extract similar event cases from historical data, compare and analyze the similarities and differences between current events and historical events, automatically generate an argument report that can reflect the impact of the event, and then use natural language generation technology to automatically generate an event interpretation report covering influencing factors, expected impact range and timeline based on the data analysis results.

6. A social event recognition and interpretation system based on a large model according to claim 1, characterized in that: The risk assessment module specifically includes: The data association unit is responsible for receiving the collected data from the collection and preprocessing module and the event interpretation report generated by the event interpretation agent, and finding the internal connection between these data according to the set rules and logic, so as to associate different data information of the same event or related events and integrate them into a data set about social events; The feature engineering unit is responsible for performing feature engineering on the integrated data set to select the features most relevant to risk assessment; The model building unit is responsible for selecting at least one algorithm from decision tree, random forest and gradient boosting machine to build a risk assessment model; The model training unit is responsible for training the risk assessment model using the selected features. During the training process, the risk assessment model continuously adjusts its own parameters and structure to minimize the difference between the predicted risk level and the actual risk level, thereby gradually learning the mapping rules between different event feature combinations and corresponding risk levels; Risk assessment model, which is responsible for quantitatively assessing the risk level of different events directly or by referring to the actual risks of similar historical events; The risk warning unit is responsible for issuing risk warnings for events that reach the set risk level based on the risk assessment results.

7. The social event recognition and interpretation system based on a large model according to claim 1 is characterized in that: The user interaction interface has the following functions: Adopt responsive design to ensure a good user experience on different devices; Provides a variety of personalization options, allowing data analysts to customize the interface according to their needs, including event type filtering, time range selection, and risk level display; Present data through charts and maps; Support interactive query so that data analysts can enter query conditions and obtain results by clicking and dragging; Support real-time data updates to ensure data analysts always have the latest information; There is a user feedback mechanism to encourage data analysts to make suggestions for improvements.

8. The social event recognition and interpretation system based on a large model according to claim 1 is characterized in that: The system further comprises: The training and optimization module is used to continuously collect new data collected by the collection and preprocessing modules, and train and optimize the event recognition agent and event interpretation agent by combining the automatic recognition of common problem models with manual fine-tuning of rare problems. At the same time, online learning and transfer learning technologies are used to enable the model to adapt to social changes and improve the accuracy of recognition and interpretation. The performance evaluation module is used to regularly evaluate the performance of the event recognition agent and the event interpretation agent based on preset rules, and adjust the agent parameters according to the evaluation results.

9. The social event recognition and interpretation system based on a large model according to claim 1 is characterized in that: The system backend uses big data processing technology to process and analyze massive social event data; The system uses the Apache Hadoop distributed file system for data storage and the MongoDB knowledge base to provide flexible storage and query capabilities for unstructured data; The system integrates Apache Spark to support complex data processing tasks, uses Spark Streaming for real-time data stream processing, processes real-time social data, captures social dynamics, and provides instant data support for event identification and risk assessment.

Images