Social media data mining agent system based on multi-module collaboration

Abstract

The invention discloses a social media data mining agent system based on multi-module collaboration, and the system comprises a target definition module which is used for determining a data mining direction through an agent according to a problem inputted by a user; the query generation module is used for generating a corresponding query statement by the intelligent agent according to the data mining direction, and obtaining information of a data mining object according to the query statement; the tool matching module is used for matching a data mining tool by an intelligent agent according to the information of the data mining object so as to execute a corresponding data mining algorithm to perform data mining and obtain a data mining result; the content screening module is used for screening one or more groups of valuable contents by an intelligent agent according to a data mining result, and generating corresponding visual chart configuration; and the chart analysis module is used for analyzing and generating a conclusion nodule by the intelligent agent according to the visual chart configuration. According to the method, the given social media data are automatically mined and analyzed by utilizing the intelligent agent, and the data mining efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of social media data mining, and particularly relates to a social media data mining intelligent agent system based on multi-module cooperation. BACKGROUND

[0002] In today's digital age, social media platforms have become the core hub of information dissemination and exchange, generating vast amounts of data every day, recording the rich interaction information, diverse opinions and dynamic trends among millions of users. Such complex and rapidly evolving data is crucial for understanding public opinion dynamics, information dissemination and topic evolution. Therefore, effective mining of social media data has become a valuable and far-reaching work.

[0003] In recent years, large language models have made remarkable new progress, bringing a promising new paradigm to automated data mining tasks. Large model-based intelligent agent systems exhibit strong capabilities, intelligently selecting appropriate data mining algorithms and precisely matching specific data characteristics and mining goals, automatically discovering insights hidden in data and generating information-rich data reports, thereby greatly reducing manual workload and significantly improving data mining efficiency.

[0004] However, developing an efficient data mining system is not easy, requiring a large investment of resources, including manpower, material and financial resources. Moreover, close cooperation between domain experts and data mining practitioners is crucial, but in practice, such cooperation often faces many difficulties. Iterative design workflows, including task definition, requirement establishment and process construction, may become inefficient and lack flexibility in urgent and time-constrained scenarios. In addition, existing social media data mining systems usually rely on predefined workflows, limiting their adaptability to new data and unforeseen mining requirements, and failing to adjust mining strategies in a timely manner. These problems highlight the need for more intelligent and automated methods.

[0005] Moreover, social media datasets have unique complexity, often involving multiple modalities, including text, user metadata and complex user networks. Text data contains rich semantic information, user metadata provides user background information, and complex user networks reflect user relationships and influence, requiring specialized algorithms to mine potential patterns, which presents unique mining challenges that current methods have not well supported.

[0006] In summary, considering the inherent diversity and dynamics of social media mining tasks, effectively capturing mining targets and dynamically selecting appropriate mining algorithms is highly challenging. Automatically determining the optimal mining method requires accurately interpreting the mining target, understanding the information that the user truly wants to obtain, and quickly adjusting the algorithm selection to adapt to the evolving data scenario, ensuring the accuracy and effectiveness of the mining results. Therefore, it is urgent to build an intelligent social media data mining algorithm dynamic selection and optimization framework to efficiently cope with the diversity and dynamics of mining tasks and improve mining efficiency and quality. SUMMARY

[0007] In view of the above, the purpose of the present application is to provide a social media data mining agent system based on multi-module cooperation, which uses an agent to automatically mine and analyze given social media data, improving data mining efficiency and quality.

[0008] To achieve the above-mentioned purpose of the application, the technical solutions provided by the present application are as follows: The social media data mining agent system based on multi-module cooperation provided by the embodiments of the present application comprises: A target definition module for summarizing data mining targets by the agent according to the user input question, determining data mining objects according to the targets, and selecting data mining directions about the objects; A query generation module for generating corresponding query statements by the agent according to the data mining directions, and obtaining information of the data mining objects according to the query statements; A tool matching module for matching data mining tools by the agent according to the information of the data mining objects to execute corresponding data mining algorithms for data mining, and obtaining data mining results; A content screening module for screening one or more groups of valuable content by the agent according to the data mining results, and generating corresponding visualization chart configurations; A chart analysis module for analyzing and generating summaries by the agent according to the visualization chart configurations.

[0009] Preferably, in each module, the agent uses the following mode to pass messages: , Wherein, Message represents the message passing mode, Action records the executed analysis operation, Result stores the key findings, Interpretation contains the agent's evaluation of these results, and Direction proposes the potential next step suggestions of the current node.

[0010] Preferably, the data mining objects include users and user-generated content.

[0011] Preferably, the data mining directions include: identity attributes, behavioral patterns, network topology, community formation, information pathways, and influence centers of users; content features, contextual metadata, distribution characteristics, participation structures, and interaction indicators of user-generated content.

[0012] Preferably, in the query generation module, the mode of data query by the agent is set as: , wherein Query represents the mode of data query, corresponding to a query statement, specifying the node type to be queried, defining the relationship type of interest, Filter setting the filtering conditions, and Propagation determining the expansion method of the query.

[0013] Preferably, the query statement specifically includes: Elasticsearch query statement, used to find relevant user-generated content from Elasticsearch according to keywords; SQL query statement, used to obtain detailed information of data mining objects from PostgreSQL database; Cypher query statement, used to find the relationship between data mining objects from Neo4J database.

[0014] Preferably, in the tool matching module, the mode of data mining by the agent is set as: , wherein Mining represents the mode of data mining, Subset specifies the input data subset obtained from the information of data mining objects obtained from the query generation module, defining the type of data mining algorithm to be applied, Param setting the algorithm parameters and configurations, and Eval determining the evaluation and optimization method of the mining results.

[0015] Preferably, the data mining algorithm corresponding to the data mining tool includes: sentiment detection algorithm, community detection algorithm, or topic modeling algorithm.

[0016] Preferably, the sentiment detection algorithm includes RoBERTa model-based detection algorithm, the community detection algorithm includes Louvain algorithm and label propagation algorithm, and the topic modeling algorithm includes LDA algorithm and BERTopic algorithm.

[0017] Preferably, the user analyzes and feeds back based on the mining results, visualization chart configurations, and summary given by the agent to optimize the future decisions of the agent, ensuring that the data mining process meets the user's expectations.

[0018] Compared with the prior art, the present application has the beneficial effects at least including: The present application provides a multi-module collaborative social media data mining agent system, which automatically mines and analyzes social media data using agents, fully considers existing conditions and user feedback during the process, thereby ensuring the effectiveness of the mining results, and further reducing the burden on users and improving data mining efficiency and quality. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0020] Figure 1 is a structural schematic diagram of the multi-module collaborative social media data mining agent system provided by the embodiments of the present application; Figure 2 is an execution flow schematic diagram of the multi-module collaborative social media data mining agent system provided by the embodiments of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the protection scope of the present application.

[0022] The inventive concept of the present application is that, in view of the problem of insufficient efficiency and quality of data mining in the prior art in the face of complex and diverse dynamic social media data, the embodiments of the present application provide a multi-module collaborative social media data mining agent system, which automatically summarizes data mining targets, determines mining objects and directions, generates query statements to obtain information, matches data mining tools and executes algorithms, filters valuable content to generate visual charts, and finally analyzes and generates summaries, to realize efficient and accurate data mining and visual display, so as to improve the quality and efficiency of data mining.

[0023] As shown in Figure 1 The embodiments provide a multi-module collaborative social media data mining agent system 100, which includes a target definition module 110, a query generation module 120, a tool matching module 130, a content filtering module 140 and a chart analysis module 150.

[0024] As shown in Figure 2The system execution flow is shown below, which mainly includes the following steps: S1, through the target definition module, based on the question input by the user, the agent summarizes the data mining target, determines the data mining object based on the target, and selects the data mining direction about the object.

[0025] In the embodiments, see Figure 2 The target In the directional process, the agent used is a large language model based on DeepSeek-V3. Different prompts are used to control its behavior, including system prompts and user prompts. System prompts introduce background knowledge to the large model and regulate its overall behavior, while user prompts describe the user's specific needs. After the user inputs the question, the agent summarizes the data mining objectives and determines the objects to be mined, then selects one or more of the most appropriate directions from specific data mining approaches.

[0026] The data mining objects include users and user-generated content (mainly posts). Data mining focuses on: user identity attributes, behavioral patterns, network topology, community formation, information pathways, and centers of influence; as well as the content characteristics, contextual metadata, distribution characteristics, participation structure, and interaction metrics of user-generated content.

[0027] Throughout the system, agents use the following pattern to communicate: , In this model, Message represents the message passing pattern, Action records the analysis operations performed, Result stores key findings, Interpretation contains the agent's evaluation of these results, and Direction proposes potential follow-up steps for the current node. This message passing pattern standardizes the model's output, requiring the large model to think in these directions, such as the appropriate actions, key findings in the current state, evaluations of existing results, and suggestions for the future. This ensures that the agent considers the existing process and results when making decisions at each step, thus avoiding loops or repetitions.

[0028] S2, through the query generation module, the agent generates corresponding query statements based on the data mining direction, and obtains information about the data mining object based on the query statements.

[0029] In the embodiments, see Figure 2 Excavation direction In the query process, the agent, based on the data mining direction determined in the previous step, generates appropriate data query statements according to the existing dataset content in order to obtain information about the data mining object. The data query mode of the intelligent agent is set as follows: , Here, Query represents the data query mode, corresponding to the query statement. Specify the type of node to query (including user or user-generated content). Define the types of relationships to focus on (including social connections, interactive behaviors, referencing relationships, etc.), set filtering conditions for filters (including time range, content features, attribute conditions, etc.), and determine the expansion method of the query (including network traversal strategy, search depth, etc.). This data query model enables the agent to clearly understand the query tasks it should complete, including the query input, output, purpose, and method, thus laying a solid data foundation for subsequent decision-making.

[0030] The query statement specifically includes: Elasticsearch query statements are used to find relevant user-generated content from Elasticsearch based on keywords; SQL query statements are used to retrieve detailed information about data mining objects from a PostgreSQL database; Cypher queries are used to find relationships between data mining objects in the Neo4J database.

[0031] S3, through the tool matching module, based on the information of the data mining object, the intelligent agent matches the data mining tool to execute the corresponding data mining algorithm to perform data mining and obtain the data mining results.

[0032] In the embodiments, see Figure 2 Information in tool In the parameterization process, the AI ​​samples the obtained dataset and then determines the appropriate data mining tools to use based on the sampling results. The data mining mode of the intelligent agent is set as follows: , Here, Mining represents the data mining pattern, and Subset specifies the subset of input data obtained from the information of the data mining objects obtained from the query generation module. Define the type of data mining algorithm to be applied (including clustering, classification, pattern mining, etc.), set the algorithm parameters and configuration with Param, and determine the evaluation and optimization methods for the mining results with Eval. This data mining mode makes full use of the data source obtained in the previous step, selects one or more of the best data mining algorithms from the available options, and optimizes them with multiple sets of different hyperparameters, ultimately enabling users to evaluate the mining effect from multiple perspectives.

[0033] Optional data mining algorithms include: Sentiment detection algorithms mainly include detection algorithms based on the RoBERTa model; Community detection algorithms mainly include the Louvain algorithm and the label propagation algorithm; Topic modeling algorithms mainly include LDA algorithm and BERTopic algorithm.

[0034] The agent also provides multiple sets of hyperparameters to adjust the tool's algorithm, allowing users to filter and judge its effectiveness. Specifically, based on the inherent characteristics of the tool's algorithm and its own knowledge base, the agent iterates within a broad parameter space of default hyperparameters. In each iteration, the agent analyzes the results of each algorithm based on a preset evaluation metric, Eval, that matches the characteristics of the tool's algorithm, optimizes one or more corresponding metrics, and adjusts the parameter Param based on the analysis results, gradually narrowing the search range in subsequent iterations. The core contribution of the system lies in the agent's control of this efficient iterative filtering process, ultimately selecting multiple sets of significantly effective locally optimal solutions from a vast array of initial parameter combinations, presenting them to the user for final selection. In the embodiment, based on this method and the agent's multiple runs, when the agent mines users, it typically uses a community detection algorithm, while when the agent mines posts, it typically uses a topic modeling algorithm.

[0035] S4, through the content filtering module, uses the intelligent agent to filter one or more sets of valuable content based on the data mining results, and generates corresponding visualization chart configurations.

[0036] In the embodiments, see Figure 2 effective The visualization chart configuration generation process combines user feedback and the agent's own judgment. The agent selects potentially valuable mining results and generates appropriate visualization chart configurations based on data characteristics and result features. These generated visualization chart configurations are primarily used for ECharts drawing, including but not limited to: line charts, bar charts, force-directed graphs, and word clouds.

[0037] S5 uses the chart analysis module to generate a summary based on the configured visualization charts, and the agent analyzes and summarizes the data.

[0038] In the embodiments, see Figure 2 The results In the summary process, the agent combines the mining results and information from the visualization charts to summarize the mining process. Finally, users can analyze and provide feedback based on the mining results, visualization chart configurations, and summary provided by the agent to optimize future decisions and ensure that the data mining process meets user expectations.

[0039] The above detailed description of the specific embodiments of the present application has described the technical solutions and beneficial effects of the present application, and it should be understood that the above description is only the most preferred embodiment of the present application and is not intended to limit the present application. Any modifications, supplements and equivalent replacements made within the principle range of the present application shall be included in the protection range of the present application.

Claims

1. A social media data mining intelligent agent system based on multi-module collaboration, characterized in that, include: The target definition module is used to have the agent summarize the data mining target based on the user-input question, determine the data mining object based on the target, and select the data mining direction about the object; The query generation module is used to generate corresponding query statements by the intelligent agent based on the data mining direction, and to obtain information about the data mining object based on the query statements; The tool matching module is used to match data mining tools with intelligent agents based on the information of the data mining object, so as to execute the corresponding data mining algorithm to perform data mining and obtain data mining results. The content filtering module is used to filter one or more sets of valuable content by an intelligent agent based on data mining results and generate corresponding visualization chart configurations. The chart analysis module is used to generate summary summaries by the agent based on the configuration of the visualization charts.

2. The social media data mining intelligent agent system based on multi-module collaboration according to claim 1, characterized in that, Within each module, the agent uses the following pattern to pass messages: ， Here, Message represents the message passing pattern, Action records the analysis operations performed, Result stores key findings, Interpretation contains the agent's evaluation of these results, and Direction proposes potential follow-up steps for the current node.

3. The social media data mining intelligent agent system based on multi-module collaboration according to claim 1, characterized in that, Data mining objects include users and user-generated content.

4. The social media data mining intelligent agent system based on multi-module collaboration according to claim 3, characterized in that, Data mining areas include: User identity attributes, behavioral patterns, network topology, community formation, information channels, and centers of influence; User-generated content features, contextual metadata, distribution characteristics, participation structure, and interaction metrics.

5. The social media data mining intelligent agent system based on multi-module collaboration according to claim 1 or 3, characterized in that, In the query generation module, the data query mode of the intelligent agent is set as follows: ， Here, Query represents the data query mode, corresponding to the query statement. Specify the type of node to query. Define the type of relationship you want to focus on, set the filter criteria, and determine how the query is expanded using Propagation.

6. The social media data mining intelligent agent system based on multi-module collaboration according to claim 5, characterized in that, The query statement specifically includes: Elasticsearch query statements are used to find relevant user-generated content from Elasticsearch based on keywords; SQL query statements are used to retrieve detailed information about data mining objects from a PostgreSQL database; Cypher queries are used to find relationships between data mining objects in the Neo4J database.

7. The social media data mining intelligent agent system based on multi-module collaboration according to claim 1, characterized in that, In the tool matching module, the data mining mode of the agent is set as follows: ， Here, Mining represents the data mining pattern, and Subset specifies the subset of input data obtained from the information of the data mining objects obtained from the query generation module. Define the type of data mining algorithm to be applied, set the algorithm parameters and configuration with Param, and determine the evaluation and optimization methods for the mining results with Eval.

8. The social media data mining intelligent agent system based on multi-module collaboration according to claim 1, characterized in that, The data mining algorithms corresponding to data mining tools include: Sentiment detection algorithms, community detection algorithms, or topic modeling algorithms.

9. The social media data mining intelligent agent system based on multi-module collaboration according to claim 8, characterized in that, Sentiment detection algorithms include RoBERTa model-based detection algorithms; community detection algorithms include Louvain algorithm and label propagation algorithm; and topic modeling algorithms include LDA algorithm and BERTopic algorithm.

10. The social media data mining intelligent agent system based on multi-module collaboration according to claim 1, characterized in that, Users analyze and provide feedback based on the mining results, visualization charts, and summaries provided by the agent to optimize the agent's future decisions and ensure that the data mining process meets user expectations.

Images