Artificial intelligence-based media data analysis method and system

Abstract

The application belongs to the technical field of data analysis, and discloses a media data analysis method and system based on artificial intelligence.The method comprises the following steps: building a media data crawling engine, a media data processing engine and a media data analysis engine in a media data analysis platform; using the media data crawling engine to generate real-time data crawling strategies of different media platforms, and crawling a plurality of real-time multi-modal media data; using the media data processing engine to process the plurality of real-time multi-modal media data to obtain a plurality of real-time media data clustering clusters and real-time cluster portraits thereof; and using the media data analysis engine to perform multi-dimensional analysis according to the plurality of real-time media data clustering clusters and real-time cluster portraits thereof to obtain a real-time media data analysis report.The application solves the problems of the prior art, such as data collection lag and one-sidedness, insufficient multi-modal data processing capacity and single analysis dimension.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data analysis, and particularly relates to a media data analysis method and system based on artificial intelligence. BACKGROUND

[0002] With the rapid development of the Internet and social media, the speed and breadth of information dissemination are growing exponentially. Media platforms (such as news websites, social media, video platforms, etc.) generate massive amounts of multi-modal data every day, including text, images, videos, user information, and interaction data. These data contain rich information about social hotspots, public sentiment, and communication patterns, which are of great significance for media organizations, governments, and enterprises to understand public opinion, develop strategies, and optimize communication.

[0003] However, traditional media data analysis methods often face the following challenges:

[0004] 1) Data collection has lag and one-sidedness: Most existing methods rely on timed scraping or offline batch processing, which cannot capture the real-time dissemination dynamics of information. Data scraping is often limited to pre-built databases on the Internet, ignoring many emerging, vertical, or regional social media, forums, short video media platforms, etc., resulting in a lack of comprehensiveness in the analysis results, which may miss important information sources or user groups. Moreover, media data crawling strategies are mostly fixed rules or simple heuristics, which are difficult to adapt to the complex and changing page structures of different platforms, anti-crawling mechanisms, and real-time priority changes in data updates.

[0005] 2) Insufficient multi-modal data processing capability: Many existing technologies can only process a single modality (such as pure text) or superficially process non-text modalities (such as images, videos) (such as only extracting titles or simple tags). Text, images, videos, user information, and interaction data are often processed separately, and cross-modal correlation analysis and information fusion cannot be performed.

[0006] 3) Single analysis dimension: Most existing technologies analyze media data in a single dimension such as data volume, theme, and user group, resulting in low value of media data analysis results, and lack of systematic integration of analysis results from multiple dimensions. SUMMARY

[0007] To solve the problems of existing technologies such as data collection lag and one-sidedness, insufficient multi-modal data processing capability, and single analysis dimension, the present application aims to provide a media data analysis method and system based on artificial intelligence.

[0008] The technical solution adopted by the present application is:

[0009] A media data analysis method based on artificial intelligence, comprising the following steps:

[0010] Based on artificial intelligence algorithm, a media data crawling engine, a media data processing engine and a media data analysis engine are built in the media data analysis platform;

[0011] Using the media data crawling engine, real-time data crawling strategies of different media platforms are generated, and according to the real-time data crawling strategies, a plurality of real-time multi-modal media data are crawled;

[0012] Using the media data processing engine, the plurality of real-time multi-modal media data are processed to obtain a plurality of real-time media data clustering clusters and real-time cluster portraits thereof;

[0013] According to the plurality of real-time media data clustering clusters and real-time cluster portraits thereof, the media data analysis engine is used for multi-dimensional analysis to obtain a real-time media data analysis report.

[0014] Further, the real-time multi-modal media data includes real-time text data, real-time image data, real-time video data, real-time user information data and real-time interaction data of the media platform.

[0015] Further, the media data crawling engine is provided with a media data crawling interface, a media data crawling executor and a data crawling strategy generator connected in sequence, the media data crawling interface is connected to a plurality of external media platforms, and the data crawling strategy generator is provided with a data crawling strategy generation model;

[0016] The data crawling strategy generation model is constructed based on the MAPGRPO algorithm, and the data crawling strategy generation model includes a platform-level data crawling strategy generation layer, a coordination layer and a server-level data crawling strategy generation layer connected in sequence, the platform-level data crawling strategy generation layer is provided with a first multi-optimization target set and a platform-level agent, the coordination layer is provided with a coordination agent and a multi-target conflict resolution mechanism, and the server-level data crawling strategy generation layer is provided with a second multi-optimization target set and a plurality of parallel server-level agents.

[0017] Further, the media data processing engine is provided with a media data preprocessor, a multi-modal feature extractor, a multi-modal feature space-time aligner, a time-space perception incremental clusterer and a cluster portrait generator connected in sequence;

[0018] The multi-modal feature extractor is provided with a multi-modal feature extraction model, which is constructed based on a BERT-ResNet-CNN-RNN-Wav2Vec2.0-AE algorithm, and includes a text feature extraction module constructed based on a BERT algorithm, an image feature extraction module constructed based on a ResNet algorithm, a video feature extraction module constructed based on a CNN-RNN-Wav2Vec2.0 algorithm, a user feature extraction module constructed based on an AE algorithm, and a propagation feature extraction module constructed based on an AE algorithm;

[0019] The space-time perception incremental clusterer is provided with a space-time perception incremental clustering model, which is constructed based on an STICM algorithm, and includes a feature clustering module constructed based on an ST-DBSCAN algorithm, a new data processing module constructed based on an IP algorithm, and a cluster management module constructed based on a DCM algorithm, which are connected in sequence.

[0020] The cluster cluster portrait generator is provided with a cluster cluster portrait generation model, which is constructed based on a GCN-DualGAN algorithm, and includes a cluster relationship construction module constructed based on a GCN algorithm and a cluster cluster portrait generation module constructed based on a DualGAN algorithm, which are connected in sequence.

[0021] Further, the media data analysis engine is provided with a media data analysis model, which includes a feature selection module constructed based on a CA algorithm, a multi-dimensional analysis module constructed based on an MDA algorithm, and a report generation module constructed based on a cGAN algorithm, which are connected in sequence, and the multi-dimensional analysis module is provided with a plurality of dimension analysis channels connected in parallel.

[0022] The dimension analysis channel includes a content theme evolution analysis channel constructed based on an LDA-DTM algorithm, a sentiment distribution analysis channel constructed based on a VADER-LSTM algorithm, a key propagation analysis channel constructed based on an SNA algorithm, a user portrait analysis channel constructed based on a GMM-SVM algorithm, a space-time distribution analysis channel constructed based on an STGNN algorithm, and a media content analysis channel constructed based on a VGG algorithm.

[0023] Further, the media data crawling engine is used to generate real-time data crawling strategies for different media platforms, and according to the real-time data crawling strategies, a plurality of real-time multi-modal media data are crawled, including the following steps:

[0024] The media data crawling interface of the media data crawling engine is used to collect basic information of different media platforms, and the basic information is input into the data crawling strategy generator;

[0025] According to the basic information, the data crawling strategy generator of the media data crawling engine is used for strategy generation to obtain a real-time data crawling strategy;

[0026] According to the real-time data crawling strategy, the media data crawling executor of the media data crawling engine is used to crawl a plurality of real-time multi-modal media data of the corresponding media platform.

[0027] Further, the media data processing engine is used to process the plurality of real-time multi-modal media data to obtain a plurality of real-time media data clustering clusters and real-time cluster portraits thereof, including the following steps:

[0028] The media data preprocessor of the media data processing engine is used to preprocess the plurality of real-time multi-modal media data from different media platforms to obtain a plurality of preprocessed real-time multi-modal media data;

[0029] The multi-modal feature extractor of the media data processing engine is used to extract real-time multi-modal feature vectors of the plurality of preprocessed real-time multi-modal media data;

[0030] The multi-modal feature spatio-temporal aligner of the media data processing engine is used to perform spatio-temporal alignment on the plurality of real-time multi-modal feature vectors to obtain a real-time spatio-temporal alignment matrix including the plurality of real-time multi-modal feature vectors;

[0031] The spatio-temporal perception incremental clusterer of the media data processing engine is used to cluster the plurality of real-time multi-modal feature vectors in the real-time spatio-temporal alignment matrix to obtain a plurality of real-time media data clustering clusters;

[0032] The clustering cluster portrait generator of the media data processing engine is used to generate a real-time cluster portrait corresponding to each real-time media data clustering cluster.

[0033] Further, the multi-dimensional analysis includes content theme evolution analysis, sentiment distribution analysis, key propagation analysis, user portrait analysis, spatio-temporal distribution analysis, and media content analysis.

[0034] Further, according to the plurality of real-time media data clustering clusters and real-time cluster portraits thereof, the media data analysis engine is used for multi-dimensional analysis to obtain a real-time media data analysis report, including the following steps:

[0035] The real-time media data clustering clusters and real-time cluster portraits thereof are input into the media data analysis engine, and the feature selection module of the media data analysis engine is used to select real-time key features of different dimensions;

[0036] The real-time key features of different dimensions are input into the multi-dimensional analysis module of the media data analysis engine corresponding to the dimension analysis channel for analysis to obtain corresponding real-time dimension analysis results;

[0037] Integrating the real-time dimensional analysis results of all dimensional analysis channel inputs to obtain real-time media data clustering clusters and their corresponding real-time media data analysis results of real-time cluster portraits;

[0038] According to the real-time media data analysis results of all real-time media data clustering clusters, a report generation module of the media data analysis engine is used to generate a real-time media data analysis report.

[0039] A media data analysis system based on artificial intelligence is used to implement a media data analysis method, the system is arranged on a media data analysis platform, and the system comprises a media data crawling engine, a media data processing engine and a media data analysis engine connected in sequence, and the media data crawling engine is connected with a plurality of external media platforms.

[0040] The beneficial effects of the present application are:

[0041] The media data analysis method and system based on artificial intelligence provided by the present application can dynamically generate real-time strategies through the media data crawling engine driven by artificial intelligence, efficiently crawl data, combine the rapid response of the processing and analysis engine, and make the entire analysis process closely follow the real-time dynamics of information dissemination, greatly shorten the time from data generation to analysis result output, solve the core problem of analysis lag in the prior art, and can instantly capture the rise and evolution of sudden events and hot topics, provide valuable first-time insights for public opinion monitoring, crisis warning, market response, etc., the media data crawling engine can generate strategies for different media platforms, effectively cover mainstream and emerging platforms, avoid information blind spots, make the analysis results more representative, dynamically adjust the crawling strategy according to data feedback and targets, automatically adapt to platform changes and hot topic shifts, improve the efficiency and pertinence of data acquisition, and improve the comprehensiveness and intelligence of data coverage; the media data processing engine can uniformly process various modal data such as text, image and video, identify the association between them, provide a more three-dimensional and complete information view, and effectively extract and analyze the spatiotemporal attributes of data to reveal the geographical distribution and time regularity of information dissemination, provide a new dimension for understanding the dissemination mechanism, the feature selection, multi-dimensional analysis and other modules of the media data processing engine are driven by artificial intelligence, can automatically identify key information, perform more in-depth and objective analysis, and reduce manual intervention and subjective bias; the media data analysis engine performs media data analysis from multiple dimensions such as content theme evolution analysis, sentiment distribution analysis, key dissemination analysis, user portrait analysis, spatiotemporal distribution analysis and media content analysis, improves the value of media data analysis results, and obtains a systematic integrated analysis report, improves the convenience and efficiency of media data analysis.

[0042] Other beneficial effects of the present application will be further described in the specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a flow chart of the media data analysis method based on artificial intelligence in the present application.

[0044] Figure 2 is a structural diagram of the media data analysis system based on artificial intelligence in the present application. DETAILED DESCRIPTION

[0045] The present application will be further explained in conjunction with the accompanying drawings and specific embodiments.

[0046] Embodiment 1:

[0047] As shown in Figure 1 , the present embodiment provides a media data analysis method based on artificial intelligence, comprising the following steps:

[0048] S1: based on artificial intelligence algorithm, building media data crawling engine, media data processing engine and media data analysis engine in media data analysis platform;

[0049] The media data crawling engine is provided with a media data crawling interface, a media data crawling executor and a data crawling strategy generator connected in sequence, the media data crawling interface is connected to a plurality of external media platforms (such as Weibo, WeChat public account, Douyin, B station, Zhihu, news website, etc.), and the data crawling strategy generator is provided with a data crawling strategy generation model;

[0050] The data crawling strategy generation model is constructed based on a Multi-Agent Parallel Group Relative Policy Optimization (MAPGRPO) algorithm, and includes a platform-level data crawling strategy generation layer, a coordination layer, and a server-level data crawling strategy generation layer connected in sequence. The platform-level data crawling strategy generation layer is provided with a first multi-optimization objective set (for example: total data volume, data quality, unit time cost, compliance risk index) and platform-level agents, responsible for formulating global collection priority, resource allocation strategy, bypassing anti-crawler mechanism, etc. The coordination layer is provided with a coordination agent and a multi-objective conflict resolution mechanism, used for processing conflicts between platform levels and platform-level objectives (for example, the system wants to be fast, but the platform level finds that it needs to be slow), and coordinating the behavior of each platform-level agent to ensure the synergy of the overall strategy. The server-level data crawling strategy generation layer is provided with a second multi-optimization objective set (for example: specific platform data acquisition success rate, request delay, detection probability) and a plurality of parallel target platform-level agents. Each agent corresponds to one or a type of target media platform, responsible for generating specific request frequency, User-Agent rotation, proxy IP usage, Cookie management, session maintenance, and bypassing sub-strategies for specific anti-crawling mechanisms of the platform (such as verification code recognition trigger processing, sliding verification simulation, and human-computer verification response, etc.).

[0051] The media data processing engine is provided with a media data preprocessor, a multi-modal feature extractor, a multi-modal feature space-time aligner, a space-time perception incremental clusterer, and a cluster cluster portrait generator connected in sequence.

[0052] The multi-modal feature extractor is provided with a multi-modal feature extraction model, the multi-modal feature extraction model is constructed based on a BERT-ResNet-CNN-RNN-Wav2Vec2.0-AE algorithm, and the multi-modal feature extraction model comprises a text feature extraction module constructed based on a BERT algorithm, an image feature extraction module constructed based on a ResNet algorithm, a video feature extraction module constructed based on a CNN-RNN-Wav2Vec2.0 algorithm, a user feature extraction module constructed based on an AE algorithm, and a propagation feature extraction module constructed based on an AE algorithm, wherein the English full name of BERT is “Bidirectional Encoder Representations from Transformers”, the Chinese name is “Bidirectional Encoder Representations from Transformers”, the English full name of ResNet is “Residual Network”, the Chinese name is “Residual Network”, the English full name of CNN is “Convolutional Neural Network”, the Chinese name is “Convolutional Neural Network”, the English full name of RNN is “Recurrent Neural Network”, the Chinese name is “Recurrent Neural Network”, the English full name of Wav2Vec2.0 is “Waveform to Vector 2.0”, the Chinese name is “Waveform to Vector 2.0”, and the English full name of AE is “AutoEncoder”, the Chinese name is “AutoEncoder”;

[0053] The spatio-temporal perception incremental clusterer is provided with a spatio-temporal perception incremental clustering model, the spatio-temporal perception incremental clustering model is constructed based on a spatio-temporal incremental clustering method (STICM), and the spatio-temporal perception incremental clustering model is provided with a feature clustering module constructed based on a spatial-temporal density-based spatial clustering of applications with noise (ST-DBSCAN) in sequence, a new data processing module constructed based on an incremental processing (IP) algorithm, and a cluster management module constructed based on a dynamic cluster management (DCM) algorithm;

[0054] The clustering cluster portrait generator is provided with a clustering cluster portrait generation model, the clustering cluster portrait generation model is constructed based on a GCN-DualGAN algorithm, and the clustering cluster portrait generation model comprises a clustering relationship construction module constructed based on a GCN algorithm and a clustering cluster portrait generation module constructed based on a DualGAN algorithm connected in sequence, wherein the English full name of GCN is "Graph Convolutional Network", the Chinese name is "graph convolutional network", the English full name of DualGAN is "Dual Generative Adversarial Network", and the Chinese name is "double generative adversarial network";

[0055] The media data analysis engine is provided with a media data analysis model, the media data analysis model comprises a feature selection module constructed based on a channel attention (Channel Attention, CA) algorithm, a multi-dimensional analysis module constructed based on a multi-dimensional analysis (Multi-dimensional Analysis, MDA) algorithm and a report generation module constructed based on a conditional generative adversarial network (Conditional Generative Adversarial Network, cGAN) algorithm connected in sequence, and the multi-dimensional analysis module is provided with a plurality of dimension analysis channels connected in parallel;

[0056] The dimension analysis channel includes a content theme evolution analysis channel constructed based on an LDA-DTM algorithm, a sentiment distribution analysis channel constructed based on a VADER-LSTM algorithm, a key propagation analysis channel constructed based on an SNA algorithm, a user portrait analysis channel constructed based on a GMM-SVM algorithm, a spatio-temporal distribution analysis channel constructed based on an STGNN algorithm, and a media content analysis channel constructed based on a VGG algorithm. The English full name of LDA is "Dynamic Topic Model", and the Chinese name is "Latent Dirichlet Allocation". The English full name of DTM is "Dual Generative Adversarial Network", and the Chinese name is "Dynamic Topic Model". The English full name of VADER is "Valence Aware Dictionary and sEntiment Reasoner", and the Chinese name is "Emotion Aware Dictionary and Sentiment Reasoner". The English full name of LSTM is "Long Short-Term Memory", and the Chinese name is "Long Short-Term Memory Network". The English full name of SNA is "Social Network Analysis", and the Chinese name is "Social Network Analysis". The English full name of GMM is "Gaussian Mixture Model", and the Chinese name is "Gaussian Mixture Model". The English full name of SVM is "Support Vector Machine", and the Chinese name is "Support Vector Machine". The English full name of STGNN is "Spatio-Temporal Graph Neural Network", and the Chinese name is "Spatio-Temporal Graph Neural Network". The English full name of VGG is "Visual Geometry Group", and the Chinese name is "Visual Geometry Group".

[0057] S2: using a media data crawling engine, generating a real-time data crawling strategy of different media platforms, and crawling a plurality of real-time multi-modal media data according to the real-time data crawling strategy, including the following steps:

[0058] S2-1: using the media data crawling interface of the media data crawling engine, collecting basic information (such as website structure, historical anti-crawling behavior mode, data value evaluation) of different media platforms, and inputting the basic information into the data crawling strategy generator;

[0059] S2-2: according to the basic information, using the data crawling strategy generator of the media data crawling engine to generate a strategy, obtaining a real-time data crawling strategy, including the following steps:

[0060] S2-2-1: According to the basic information, the platform-level agent of the platform-level data crawling strategy generation layer of the platform using the data crawling strategy generation model generates a platform-level intelligent agent, combines the first multi-optimization objective set, and generates a strategy to obtain a platform-level real-time data crawling strategy , wherein is the th platform-level real-time data crawling strategy , wherein is the th platform-level real-time data crawling strategy

[0061] , wherein

[0062] S2-2-2: Based on the multi-objective conflict resolution mechanism, according to the state of the media data analysis platform, the coordination agent of the coordination layer using the data crawling strategy generation model resolves the multi-objective conflict of the platform-level real-time data crawling strategy to obtain an optimized platform-level real-time data crawling strategy, including the following steps:

[0063] S2-2-2-1: Based on the multi-objective conflict resolution mechanism, using the third generation non-dominated sorting genetic algorithm (Non-dominated Sorting Genetic Algorithm III, NSGA-III) algorithm, obtaining a real-time Pareto frontier solution set from the platform-level real-time data crawling strategy, and obtaining a real-time strategy similarity matrix of the real-time Pareto frontier solution set, including the following steps: , wherein

[0064] S2-2-2-1-2: Using the NSGA-III algorithm, generating a Pareto frontier solution set containing strategies from the strategy pool , wherein is the total number of Pareto frontier solutions; these strategies are Pareto optimal among multiple objectives

[0065] S2-2-2-1-3: Calculate the similarity between any two strategies in the Pareto frontier set, and construct a similarity matrix , the similarity matrix is symmetric

[0066] ​The platform-level real-time data crawling strategy can contain multiple objectives, and there can be conflicts among these objectives. NSGA-III is a multi-objective optimization algorithm that filters out the Pareto front solution from the given strategy set of the platform-level real-time data crawling strategy. The solution on the Pareto front represents the "non-dominated" solution that cannot improve any objective without sacrificing at least one other objective in the current strategy set.

[0067] If the strategies in the strategy set of the platform-level real-time data crawling strategy perform significantly differently on multiple objectives, the Pareto front obtained by NSGA-III can be relatively "dispersed", which indirectly reflects that there are more obvious multi-objective conflicts in the original strategy set. Conversely, if the Pareto front is relatively "compact", it may indicate that the conflict is relatively small, or the original strategy is already close to the Pareto optimal;

[0068] The Pareto front solution set itself also provides a set of potential multi-objective optimized strategy options, which represent different possibilities of trade-offs between conflicting objectives.

[0069] The similarity matrix quantifies the degree of similarity between different strategies on the Pareto front. Similar strategies may perform similarly on some objectives, but may differ on other objectives. By analyzing the similarity matrix, it can be identified which strategy combination may cause more serious conflicts (for example, highly similar but large difference in objective value strategies) or which strategy is highly redundant.

[0070] S2-2-2-2: Integrate the media data analysis platform state, real-time Pareto front solution set, and real-time strategy similarity matrix to obtain the real-time complete state observation that the coordination agent in the coordination layer of the data crawling strategy generation model can identify , wherein, is the encoding representation of the strategy in the Pareto front set, is the real-time complete state observation, is t the media data analysis platform state at time t, is the similarity matrix, t is the time indicator;

[0071] S2-2-2-3: Map the real-time complete state observation to the coordination state space of the coordination agent to obtain the updated coordination state space of the coordination agent;

[0072] S2-2-2-4: According to the updated coordination state empty space and the preset coordination action space (selecting a strategy in the real-time Pareto frontier solution set), using the coordination agent, multi-objective conflict resolution is performed to obtain an optimized platform-level real-time data crawling strategy, and the optimized platform-level real-time data crawling strategy is transmitted to the server-level data crawling strategy generation layer of the data crawling strategy generation model;

[0073] S2-2-3: According to the basic information, the media data analysis platform state, and the optimized platform-level real-time data crawling strategy, using the server-level agent of the server-level data crawling strategy generation layer of the data crawling strategy generation model, strategy generation is performed to obtain server-level real-time data crawling strategies for a plurality of media platforms;

[0074] S2-2-4: Integrating the optimized platform-level real-time data crawling strategy and the plurality of server-level real-time data crawling strategies, real-time data crawling strategies for each media platform are obtained;

[0075] S2-3: According to the real-time data crawling strategy, using the media data crawling executor of the media data crawling engine, a plurality of real-time multi-modal media data of the corresponding media platform are crawled;

[0076] The real-time multi-modal media data includes real-time text data, real-time image data, real-time video data, real-time user information data, and real-time interaction data of the media platform;

[0077] S3: Using the media data processing engine, a plurality of real-time multi-modal media data are processed to obtain a plurality of real-time media data clustering clusters and real-time cluster portraits, including the following steps:

[0078] S3-1: Using the media data preprocessor of the media data processing engine, a plurality of real-time multi-modal media data from different media platforms are preprocessed to obtain a plurality of preprocessed real-time multi-modal media data;

[0079] The preprocessing is a cleaning and standardization process for the collected raw data, including removing noise data (such as hypertext markup language tags, special characters, and duplicate data), handling missing values, text segmentation, removing stop words, part-of-speech tagging, and stem extraction, etc. Natural language processing operations; For image and video data, format conversion, resolution adjustment, etc. Preprocessing; Different sources and different formats of data are uniformly converted into structured or semi-structured data formats for subsequent analysis;

[0080] S3-2: Using the multi-modal feature extractor of the media data processing engine, real-time multi-modal feature vectors of the plurality of preprocessed real-time multi-modal media data are extracted, including the following steps:

[0081] S3-2-1: inputting the pre-processed real-time multi-modal media data into a multi-modal feature extraction model of a multi-modal feature extractor of a media data processing engine;

[0082] S3-2-2: using a text feature extraction module of the multi-modal feature extraction model to extract real-time text features of real-time text data in the real-time multi-modal media data; the real-time text features include semantic vectors, capturing deep semantic information, and keywords, topic labels, sentiment polarity;

[0083] S3-2-3: using an image feature extraction module of the multi-modal feature extraction model to extract real-time image features of real-time image data in the real-time multi-modal media data; the real-time image features are visual feature vectors, used to identify image content, scene, and object;

[0084] S3-2-4: using a video feature extraction module of the multi-modal feature extraction model to extract real-time video features of real-time video data in the real-time multi-modal media data;

[0085] The real-time video features include real-time frame-level features of the real-time video data extracted based on a CNN algorithm, real-time time-series features of the real-time video data extracted based on an RNN algorithm, and real-time audio features of the real-time video data extracted based on a Wav2Vec2.0 algorithm;

[0086] S3-2-6: using a user feature extraction module of the multi-modal feature extraction model to extract real-time user features of real-time user information data in the real-time multi-modal media data; the real-time user features include user ID, number of fans, activity level, historical published content labels, etc.

[0087] S3-2-7: using a propagation feature extraction module of the multi-modal feature extraction model to extract real-time propagation features of real-time interaction data in the real-time multi-modal media data; the real-time propagation features include number of forwards, number of comments, number of likes, number of shares, etc.

[0088] S3-2-8: concatenating and combining the real-time text features, real-time image features, real-time video features, real-time user features, and real-time propagation features to obtain corresponding real-time multi-modal feature vectors;

[0089] S3-3: using a multi-modal feature spatio-temporal aligner of the media data processing engine to perform spatio-temporal alignment on the real-time multi-modal feature vectors to obtain a real-time spatio-temporal alignment matrix including the real-time multi-modal feature vectors, including the following steps:

[0090] S3-3-1: aligning the timestamps of all real-time multi-modal feature vectors with a unified time reference, processing cases of missing or incorrect timestamps (such as using an interpolation method or inferring based on content similarity), to obtain a plurality of aligned real-time multi-modal feature vectors;

[0091] S3-3-2: Calculate the time distance (considering the time decay effect, the recent data has higher weight) and content similarity (based on the similarity of the fused multi-modal feature vectors, such as using cosine similarity or Siamese network metric) between the aligned real-time multi-modal feature vectors;

[0092] S3-3-3: According to the time distance and content similarity of several aligned real-time multi-modal feature vectors, construct a spatio-temporal alignment matrix, and represent the data points of the spatio-temporal alignment matrix corresponding to each aligned real-time multi-modal feature vector as an information vector containing its multi-modal feature, timestamp, spatial position (if available, such as user geographic location) and spatio-temporal relationship with other data points;

[0093] S3-4: Use the spatio-temporal aware incremental clusterer of the media data processing engine to cluster the several real-time multi-modal feature vectors in the real-time spatio-temporal alignment matrix, and obtain several real-time media data clustering clusters;

[0094] S3-4-1: Use the feature clustering module of the spatio-temporal aware incremental clusterer of the media data processing engine to cluster the several real-time multi-modal feature vectors in the real-time spatio-temporal alignment matrix in the initial time period, find the spatio-temporal cluster of any shape, and obtain several real-time media data clustering clusters; the core parameters (such as neighborhood radius, minimum point number) are dynamically adjusted according to the data density;

[0095] S3-4-2: For the newly input real-time multi-modal feature vector, use the new data processing module for incremental update;

[0096] First, judge the spatio-temporal proximity of the newly input real-time multi-modal feature vector and the existing clusters (based on the distance between the new data and the core points of each cluster calculated by the spatio-temporal alignment matrix); if the newly input real-time multi-modal feature vector falls within the neighborhood of a cluster, assign it to the cluster and update the spatio-temporal boundary and feature of the cluster; if the newly input real-time multi-modal feature vector does not fall within the neighborhood of any cluster, but meets the condition of forming a new cluster (there are enough new data points around to form a density connection), create a new cluster; if the newly input real-time multi-modal feature vector does not fall within the existing cluster, nor meets the condition of forming a new cluster, mark it as noise or pending data temporarily;

[0097] S3-4-3: Use the cluster management module to manage the spatio-temporal clusters corresponding to the several real-time media data clustering clusters;

[0098] Periodically assess the activity of existing clusters (such as the rate of new data inflow, the rate of content change within the cluster); for "aging" clusters that have no new data inflow for a long time or have very low activity, mark or archive them to avoid interfering with real-time analysis; monitor the evolution of clusters, identify dynamic behaviors such as cluster splitting, merging, migration, etc., and record their evolution trajectory;

[0099] S3-5: Using the cluster cluster portrait generator of the media data processing engine, generate a real-time cluster portrait corresponding to each real-time media data cluster cluster, including the following steps:

[0100] S3-5-1: Define the portrait feature dimensions of the cluster portrait, including but not limited to: topic information, sentiment tendency information, key person / agency, typical user portrait (age, gender, interest label inference), propagation mode (explosive / continuous / cyclical), regional distribution (if data support), spatiotemporal pattern, and association with other clusters, etc.

[0101] S3-5-2: Input each real-time media data cluster cluster into the cluster cluster portrait generation model of the cluster cluster portrait generator of the media data processing engine;

[0102] S3-5-3: Use the cluster relationship construction module of the cluster cluster portrait generation model to model the real-time multi-modal feature vectors within the real-time media data cluster cluster, and obtain a set of real-time node embedding vectors including real-time relationship edges between real-time multi-modal feature vectors;

[0103] Each real-time multi-modal feature vector data point is regarded as a node in the graph, and the node feature is its multi-modal fusion feature, and the edge represents the similarity or propagation relationship between data points; through GNN node representation learning, obtain the embedding vector of each data point in the low-dimensional space, which can better reflect its structure and semantic information within the cluster;

[0104] S3-5-4: According to the portrait feature dimensions, use the cluster cluster portrait generation module of the cluster cluster portrait generation model to generate a portrait for the set of real-time node embedding vectors, and obtain a real-time cluster portrait corresponding to each real-time media data cluster cluster;

[0105] The cluster cluster portrait generation module includes a generator G and a discriminator D. The generator G inputs the real-time node embedding vector set learned by the GNN of the cluster cluster (or the aggregated features of the cluster) and aims to generate the complete portrait feature vector of the cluster. The discriminator D inputs the real portrait feature vector (labeled by an expert or generated by historical data statistics) and the portrait feature vector generated by the generator G, and aims to distinguish the true and false portraits. Through adversarial training, the generator G learns to extract the most representative portrait features from the data, and the discriminator D improves the recognition ability of the real portrait. Finally, the generator can output a high-quality, rich, and generalizable cluster portrait, i.e., a cluster portrait.

[0106] The generated real-time cluster portrait is checked for consistency and enhanced in semantics to ensure logical consistency between features. The final real-time cluster portrait (including feature vectors, interpretable labels, and confidence) is stored in association with the corresponding data cluster cluster.

[0107] S4: Based on the real-time media data cluster clusters and their real-time cluster portraits, use the media data analysis engine for multi-dimensional analysis to obtain a real-time media data analysis report.

[0108] The multi-dimensional analysis includes content theme evolution analysis, sentiment distribution analysis, key transmission analysis, user portrait analysis, spatio-temporal distribution analysis, and media content analysis.

[0109] Based on the real-time media data cluster clusters and their real-time cluster portraits, use the media data analysis engine for multi-dimensional analysis to obtain a real-time media data analysis report, including the following steps:

[0110] S4-1: Input the real-time media data cluster clusters and their real-time cluster portraits into the media data analysis engine, and use the feature selection module of the media data analysis engine to select real-time key features of different dimensions.

[0111] The key features of the content theme evolution analysis dimension include the text features of each multi-modal feature vector in the media data clustering cluster and the theme information of the image feature vector in the cluster image. The key features of the sentiment distribution analysis dimension include the text features of each multi-modal feature vector and the sentiment tendency information of the image feature vector in the cluster image. The key features of the key propagation analysis dimension include the propagation features of each multi-modal feature vector and the propagation mode (explosive, continuous, or periodic) of the image feature vector in the cluster image. The key features of the user image analysis dimension include the user features of each multi-modal feature vector and the key figures / institutions, typical user images (age, gender, interest label inference), and association relationships with other clusters of the image feature vector in the cluster image. The key features of the spatio-temporal distribution analysis dimension include the timestamps and spatial positions of each multi-modal feature vector and the spatio-temporal mode of the image feature vector in the cluster image. The key features of the media content analysis dimension include the image features and video features of each multi-modal feature vector.

[0112] S4-2: input the real-time key features of different dimensions into the dimension analysis channels of the multi-dimensional analysis module of the media data analysis engine for analysis to obtain the corresponding real-time dimension analysis results, including the following steps:

[0113] S4-2-1: input the real-time key features of the content theme evolution analysis dimension into the sentiment distribution analysis channel of the multi-dimensional analysis module of the media data analysis engine, use the LDA algorithm to perform theme modeling according to the real-time text features of each real-time multi-modal feature vector, analyze the theme distribution, combine the theme information in the real-time image feature vector in the real-time cluster image, analyze the real-time core theme, use the DTM algorithm to analyze the trend of the real-time core theme over time, capture the real-time theme time sequence features, and combine the real-time core theme to obtain the real-time content theme evolution analysis result;

[0114] S4-2-2: input the real-time key features of the sentiment distribution analysis dimension into the sentiment distribution analysis channel of the multi-dimensional analysis module of the media data analysis engine, use the VADER algorithm to perform sentiment analysis according to the real-time text features of each real-time multi-modal feature vector, analyze the sentiment classification, combine the sentiment tendency information in the real-time image feature vector in the real-time cluster image to obtain the real-time sentiment classification, use the LSTM algorithm to analyze the trend of the real-time sentiment classification over time, and combine the real-time sentiment classification to obtain the real-time sentiment distribution analysis result;

[0115] S4-2-3: input the real-time key features of the key propagation analysis dimension into the key propagation analysis channel of the multi-dimension analysis module of the media data analysis engine, construct a real-time propagation network of users or contents within the cluster according to the real-time propagation features of each real-time multi-modal feature vector and the real-time propagation mode (explosive / continuous / cyclical) of the real-time image feature vector in the real-time cluster image, identify key propagation nodes and paths using the SNA algorithm based on the real-time propagation network, and calculate efficiency indicators (such as average path length, clustering coefficient) of the real-time propagation network to evaluate the propagation effect and obtain real-time key propagation analysis results;

[0116] S4-2-4: input the real-time key features of the user image analysis dimension into the user image analysis channel of the multi-dimension analysis module of the media data analysis engine, identify finer-grained real-time user groups (such as "deeply involved type", "spectator type", "opinion leader type") within the cluster using the GMM algorithm for secondary clustering according to the real-time image feature vector in the real-time cluster image, and output real-time classification labels (such as user type) of the real-time user groups using the SVM to obtain real-time user image analysis results;

[0117] S4-2-5: input the real-time key features of the space-time distribution analysis dimension into the space-time distribution analysis channel of the multi-dimension analysis module of the media data analysis engine, capture the space-time dependence relationship using the STGNN algorithm and perform space-time distribution analysis according to the time stamp, spatial position of each real-time multi-modal feature vector, and the space-time mode of the real-time image feature vector in the real-time cluster image to obtain real-time space-time distribution analysis results;

[0118] S4-2-6: input the real-time key features of the media content analysis dimension into the media content analysis channel of the multi-dimension analysis module of the media data analysis engine, analyze the real-time image features and real-time video features of each real-time multi-modal feature vector using the VGG algorithm to obtain real-time media content analysis results;

[0119] S4-3: integrate all real-time dimension analysis results input by the dimension analysis channels (including real-time content theme evolution analysis results, real-time emotion distribution analysis results, real-time key propagation analysis results, real-time user image analysis results, real-time space-time distribution analysis results, and real-time media content analysis results) to obtain real-time media data analysis results corresponding to the real-time media data clustering clusters and their real-time cluster images;

[0120] S4-4: generate real-time media data analysis reports using the report generation module of the media data analysis engine according to the real-time media data analysis results of all real-time media data clustering clusters.

[0121] Embodiment 2:

[0122] AsFigure 2 The embodiment shown provides an artificial intelligence-based media data analysis system for implementing a media data analysis method.

[0123] The media data crawling engine is used to generate real-time data crawling strategies for different media platforms and crawl real-time multi-modal media data according to the real-time data crawling strategies.

[0124] The media data processing engine is used to process the real-time multi-modal media data to obtain real-time media data clustering clusters and real-time cluster portraits.

[0125] The media data analysis engine is used to perform multi-dimensional analysis according to the real-time media data clustering clusters and real-time cluster portraits to obtain real-time media data analysis reports.

[0126] The artificial intelligence-based media data analysis method and system provided by the embodiment can dynamically generate real-time strategies through an artificial intelligence-driven media data crawling engine, efficiently capture data, and quickly respond to processing and analysis engines, so that the entire analysis process can follow the real-time dynamics of information dissemination, greatly shorten the time from data generation to analysis result output, solve the core problem of analysis lag in the prior art, and instantly capture the rise and evolution of sudden events and hot topics, providing valuable first-time insights for public opinion monitoring, crisis warning, market response, etc. The media data crawling engine can generate strategies for different media platforms, effectively cover mainstream and emerging platforms, avoid information blind spots, make the analysis results more representative, dynamically adjust the crawling strategies according to data feedback and targets, automatically adapt to platform changes and hot topic shifts, improve the efficiency and pertinence of data acquisition, and improve the comprehensiveness and intelligence of data coverage. The media data processing engine can uniformly process various modal data such as text, images, and videos, identify the associations between them, provide a more three-dimensional and complete information view, and effectively extract and analyze the spatiotemporal properties of data to reveal the geographical distribution and time regularity of information dissemination and provide a new dimension for understanding the dissemination mechanism. The feature selection, multi-dimensional analysis, and other modules of the media data processing engine are driven by artificial intelligence, which can automatically identify key information, perform more in-depth and objective analysis, and reduce manual intervention and subjective bias. The media data analysis engine performs media data analysis from multiple dimensions such as content theme evolution analysis, sentiment distribution analysis, key dissemination analysis, user portrait analysis, spatiotemporal distribution analysis, and media content analysis, improves the value of media data analysis results, and obtains a systematic integrated analysis report, improving the convenience and efficiency of media data analysis.

[0127] The application is not limited to the above-mentioned optional embodiments, and anyone can derive other various forms of products under the inspiration of the application. The above-mentioned specific embodiments should not be understood as limiting the protection scope of the application, and the protection scope of the application should be defined by the claims, and the specification can be used to interpret the claims.

Claims

1. An artificial intelligence-based media data analysis method, characterized by: It comprises the following steps: Based on artificial intelligence algorithm, the media data crawling engine, media data processing engine and media data analysis engine are built in the media data analysis platform; The media data crawling engine is provided with a media data crawling interface, a media data crawling executor and a data crawling strategy generator connected in sequence, the media data crawling interface is connected to a plurality of external media platforms, and the data crawling strategy generator is provided with a data crawling strategy generation model; The data crawling strategy generation model is constructed based on the MAPGRPO algorithm, and comprises a platform-level data crawling strategy generation layer, a coordination layer and a server-level data crawling strategy generation layer connected in sequence, the platform-level data crawling strategy generation layer is provided with a first multi-optimization target set and a platform-level agent, the coordination layer is provided with a coordination agent and a multi-objective conflict resolution mechanism, and the server-level data crawling strategy generation layer is provided with a second multi-optimization target set and a plurality of parallel server-level agents; Using the media data crawling engine, real-time data crawling strategies of different media platforms are generated, and a plurality of real-time multi-modal media data are crawled according to the real-time data crawling strategies, comprising the following steps: Using the media data crawling interface of the media data crawling engine, the basic information of different media platforms is collected, and the basic information is input into the data crawling strategy generator; According to the basic information, the data crawling strategy generator of the media data crawling engine is used for strategy generation to obtain real-time data crawling strategies, comprising the following steps: According to the basic information, a platform-level agent of a platform-level data crawling strategy generation layer of a data crawling strategy generation model generation platform generates a strategy in combination with the first multi-optimization objective set, and obtains a platform-level real-time data crawling strategy wherein, is the first platform-level real-time data crawling strategy, is the first, second, …, and nth platform-level real-time data crawling strategy, is a strategy indicator. Based on the multi-objective conflict resolution mechanism, according to the state of the media data analysis platform, the coordination agent of the coordination layer of the data crawling strategy generation model is used to resolve the multi-objective conflict of the platform-level real-time data crawling strategy to obtain the optimized platform-level real-time data crawling strategy, comprising the following steps: Based on the multi-objective conflict resolution mechanism, the NSGA-III algorithm is used to obtain the real-time Pareto frontier solution set from the platform-level real-time data crawling strategy, and the real-time strategy similarity matrix of the real-time Pareto frontier solution set is obtained, comprising the following steps: combining all platform-level real-time data crawling strategies involved in the platform-level real-time data crawling strategy into a strategy pool wherein, is the total number of platform-level real-time data crawling strategies Using NSGA-III algorithm, a Pareto front solution set containing strategies is generated from the strategy pool wherein, is the total number of Pareto front solutions.​ Calculate the similarity between any two strategies in the Pareto front and construct a similarity matrix ; Integrate the media data analysis platform state, real-time Pareto frontier solution set and real-time strategy similarity matrix to obtain a real-time complete state observation that can be recognized by the coordination agent in the coordination layer of the data crawling strategy generation model wherein, is an encoding representation of the strategy in the Pareto frontier set, is a real-time complete state observation, is t the media data analysis platform state at the moment, is a similarity matrix, t is a moment indicator; The real-time complete state observation is mapped to the coordination state space of the coordination agent to obtain the updated coordination state space of the coordination agent; According to the updated coordination state space and the preset coordination action space, the coordination agent is used to resolve the multi-objective conflict to obtain the optimized platform-level real-time data crawling strategy, and the optimized platform-level real-time data crawling strategy is transmitted to the server-level data crawling strategy generation layer of the data crawling strategy generation model; According to the basic information, the state of the media data analysis platform and the optimized platform-level real-time data crawling strategy, the server-level agent of the server-level data crawling strategy generation layer of the data crawling strategy generation model is used for strategy generation to obtain the server-level real-time data crawling strategy of a plurality of media platforms; The optimized platform-level real-time data crawling strategy and the plurality of server-level real-time data crawling strategies are integrated to obtain the real-time data crawling strategy of each media platform; According to the real-time data crawling strategy, the media data crawling executor of the media data crawling engine is used to crawl a plurality of real-time multi-modal media data of the corresponding media platform; Using the media data processing engine, the data processing is performed on the plurality of real-time multi-modal media data to obtain a plurality of real-time media data clustering clusters and real-time cluster portraits; According to the plurality of real-time media data clustering clusters and real-time cluster portraits, the multi-dimensional analysis is performed using the media data analysis engine to obtain a real-time media data analysis report. 2.The media data analysis method based on artificial intelligence according to claim 1, characterized in that: The real-time multi-modal media data includes real-time text data, real-time image data, real-time video data, real-time user information data and real-time interaction data of the media platform. 3.The method of claim 2, wherein: The media data processing engine is provided with a media data preprocessor, a multi-modal feature extractor, a multi-modal feature space-time aligner, a space-time perception incremental clusterer and a cluster portrait generator connected in sequence; The multi-modal feature extractor is provided with a multi-modal feature extraction model, which is constructed based on the BERT-ResNet-CNN-RNN-Wav2Vec2.0-AE algorithm, and the multi-modal feature extraction model includes a text feature extraction module constructed based on the BERT algorithm, an image feature extraction module constructed based on the ResNet algorithm, a video feature extraction module constructed based on the CNN-RNN-Wav2Vec2.0 algorithm, a user feature extraction module constructed based on the AE algorithm and a propagation feature extraction module constructed based on the AE algorithm; The space-time perception incremental clusterer is provided with a space-time perception incremental clustering model, which is constructed based on the STICM algorithm, and the space-time perception incremental clustering model is provided with a feature clustering module constructed based on the ST-DBSCAN algorithm, a new data processing module constructed based on the IP algorithm and a cluster management module constructed based on the DCM algorithm connected in sequence; The cluster cluster portrait generator is provided with a cluster cluster portrait generation model, which is constructed based on the GCN-DualGAN algorithm, and the cluster cluster portrait generation model includes a cluster relationship construction module constructed based on the GCN algorithm and a cluster cluster portrait generation module constructed based on the DualGAN algorithm connected in sequence. 4.The method of claim 3, wherein: The media data analysis engine is provided with a media data analysis model, which includes a feature selection module constructed based on the CA algorithm, a multi-dimensional analysis module constructed based on the MDA algorithm and a report generation module constructed based on the cGAN algorithm connected in sequence, and the multi-dimensional analysis module is provided with a plurality of dimension analysis channels connected in parallel; The dimensional analysis channel includes a content theme evolution analysis channel constructed based on an LDA-DTM algorithm, an emotion distribution analysis channel constructed based on a VADER-LSTM algorithm, a key propagation analysis channel constructed based on an SNA algorithm, a user portrait analysis channel constructed based on a GMM-SVM algorithm, a spatiotemporal distribution analysis channel constructed based on an STGNN algorithm, and a media content analysis channel constructed based on a VGG algorithm.

5. The method of claim 4, wherein the method further comprises: The media data processing engine is used to perform data processing on the real-time multi-modal media data to obtain real-time media data clustering clusters and real-time cluster portraits, including the following steps: The media data preprocessor of the media data processing engine is used to pre-process the real-time multi-modal media data from different media platforms to obtain pre-processed real-time multi-modal media data. The multi-modal feature extractor of the media data processing engine is used to extract real-time multi-modal feature vectors of the pre-processed real-time multi-modal media data. The multi-modal feature spatiotemporal aligner of the media data processing engine is used to perform spatiotemporal alignment on the real-time multi-modal feature vectors to obtain a real-time spatiotemporal alignment matrix including the real-time multi-modal feature vectors. The spatiotemporal perception incremental clusterer of the media data processing engine is used to cluster the real-time multi-modal feature vectors in the real-time spatiotemporal alignment matrix to obtain real-time media data clustering clusters. The clustering cluster portrait generator of the media data processing engine is used to generate real-time cluster portraits corresponding to each real-time media data clustering cluster. 6.The method of claim 5, wherein the method further comprises: The multi-dimensional analysis includes content theme evolution analysis, emotion distribution analysis, key propagation analysis, user portrait analysis, spatiotemporal distribution analysis, and media content analysis. 7.The media data analysis method based on artificial intelligence according to claim 6, characterized in that: According to the real-time media data clustering clusters and the real-time cluster portraits, the media data analysis engine is used to perform multi-dimensional analysis to obtain a real-time media data analysis report, including the following steps: The real-time media data clustering clusters and the real-time cluster portraits are input into the media data analysis engine, and the feature selection module of the media data analysis engine is used to select real-time key features of different dimensions. The real-time key features of different dimensions are input into the dimensional analysis channels corresponding to the multi-dimensional analysis module of the media data analysis engine for analysis to obtain corresponding real-time dimensional analysis results. All real-time dimensional analysis results input into the dimensional analysis channels are integrated to obtain real-time media data analysis results corresponding to the real-time media data clustering clusters and the real-time cluster portraits. According to the real-time media data analysis results of all real-time media data clustering clusters, the report generation module of the media data analysis engine is used to generate a real-time media data analysis report.

8. An artificial intelligence-based media data analysis system for implementing the media data analysis method according to any one of claims 1-7, characterized in that: The system is arranged in a media data analysis platform, and the system includes a media data crawling engine, a media data processing engine, and a media data analysis engine connected in sequence, and the media data crawling engine is connected with external media platforms.

Images