Social media intelligent interaction data processing method and system based on multi-modal analysis

Abstract

The invention discloses a social media intelligent interaction data processing method and system based on multi-modal analysis, and relates to the field of multi-modal analysis, and the method comprises the steps: collecting an original interaction data stream from a target social media platform in real time, the original interactive data stream at least comprises text modal data, image modal data and user behavior sequence modal data; and performing synchronous alignment and cleaning on the original interactive data stream to generate a standardized multi-modal data unit with aligned timestamps. According to the method, through a cross-modal attention fusion mechanism, the defects of shallow multi-modal information fusion and one-sided analysis dimension in the prior art are overcome, unified representation and collaborative understanding of the text, the image and the user behavior sequence on the deep semantic level are realized, and the recognition accuracy of the real intention and the interactive context of the user is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of social media intelligent interaction data processing method and system based on multi-modal analysis. BACKGROUND

[0002] With the rapid development of social media platforms, it has become the core field of information dissemination and interpersonal interaction. The platform generates massive, heterogeneous and dynamically changing interaction data every day, which usually contains multiple modalities such as text, image, video, user click sequence, and attention relationship. How to intelligently process these multi-modal interaction data to accurately understand user intent, emotion and interaction context, and automatically generate appropriate, timely and influential responses or strategies has become a key technical challenge to improve user experience, enhance platform stickiness and achieve precise content operation. This is a frontier topic in the cross-field of artificial intelligence, natural language processing, computer vision and data analysis.

[0003] Currently, social media data processing technology has made some progress. The existing technical solutions mainly include the following categories: first, the analysis method based on single text modality, such as processing comment and post text through sentiment analysis, keyword extraction or topic model; second, the dual-modal analysis method combining text and image, which usually analyzes the two modalities respectively, then performs simple feature splicing or rule-based fusion to judge content consistency or emotional tendency; third, the collaborative filtering or sequence prediction method based on user historical behavior, which is used to recommend content or predict user click behavior. In addition, some systems also try to automatically respond to interactions, but mostly rely on pre-set templates or only in single text dialogue scenarios.

[0004] However, the above existing technologies have obvious limitations and shortcomings. First, the multi-modal fusion is shallow, most methods only perform late splicing or simple association, and fail to achieve cross-modal collaborative understanding at the deep semantic level, resulting in biased interpretation of complex interactions (such as irony with images, behavior sequence implied intent). Second, the analysis dimension is one-sided, either focusing on content itself or on user behavior, lacking a mechanism for integrating content semantics, visual information and dynamic behavior sequences in a unified time dimension, resulting in incomplete and inaccurate user intent recognition. Third, the interaction strategy generation and prediction ability is insufficient, existing methods are mostly limited to immediate response or static recommendation, lacking the ability to dynamically predict interaction trends, prospectively identify high-potential interaction nodes, and generate dynamic and strategic interaction strategies based on multi-modal deep analysis, resulting in low interaction efficiency and limited influence. Therefore, there is an urgent need for an innovative method and system that can deeply integrate and understand multi-modal data, achieve intelligent analysis and forward-looking strategy generation. SUMMARY

[0005] Based on this, the purpose of the present application is to provide a social media intelligent interaction data processing method and system based on multi-modal analysis, to solve the technical problems of the prior art that the multi-modal information fusion is shallow, the user intention understanding is incomplete, and the intelligent interaction strategy generation and prediction ability based on deep analysis is lacking.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a social media intelligent interaction data processing method and system based on multi-modal analysis, comprising: collecting real-time original interaction data stream from a target social media platform, the original interaction data stream at least containing text modal data, image modal data and user behavior sequence modal data; synchronously aligning and cleaning the original interaction data stream to generate timestamp-aligned standardized multi-modal data units; inputting the standardized multi-modal data units into pre-trained text feature extraction model, image feature extraction model and behavior sequence analysis model respectively, and extracting text deep semantic feature vector, image content understanding feature vector and user behavior intention feature vector; inputting the text deep semantic feature vector, image content understanding feature vector and user behavior intention feature vector into a cross-modal attention fusion module, and generating a unified multi-modal representation vector after fusion through a learnable weight distribution mechanism; inputting the unified multi-modal representation vector into an intelligent interaction analysis model, the intelligent interaction analysis model at least performing one of the following analysis tasks: generating a targeted interaction response content, predicting an interaction trend evolution path, and identifying a potential high-influence interaction node; generating an interaction strategy control instruction according to the output result of the intelligent interaction analysis model, and feeding back the instruction to the interaction interface of the target social media platform to perform automatic or assisted intelligent interaction operation.

[0007] The present application is further provided that the original interaction data stream is synchronously aligned and cleaned, comprising: establishing a cross-modal time synchronization axis based on the time metadata in each modal data in the original interaction data stream or through event trigger marking; according to a preset data quality threshold, eliminating irrelevant advertising text in the text modal data, low-resolution images in the image modal data and abnormal click stream data in the user behavior sequence modal data; cutting the cleaned modal data into equal-length or indefinite-length standardized multi-modal data units according to the cross-modal time synchronization axis.

[0008] The present application is further provided that the behavior sequence analysis model adopts a hybrid neural network architecture based on time convolution network (TCN) and self-attention mechanism, for extracting deep intention features evolving over time from user behavior sequence modal data.

[0009] The application is further configured that the working process of the cross-modal attention fusion module includes: respectively calculating cross-attention weights of the text deep semantic feature vector and the image content understanding feature vector relative to the user behavior intention feature vector; according to the cross-attention weights, performing dynamic weighted summation on the text deep semantic feature vector and the image content understanding feature vector to generate a preliminary fusion vector; splicing the preliminary fusion vector with the user behavior intention feature vector, and performing dimension reduction and nonlinear transformation through a fully connected layer to generate the unified multi-modal representation vector.

[0010] The application is further configured that the intelligent interaction analysis model includes a parallel interaction content generation subnetwork and an interaction influence prediction subnetwork; the interaction content generation subnetwork is a sequence generation model based on a Transformer architecture, used for generating text or multimedia response content conforming to context semantics according to the unified multi-modal representation vector; the interaction influence prediction subnetwork is a prediction model based on a graph neural network (GNN), used for predicting an interaction propagation range and a key propagation path in a future time window according to the unified multi-modal representation vector and a historical interaction relationship graph.

[0011] The application is further configured that the method further includes: constructing or updating a dynamic interaction relationship graph based on an output of the interaction influence prediction subnetwork; wherein nodes in the dynamic interaction relationship graph represent users or content themes, and weights of edges are dynamically adjusted according to interaction frequencies, emotional tendencies and multi-modal feature similarities.

[0012] The application is further configured that the system includes: a multi-modal data acquisition module configured to acquire a raw interaction data stream from a target social media platform in real time; a data preprocessing and alignment module configured to clean, standardize and time-synchronize and align the raw interaction data stream to generate a standardized multi-modal data unit; a multi-modal feature extraction module configured to extract deep feature vectors of text, image and behavior sequences from the standardized multi-modal data unit; a cross-modal fusion module configured to fuse deep feature vectors of different modalities into a unified multi-modal representation vector through an attention mechanism; an intelligent interaction analysis engine configured to perform an intelligent analysis task based on the unified multi-modal representation vector and output an analysis result; and an interaction strategy execution and feedback module configured to generate a control instruction according to the analysis result and feed back to a social media platform interface to perform an intelligent interaction operation.

[0013] The application is further configured that the multi-modal feature extraction module comprises a text feature extraction submodule, which is internally provided with a pre-training model based on BERT or similar architecture; an image feature extraction submodule, which is internally provided with a pre-training model based on ResNet, VisionTransformer or similar architecture; and a behavior sequence feature extraction submodule, which is internally provided with a hybrid neural network model based on a time convolution network (TCN) and a self-attention mechanism.

[0014] The application is further configured that the system further comprises a knowledge graph management module, which is configured to store and manage historical analysis results output by the intelligent interactive analysis engine, construct and maintain a field knowledge graph and a user portrait graph based on the historical analysis results, and provide a graph-based context prior knowledge query service for the intelligent interactive analysis engine.

[0015] To sum up, the application mainly has the following beneficial effects: The application realizes accurate understanding and forward-looking strategy generation for complex interactive scenarios of social media by constructing a deep collaborative multi-modal analysis framework, and has the following beneficial effects: through a cross-modal attention fusion mechanism, the application overcomes the defects of shallow multi-modal information fusion and one-dimensional analysis in the prior art, realizes unified representation and collaborative understanding of text, image and user behavior sequence at a deep semantic level, and significantly improves the recognition accuracy of user real intention and interactive context; further, the intelligent analysis model based on the unified representation can perform all-round tasks from accurate content generation to dynamic trend prediction, systematically improves the pertinence, timeliness and influence of interactive response, and finally realizes the leap from passive response to active and intelligent strategic interactive management, greatly enhancing the automation level and efficiency of social media operation. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The control flowchart of the application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. The embodiments described below with reference to the drawings are exemplary and are used only to explain the application, and cannot be understood as a limitation of the application.

[0018] The embodiments of the application will be described below according to the overall structure of the application.

[0019] A social media intelligent interactive data processing method and system based on multi-modal analysis, as shown in Figure 1 The method comprises the following steps: Embodiment one.

[0020] The multi-modal analysis-based social media intelligent interaction data processing method of the embodiment includes the following steps S1 to S6.

[0021] S1: Multi-modal raw data acquisition.

[0022] Real-time or quasi-real-time acquisition of raw interaction data streams from the open interface or authorized data channel of the target social media platform (such as Weibo, TikTok, Twitter, Facebook, etc.). The raw interaction data stream is a time-ordered collection of data related to a specific user, topic or event, which at least contains the following three modal core data: Text modal data: including user-posted post text, comments, replies, private message content, topic tags, and associated metadata (such as publisher ID, posting time, etc.).

[0023] Image modal data: including user-uploaded static pictures, dynamic pictures, video cover frames, and descriptive text accompanying pictures / videos (which can be processed as associated text).

[0024] User behavior sequence modal data: This is a series of user interaction behavior records sorted by timestamp, including at least but not limited to: "like", "forward", "collect" actions and time of post, "reply" behavior of comment, user page browsing path (click stream), follow / unfollow events, and implicit feedback data such as dwell time.

[0025] The acquisition process is implemented by calling platform API, using a compliant web crawler framework, or receiving platform-pushed data streams to ensure the timeliness and completeness of the data.

[0026] S2: Data synchronization alignment and cleaning preprocessing.

[0027] This step aims to transform heterogeneous, heterogeneous, and non-strictly synchronized raw data into clean, aligned, and standard data units that can be used for in-depth analysis. Specifically, the following sub-steps are included: S21: Time synchronization axis establishment. Since the time points of each modal data may not be completely consistent, a unified cross-modal time synchronization axis needs to be established. One implementation is to use the timestamp of text publishing or a user's explicit behavior event (such as "posting a post") as the reference time point T0. For image data, the timestamp is taken at the time of image upload or the publishing time of associated text. For user behavior sequences, each behavior has a precise timestamp to the millisecond. Map all modal data points to the time axis with T0 as the origin. For images without precise timestamps, they can be assigned the same timestamp as the associated text according to their association with the text.

[0028] S22: Data cleaning and denoising. According to the preset quality threshold and rule base, low-quality and noise data are removed. Specifically, it includes: For text modal data: using regular expressions and keyword filters, remove obvious advertising text, meaningless random code, repeated screen content, and system-generated fixed format text.

[0029] For image modal data: calculate the resolution, sharpness (such as through Laplacian variance) and color distribution of the image. Remove images with resolution below a certain threshold (such as 100x100 pixels), too low sharpness or dominated by solid color / blank.

[0030] For user behavior sequence modal data: identify and remove abnormal click streams generated by robots or scripts (such as consecutive same operations within milliseconds), and stay time data that is obviously beyond the normal range.

[0031] S23: Standardized multi-modal data unit generation. The cleaned data is cut and packaged according to the time synchronization axis. One implementation is to cut it with a fixed time window (such as 10 minutes before and 2 hours after a core post) to generate a data unit. Another way is to generate indefinite length data units with specific interaction events (such as the complete life cycle of a hot topic) as boundaries. Each standardized multi-modal data unit contains a text data block, one or more image data blocks, and a continuous user behavior sequence associated in time.

[0032] S3: Deep feature vector extraction.

[0033] Different modal data in the standardized multi-modal data unit is input into a specially optimized deep neural network model to extract high-level abstract feature vectors.

[0034] Text deep semantic feature vector extraction: input the text data block into the pre-trained text feature extraction model. Preferably, a pre-trained language model based on the Transformer architecture is used, such as BERT, RoBERTa or ERNIE. In specific implementation, after tokenization and encoding of the input text, the output vector corresponding to the [CLS] tag in the last layer of the model is taken, or the output of all tokens in the sequence is averaged and pooled to obtain a fixed dimension (such as 768 dimensions) text deep semantic feature vector V_text. The vector contains deep semantic information such as sentiment, theme, and opinion of the text.

[0035] Image content understanding feature vector extraction: input the image data block into a pre-trained computer vision model. Optionally use a deep convolutional neural network (such as ResNet-50, ResNet-101) or a visual Transformer model (such as ViT). After pre-processing the image, input it into the model, take the features after the global average pooling layer or the features before the classification layer, and obtain an image content understanding feature vector with a fixed dimension (such as 2048 dimensions or 512 dimensions) . This vector encodes visual semantic information such as scene, object, facial expression, and color composition of the image.

[0036] User behavior intention feature vector extraction: input the user behavior sequence (encoded as a series of discrete behavior types and time intervals) into a specially designed behavior sequence analysis model. The model preferably adopts a hybrid architecture combining a temporal convolutional network and a self-attention mechanism. The temporal convolutional network can effectively capture long-term dependencies in the behavior sequence, while the self-attention mechanism can dynamically measure the importance of different historical behaviors to the current intention. Specifically, first embed the behavior types into dense vectors and concatenate them with normalized time interval features, then input them into multiple temporal convolutional blocks. Then input the output sequence of TCN into the self-attention layer, and finally obtain the user behavior intention feature vector through pooling. This vector reflects the user's interest preferences, interaction habits, and real-time intention tendencies.

[0037] S4: Cross-modal attention fusion.

[0038] This step is the core innovation, aiming to realize deep semantic fusion. Through a trainable cross-modal attention fusion module, dynamically integrate , , .

[0039] S41: Cross-attention weight calculation. Take the user behavior intention feature vector as the "query" (Query), and take the text feature vector and the image feature vector as the "key" and "value" respectively. Calculate the attention scores of and , and . Specifically, it can be realized through scaled dot-product attention: . Here, is the linear projection of , and are linear projections of or . Thus, the text attention weight and the behavior-image attention weight .

[0040] S42: Dynamic weighting and preliminary fusion. The computed attention weights are used to weight the original and features: . The weighted feature vectors are then added or concatenated to form a preliminary fusion vector . This process makes the fusion more focused on the text and visual information most relevant to the current user behavior intent.

[0041] S43: Unified representation generation. The preliminary fusion vector V_preliminary is concatenated with the core user behavior intent feature vector to form a wider feature vector. This concatenated vector is then passed through a fully connected layer (which can include an activation function such as ReLU) for dimension reduction and non-linear transformation, ultimately outputting a unified multi-modal representation vector . This vector compactly integrates core complementary information from the three modalities.

[0042] S5: Intelligent interaction analysis.

[0043] is input into the intelligent interaction analysis model to perform advanced analysis tasks. This model can include multiple parallel or switchable subnetworks.

[0044] Interaction content generation subnetwork: for generating targeted replies or new content. This subnetwork is preferably based on a Transformer-based encoder-decoder architecture (such as the GPT series, T5) or a conditional generative adversarial network. Taking as the conditional input or the initial hidden state of the decoder, the model can automatically generate text replies, comments, or post drafts that are consistent with the current multi-modal context (such as text sentiment, image content, user interest) and natural and fluent. For example, given a food image and the text "I want to eat", combined with the user's historical preference for desserts, a recommendation reply such as "This newly opened dessert shop's signature mango pavlova looks even better!" can be generated.

[0045] Interaction influence prediction subnetwork: for predicting trends and identifying key nodes. This subnetwork is preferably based on a graph neural network. In addition to , it also incorporates historical interaction relationship graphs (nodes in the graph are users and topics, and edges are interaction relationships). GNN can predict the propagation range (such as the number of forwards and reading volume prediction) and emotional evolution trend of the current interaction data unit in the future by performing message propagation on the graph, and identify the "high-impact nodes" (such as key opinion leaders or active communities) that are most likely to drive the propagation. The prediction results can be quantified as probabilities or influence scores. ​

[0046] S6: Strategy execution and feedback.

[0047] According to the analysis result of S5, a specific interactive strategy control instruction is generated. For example: If the analysis result is to generate a reply, the instruction includes the generated text content and the target reply object.

[0048] If a high-influence node is identified, the instruction can include "preferentially interacting with the user", "pushing relevant high-quality content to the node", etc.

[0049] If it is predicted that negative emotions will spread, the instruction can include "starting a crisis public relations template", "notifying human customer service to intervene".

[0050] These instructions are encapsulated as calling requests conforming to the API specification of the target social media platform, and are sent through the interaction interface of the platform, so as to automatically perform operations such as liking, replying, publishing, and following, or to provide an auxiliary decision-making panel and operation suggestions for the operator.

[0051] Embodiment two.

[0052] Multimodal data acquisition module: This module is responsible for communication with external social media platforms, and integrates API calling clients, stream data receivers, and data cache queues. It is configured with adapters for different platforms, and can continuously capture or receive text, image, and behavior stream data according to preset acquisition rules (such as keywords, user lists, topic IDs).

[0053] Data preprocessing and alignment module: This module includes a time synchronizer, a data cleaner, and a data unit assembler. The time synchronizer establishes a time axis according to an algorithm (such as dynamic time warping or event-based alignment). The data cleaner has a built-in rule engine and a lightweight quality evaluation model. The data unit assembler packages the processed data into a standardized format (such as JSON sequence) and passes it to the downstream.

[0054] Multimodal feature extraction module: This module is a computationally intensive component, containing three independent feature extraction submodules, which can be deployed on GPU servers.

[0055] Text feature extraction submodule: Loads a pre-trained BERT model and its tokenizer, and vectorizes the input text.

[0056] Image feature extraction submodule: Loads a pre-trained ResNet model, and performs forward propagation to extract features from the input image.

[0057] Behavior sequence feature extraction submodule: Loads the TCN-self-attention hybrid model described in the present application, which needs to be pre-trained using historical behavior data to learn the mapping of behavior patterns to intent vectors.

[0058] Cross-modal fusion module: This module is implemented as a trainable neural network layer (i.e., the aforementioned cross-modal attention fusion module). After system initialization, the parameters of this module need to be trained end-to-end or in stages with labeled multi-modal interaction data (labels such as "intent category", "sentiment polarity") to learn the optimal fusion weight allocation strategy.

[0059] Intelligent interaction analysis engine: This is the "brain" of the system, containing at least two configurable analyzers.

[0060] Interaction content generator: In essence, it is a conditional text generation model that generates text based on V_unified.

[0061] Interaction impact predictor: In essence, it is a graph neural network prediction model that needs to obtain the latest interaction relationship graph from the knowledge graph management module (see below) as input in real time or periodically.

[0062] The analysis engine calls different analyzers or performs combined analysis according to business needs.

[0063] Interaction strategy execution and feedback module: This module includes a strategy decision maker, an instruction wrapper, and an API call agent. The strategy decision maker decides the type and parameters of the final strategy to be executed based on the structured results output by the analysis engine (such as generated text, predicted scores, node list), combined with pre-set business rules (such as brand safety rules, interaction frequency limits). The instruction wrapper converts it into specific platform API call instructions. The API call agent is responsible for identity authentication, request sending and response processing, and completes the closed-loop interaction.

[0064] Optionally, knowledge graph management module: This module serves as the memory and knowledge base of the system, independent or integrated into the system. It continuously extracts entities (users, topics, keywords) and relationships (interactions, similarities, inclusions) from pre-processed data and analysis results to build and update domain knowledge graphs and dynamic user portrait graphs. This module provides graph query services for the intelligent interaction analysis engine (especially the GNN predictor) and provides historical context and knowledge support for strategy decision-making.

[0065] Embodiment three.

[0066] This embodiment also provides a computer-readable storage medium, such as a U disk, a hard disk, an optical disk, a server storage space, etc., which stores a computer program. When the program is executed by one or more processors (such as a server CPU), it can drive a computer system or a distributed computing cluster to completely implement all steps of the multi-modal analysis-based social media intelligent interaction data processing method described in embodiment one.

[0067] The embodiment of the present application discloses a social media intelligent data processing scheme deeply fusing text, image and behavior sequence, which realizes deep semantic understanding and forward-looking intelligent strategy generation for complex interactive scenarios through constructing a complete technical chain from real-time collection of multi-source heterogeneous data, time alignment and cleaning, deep feature extraction based on pre-training model and special hybrid neural network (TCN-self-attention), to cross-modal attention dynamic fusion guided by user behavior intention, and content generation and graph neural network trend prediction based on unified representation, and supporting modular system architecture and knowledge graph, and finally completes systematic improvement from passive response to active strategic interaction.

[0068] The steps of methods or algorithms described in the present application can be directly embedded in hardware, software units executed by a processor, or a combination of the two. Illustratively, a storage medium can be connected to a processor so that the processor can read information from the storage medium and can store information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. Alternatively, the processor and the storage medium can also be arranged in different components in the terminal. These computer program instructions can also be loaded onto a computer or other programmable data processing device to make the computer or other programmable data processing device execute a series of operation steps to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the flow Figure 1 The steps of the functions specified in one flow or multiple flows and / or blocks Figure 1 The steps of the functions specified in one flow or multiple flows and / or blocks

[0069] Although the present application is described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of the application. Accordingly, the present specification and drawings are merely exemplary and illustrative of the present application, and it is considered that any and all modifications, variations, combinations or equivalents that fall within the scope of the present application have been covered. Obviously, those skilled in the art can make various modifications and changes to the present application without departing from the scope of the present application. Thus, if these modifications and changes of the present application belong to the scope of the present application and its equivalents, the present application is intended to include these modifications and changes.

Claims

1. A method and system for intelligent interactive data processing on social media based on multimodal analysis, characterized in that, include: S1: Collect raw interaction data streams in real time from the target social media platform. The raw interaction data streams include at least text modal data, image modal data, and user behavior sequence modal data. S2: Synchronize, align, and clean the original interactive data stream to generate timestamp-aligned standardized multimodal data units; S3: Input the standardized multimodal data units into the pre-trained text feature extraction model, image feature extraction model and behavior sequence analysis model respectively to extract text deep semantic feature vectors, image content understanding feature vectors and user behavior intent feature vectors; S4: Input the text deep semantic feature vector, image content understanding feature vector and user behavior intent feature vector into the cross-modal attention fusion module, and generate a fused unified multimodal representation vector through a learnable weight allocation mechanism; S5: Input the unified multimodal representation vector into the intelligent interaction analysis model, wherein the intelligent interaction analysis model performs at least one of the following analysis tasks: generating targeted interactive response content, predicting the evolution path of interactive trends, and identifying potential high-influence interactive nodes; S6: Based on the output of the intelligent interaction analysis model, generate interaction strategy control instructions and feed the instructions back to the interaction interface of the target social media platform to execute automated or assisted intelligent interaction operations.

2. The social media intelligent interaction data processing method and system based on multimodal analysis according to claim 1, characterized in that: The process of synchronizing, aligning, and cleaning the original interactive data stream includes: S21: Establish a cross-modal time synchronization axis based on the time metadata of each modal data in the original interactive data stream or through event-triggered markers; S22: Based on a preset data quality threshold, remove irrelevant advertising text from the text modal data, low-resolution images from the image modal data, and abnormal clickstream data from the user behavior sequence modal data; S23: The cleaned modal data is cut into standardized multimodal data units of equal or variable length according to the cross-modal time synchronization axis.

3. The social media intelligent interaction data processing method and system based on multimodal analysis according to claim 1, characterized in that: The behavior sequence analysis model employs a hybrid neural network architecture combining temporal convolutional networks and self-attention mechanisms to extract deep intent features that evolve over time from user behavior sequence modality data.

4. The social media intelligent interaction data processing method and system based on multimodal analysis according to claim 1, characterized in that: The operation of the cross-modal attention fusion module includes: S41: Calculate the cross-attention weights of the text deep semantic feature vector and the image content understanding feature vector relative to the user behavior intent feature vector, respectively; S42: Based on the cross-attention weights, dynamically weight and sum the text deep semantic feature vector and the image content understanding feature vector to generate a preliminary fusion vector; S43: The preliminary fusion vector is concatenated with the user behavior intent feature vector, and then dimensionality reduction and nonlinear transformation are performed through a fully connected layer to generate the unified multimodal representation vector.

5. The social media intelligent interaction data processing method and system based on multimodal analysis according to claim 1, characterized in that: The intelligent interaction analysis model includes a parallel interactive content generation subnetwork and an interactive impact prediction subnetwork; The interactive content generation subnetwork is a sequence generation model based on the Transformer architecture, used to generate text or multimedia response content that conforms to contextual semantics based on the unified multimodal representation vector. The interaction impact prediction subnetwork is a prediction model based on graph neural networks, used to predict the scope of interaction propagation and key propagation paths within a future time window based on the unified multimodal representation vector and historical interaction relationship graph.

6. The social media intelligent interaction data processing method and system based on multimodal analysis according to claim 5, characterized in that: The method further includes step S7: constructing or updating a dynamic interaction relationship graph based on the output of the interaction influence prediction sub-network; wherein, the nodes in the dynamic interaction relationship graph represent users or content topics, and the weights of the edges are dynamically adjusted according to the interaction frequency, sentiment tendency, and multimodal feature similarity.

7. The social media intelligent interaction data processing method and system based on multimodal analysis according to claim 1, characterized in that: The system for implementing the method as described in any one of claims 1 to 6 comprises: A multimodal data acquisition module, configured to acquire raw interaction data streams from target social media platforms in real time; The data preprocessing and alignment module is configured to clean, standardize, and time-synchronize the raw interactive data stream, generating standardized multimodal data units. A multimodal feature extraction module is configured to extract deep feature vectors of text, images, and behavioral sequences from the standardized multimodal data units; The cross-modal fusion module is configured to fuse deep feature vectors from different modalities into a unified multimodal representation vector through an attention mechanism. An intelligent interactive analysis engine is configured to perform intelligent analysis tasks based on the unified multimodal representation vector and output analysis results; The interactive strategy execution and feedback module is configured to generate control commands based on the analysis results and feed them back to the social media platform interface to execute intelligent interactive operations.

8. The social media intelligent interaction data processing method and system based on multimodal analysis according to claim 7, characterized in that, The multimodal feature extraction module includes: The text feature extraction submodule has a built-in pre-trained model based on BERT or a similar architecture; The image feature extraction submodule has a built-in pre-trained model based on ResNet, Vision Transformer or similar architecture. The behavioral sequence feature extraction submodule has a built-in hybrid neural network model based on temporal convolutional networks and self-attention mechanisms.

9. The social media intelligent interaction data processing method and system based on multimodal analysis according to claim 7, characterized in that: The system also includes a knowledge graph management module, which is configured to: Store and manage historical analysis results output by the intelligent interactive analysis engine; Construct and maintain domain knowledge graphs and user profile graphs based on historical analysis results; Provide a graph-based contextual prior knowledge query service for the intelligent interactive analysis engine.

Images