Multimodal Content Source Tracing Analysis Method and System for Social Media Propagation Links

Abstract

This application discloses a method and system for multimodal content source tracing analysis of social media propagation chains, including: collecting and preprocessing multimodal propagation data; encoding visual and semantic modal data through a deep feature extraction network and projecting it into a cross-modal alignment space to generate a fused semantic representation vector; calculating the weighted cosine distance through a modality adaptive attention mechanism to obtain the semantic fidelity attenuation, and combining it with platform encoding degradation estimation to obtain the net semantic attenuation; constructing a propagation dynamic graph based on propagation metadata with the net semantic attenuation as edge weights; aggregating propagation features through an edge-weight-aware graph convolutional network; tracing and locating the source node by performing reverse semantic gradient tracing through a source likelihood scoring network; and achieving specialized source tracing of adversarial mutation propagation through semantically invariant region detection and deep semantic structure correlation mining. This invention achieves deep coupling source tracing of content-level semantic information and propagation topology information.

Description

Technical Field

[0001] This application relates to the fields of multimodal data processing, graph data analysis, and graph neural network technology, specifically to a method and system for multimodal content tracing and analysis of social media dissemination links. Background Technology

[0002] With the widespread adoption of social media platforms, the generation and dissemination of multimodal information, including images, videos, and text, have exploded. When the same original content is disseminated across different social media platforms, it inevitably undergoes a series of irreversible information transformation operations, such as screenshot compression, secondary editing and cropping, text rewriting and summarizing, watermarking, video clip splicing, and translation. These operations cause the multimodal content carried by each node in the downstream dissemination chain to suffer pixel-level degradation and compositional distortion at the visual level, and topic drift and loss of key information at the semantic level, compared to the original content. As the number of dissemination hops increases, the multimodal semantic fidelity of the content decays non-linearly and progressively. The greater the semantic deviation between the content held by the dissemination node farther from the source node and the original content, the greater this decay is. Furthermore, this decay is intricately coupled with the type of information transformation along the dissemination path, the encoding and compression strategies of the platforms involved, and the editing habits of the disseminating users.

[0003] Current social media content attribution technologies primarily follow two independent paths. The first path is passive forensic techniques based on digital watermarking or hash fingerprinting, which traces the source by embedding or extracting digital identifiers from content. However, the applicability of this type of technology is strictly limited to scenarios involving precise matching of unmodified content. Once content undergoes multiple rounds of editing and variation, it loses its matching ability, making it difficult to provide effective attribution support when facing the prevalent content variations in social media dissemination. The second path is propagation topology analysis techniques based on graph neural networks, which mainly focus on the structural features of the propagation graph, including cascade scale, propagation speed, and user social relationship graphs, for the detection of misinformation or prediction of propagation trends. However, while this type of technology can model the propagation topology, it fails to incorporate the multimodal semantic evolution of the content itself during propagation into the modeling scope of graph node features, resulting in insufficient accuracy in content-level attribution. From a data processing technology perspective, existing solutions have failed to establish a data processing framework that jointly models the multimodal semantic features of content as attributes of graph data structure nodes.

[0004] Furthermore, while current cross-modal representation alignment techniques can solve the static matching problem between visual and linguistic modalities, these solutions struggle to capture the dynamic decay of semantic fidelity with the number of propagation hops in dynamic propagation scenarios. Existing technologies lack a scheme for quantifying semantic fidelity as edge weights in the propagation graph to drive source tracing inference. This results in insufficient utilization of the coupling between content-level semantic information and topological structural information. Moreover, existing technologies lack the technical means to leverage the decreasing semantic fidelity along the propagation direction for source-level reverse inference and localization. From a data processing perspective, existing graph data processing methods lack the technical means to quantify and encode semantic distance metrics in the multimodal feature space as edge attributes in directed graph edge weight modeling, and also lack data processing methods for directed graph reverse path search based on edge weight gradients.

[0005] Furthermore, existing source tracing techniques based on graph neural networks rely on pre-constructing a complete propagation topology graph. When the propagation scale is large or the propagation relationships are incomplete, the candidate link selection process during graph construction may introduce erroneous edges, causing the accuracy of source tracing to be constrained by the quality of graph construction. Moreover, these techniques lack the ability to quantify the uncertainty of source tracing results. From a data processing technology perspective, the pre-construction requirement of the graph data in the above-mentioned solutions limits their applicability to large-scale, incomplete data scenarios, and they lack technical means to probabilistically assess the uncertainty of data processing results.

[0006] Furthermore, in the actual dissemination of malicious infringing content or disinformation, a special and frequently occurring variant propagation scenario exists: propagators intentionally circumvent conventional content matching detection by making multimodal, localized modifications to the original content. In this scenario, the content at each hop in the propagation chain differs significantly in surface features, but maintains a traceable decaying correlation in its deep semantic structure. Currently, no existing technology offers a systematic source tracing solution for this type of adversarial variant propagation scenario, leaving the problem of tracing the source of malicious variant content unresolved.

[0007] Meanwhile, in cross-platform dissemination scenarios, when the same content spreads from one social media platform to another, it undergoes encoding compression on the source platform and transcoding and recompression on the target platform. The semantic loss introduced by this dual encoding operation overlaps with the semantic loss introduced by the disseminator's editing behavior. Existing source tracing technologies fail to differentiate between platform-inherent encoding degradation and user-initiated editing variations, resulting in severe interference with the source tracing accuracy of cross-platform dissemination links. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a multimodal content source tracing analysis method for social media propagation chains. This method constructs a multimodal semantic fidelity decay quantification model, encodes the semantic evolution of content during propagation as edge weight features of a propagation dynamic graph, and utilizes a reverse semantic gradient tracing mechanism to automatically locate the original propagation source node and reconstruct the complete diffusion trajectory from the propagation graph. The method includes the following steps: Step S1 involves performing standardized preprocessing on the acquired multimodal propagation data. This multimodal propagation data includes visual modal data, semantic modal data, and propagation metadata for each data item. The propagation metadata includes a publication timestamp and forwarding / referencing relationships. The standardized preprocessing includes size normalization and keyframe extraction for the visual modal data, and word segmentation and noise reduction for the semantic modal data. Specifically, the multimodal propagation data is collected from a group of target social media platforms. The visual modal data includes image and video data, and the semantic modal data includes text data associated with the visual modal data. The propagation metadata also includes publishing platform identifiers and publishing user identifiers. In one embodiment, the keyframe extraction employs an adaptive keyframe extraction strategy, selecting semantically representative frames based on abrupt changes in inter-frame difference values ​​to reduce the computational burden of redundant frames.

[0009] Step S2 involves using a deep feature extraction network to extract visual semantic feature vectors from the visual modality data and semantic feature vectors from the semantic modality data. These visual semantic feature vectors and semantic feature vectors are then projected into a cross-modal semantic alignment space to generate a multimodal fused semantic representation vector (hereinafter referred to as the fused semantic representation vector). Further, the cross-modal semantic alignment space is trained using a contrastive learning objective function, ensuring that the representations of multimodal content originating from the same source but undergoing different transformations remain clustered, while the representations of non-originating content remain dispersed. The contrastive learning training process incorporates data augmentation strategies such as simulated screenshot compression, text rewriting, and image cropping. In one embodiment, the data augmentation strategies further include overlaying text watermarks with random positions and transparency onto the image to simulate watermark overlay, and performing random truncation of the text to simulate platform character limits, thereby enhancing the robustness of the fused semantic representation to propagation variations.

[0010] Step S3 involves calculating the weighted cosine distance between the multimodal fusion semantic representation vectors of two propagation nodes to obtain the semantic fidelity attenuation. The weights of the weighted cosine distance are dynamically adjusted using a modality-adaptive attention mechanism. This mechanism is implemented through a lightweight attention network, which takes the concatenation of the visual modality feature difference vector and the semantic modality feature difference vector of the two nodes as input, passes through two fully connected layers and a Softmax normalization layer, and outputs visual modality weight coefficients and semantic modality weight coefficients. The calculation of the semantic fidelity attenuation also introduces an information transformation type perception factor. An information transformation type classifier identifies the type of information transformation experienced by the content between adjacent propagation nodes, and the semantic fidelity attenuation is calibrated based on this information transformation type. In one embodiment, this calibration makes the attenuation caused by the same type of transformation comparable across different content pairs, thereby improving the accuracy of subsequent reverse tracing.

[0011] Furthermore, step S3 introduces a platform encoding degradation estimation submodule. This submodule pre-trains a platform encoding degradation model for each target social media platform. The platform encoding degradation model outputs an estimated value of the expected semantic loss after platform encoding degradation. When calculating the semantic fidelity decay, the estimated expected semantic loss is subtracted to obtain the net semantic decay, which serves as the edge weight of the propagation dynamic graph. In one implementation, the platform encoding degradation model uses the fused semantic representation vector of the original high-fidelity multimodal content as input to simulate the platform's unique encoding compression process. This allows the reverse gradient tracing mechanism to filter out inherent platform encoding noise and focus on semantic changes caused by intentional manipulation by the propagator, thereby improving the accuracy of cross-platform source tracing.

[0012] Step S4: Determine the propagation direction and construct a propagation dynamic graph based on the forwarding and referencing relationships in the propagation metadata, and assign edge weights calculated based on the semantic fidelity attenuation. In the propagation dynamic graph, each node represents a propagation event, i.e., a multimodal content posted or forwarded by a user on a platform at a certain time; the node feature is the multimodal fusion semantic representation vector of the content corresponding to the propagation event. The directed edges between nodes represent the propagation direction of the content, i.e., from the referenced content to the referencing content. Further, for implicit propagation node pairs whose explicit referencing relationships cannot be determined through the propagation metadata, a candidate link filtering strategy is used to supplement the edges. The candidate link filtering strategy includes: filtering node pairs with semantic similarity exceeding a preset threshold based on the approximate nearest neighbor retrieval of the multimodal fusion semantic representation vector, filtering unreasonable candidate links in combination with time series constraints, calculating the semantic fidelity attenuation of the filtered candidate links and assigning edge weights. In one embodiment, the preset threshold is adaptively adjusted according to the statistical distribution characteristics of the known propagation links.

[0013] Step S5 involves aggregating the neighborhood propagation features of each node using a graph convolutional network. This graph convolutional network introduces an edge-weight-aware attention aggregation mechanism, where neighboring edges with lower semantic fidelity decay receive higher attention weights. The inter-layer connections of the graph convolutional network employ a residual skip connection structure, and a gated feature fusion operation is performed between the node features and the original multimodal fused semantic representation vector after each convolutional layer. In one implementation, the gated feature fusion operation dynamically controls the fusion ratio between the graph convolutional output features and the original fused semantic representation vector through a gating coefficient, thereby maintaining the semantic anchoring of the node's content.

[0014] Step S6: Iterate along the directed edge in the opposite direction from the target node, and calculate the source likelihood score of the incoming edge neighbor nodes through the source likelihood scoring network. The source likelihood score integrates three factors: semantic fidelity decay, propagation timestamp order, and Mahalanobis distance. The Mahalanobis distance is the Mahalanobis distance between the graph convolution enhancement features of the candidate node and the feature distribution of the typical source node. Select the node with the highest score and backtrack until the termination condition is met to locate the source node. Further, the source likelihood scoring network is a Multi-Layer Perceptron (MLP), whose input features include: the encoded value of semantic fidelity decay obtained by normalization and embedding layer transformation, the temporal feature obtained by logarithmic transformation of the propagation timestamp difference, and the Mahalanobis distance between the graph convolutional enhancement features of candidate nodes and the feature distribution of typical source nodes. The feature distribution of typical source nodes is obtained by statistical analysis of the graph convolutional enhancement features of known source nodes. The termination conditions include the current node having no incoming edge neighbors, or the source likelihood score exceeding the source confirmation threshold, or the semantic fidelity decay increment of continuous multi-hop backtracking being lower than the convergence threshold. The iterative traversal process includes the following sub-steps: Sub-step 1, initialize the target node to be traced as the current node, and initialize the cumulative semantic fidelity recovery path as an empty set. Sub-step 2, obtain the set of all incoming edge neighbors of the current node, and calculate the semantic fidelity decay value between the current node and each incoming edge neighbor. Sub-step 3: Based on the enhanced node features output by the graph convolutional network, calculate the likelihood score of each incoming neighbor node as a closer source candidate node through the source likelihood scoring network. Sub-step 4: Select the incoming neighbor node with the highest source likelihood score as the next-hop backtracking node, add this node to the cumulative semantic fidelity recovery path, and update the current node as the next-hop backtracking node. Sub-step 5: Determine whether the current node meets the source node termination condition; if the termination condition is met, determine the current node as the original propagation source node, and output the cumulative semantic fidelity recovery path as the restored complete propagation path; otherwise, return to sub-step 2 to continue iteration. In one implementation, step S6 also adopts a multi-path parallel backtracking strategy, while retaining the top K candidate backtracking paths ranked by source likelihood score, performing subsequent iterations in parallel for each path, and selecting the path with the highest cumulative source likelihood score as the source tracing result.

[0015] Step S7, specific source tracing processing for adversarial mutation propagation, includes: performing multi-granularity alignment analysis on the content of adjacent nodes in the propagation dynamic graph using a semantically invariant region detection module; identifying visually semantically invariant regions at the visual modality level through region-level feature matching; identifying semantically preserved text fragments at the semantic modality level through sentence-level semantic similarity comparison; and calculating the adversarial mutation confidence index. When the adversarial mutation confidence index exceeds a preset threshold, the deep semantic structure features of the narrative logic chain, visual composition grammar, and multimodal element combination patterns are extracted using a deep semantic structure correlation mining module; deep semantic correlation edges that cross surface mutations are established and integrated into the edge weights of the propagation dynamic graph. In one implementation, when mutations are concentrated in a specific local area while the overall semantic structure remains consistent, the adversarial mutation confidence index is high, indicating that the propagation jump has the characteristics of targeted adversarial mutations, thereby enabling the reverse semantic gradient tracing mechanism to penetrate the surface interference of adversarial mutations and accurately reconstruct the underlying propagation path.

[0016] Step S8 involves outputting the source tracing results and visualizing the propagation path. The original source node information, the reconstructed complete propagation path information, the semantic fidelity attenuation values ​​of each propagation hop, and the information transformation type identification results are structured and encapsulated for output, generating a propagation path visualization map. The color intensity of a node indicates the degree of semantic fidelity attenuation from its source node, and the thickness of the edges indicates the semantic association strength of the propagation link. Nodes identified as exhibiting adversarial mutation propagation are highlighted with special markers. In one implementation, the visualization map provides intuitive quantitative support for the correlation analysis of electronic evidence.

[0017] Another aspect of the present invention provides a multimodal content tracing and analysis system for social media dissemination links. This system includes the following modules: a multimodal data acquisition and preprocessing module, used to perform standardized preprocessing operations on the acquired multimodal dissemination data. The multimodal dissemination data includes visual modal data, semantic modal data, and dissemination metadata corresponding to each data item. The dissemination metadata includes a publication timestamp and forwarding / referencing relationships. The standardized preprocessing operations include size normalization and keyframe extraction for the visual modal data, and word segmentation and noise reduction for the semantic modal data. A multimodal semantic feature extraction and cross-modal alignment module is used to utilize deep features. The extraction network extracts visual semantic feature vectors from visual modal data and semantic feature vectors from semantic modal data, respectively. These visual semantic feature vectors and semantic feature vectors are then projected into a cross-modal semantic alignment space to generate a multimodal fusion semantic representation vector. A semantic fidelity attenuation quantification module is used to obtain the semantic fidelity attenuation by calculating the weighted cosine distance between the multimodal fusion semantic representation vectors of two propagation nodes. The weights of the weighted cosine distance are dynamically adjusted using a modality-adaptive attention mechanism. A propagation dynamic graph construction module is used to determine the propagation direction based on the forwarding and referencing relationships in the propagation metadata and construct a propagation dynamic graph based on the... The edge weights obtained from semantic fidelity decay calculation are assigned values; the graph convolution feature aggregation module is used to aggregate the neighborhood propagation features of each node through a graph convolutional network; the reverse semantic gradient tracing module is used to iteratively traverse from the target node along the directed edge in the reverse direction, and calculate the source likelihood score of the incoming edge neighbor nodes through the source likelihood scoring network. The source likelihood score integrates three factors: semantic fidelity decay, propagation timestamp order, and Mahalanobis distance. The Mahalanobis distance is the Mahalanobis distance between the graph convolutional enhanced features of the candidate node and the feature distribution of the typical source node. The node with the highest score is selected to backtrack until the termination condition is met to locate the source node; adversarial mutation propagation... The broadcast processing module is used to perform multi-granularity alignment analysis on the content of adjacent nodes on the propagation link in the propagation dynamic graph through the semantically invariant region detection submodule. At the visual modality level, it identifies visually semantically invariant regions through region-level feature matching, and at the semantic modality level, it identifies semantically preserved text segments through sentence-level semantic similarity comparison, and calculates the adversarial variation confidence index. When the adversarial variation confidence index exceeds a preset threshold, the deep semantic structure features of the narrative logic chain, visual composition grammar, and multimodal element combination patterns are extracted through the deep semantic structure correlation mining submodule. Deep semantic correlation edges that cross surface variations are established and integrated into the edge weights of the propagation dynamic graph.The source tracing result output module is used to structurally encapsulate and output the original propagation source node information obtained from the location, the restored complete propagation path information, the semantic fidelity attenuation value of each propagation hop, and the information transformation type identification results, and generate a propagation path visualization map. The color intensity of a node indicates the degree of semantic fidelity attenuation from its source node, the thickness of the edge indicates the semantic association strength of the propagation link, and nodes identified as propagating adversarial mutations are highlighted with special markers.

[0018] This invention models multimodal semantic fidelity decay as a quantifiable technical metric, making the semantic evolution of content during propagation measurable and comparable, thus addressing the problem in existing technologies where content-level semantic information is not incorporated into propagation topology modeling. Building upon this, the invention uses deep semantic features rather than surface features for source tracing matching, ensuring that even after multiple rounds of editing and mutation, the system maintains its source tracing capability through semantic fidelity decay, overcoming the limitations of traditional digital watermarking and hash fingerprinting technologies in variant content source tracing scenarios. Furthermore, by constructing a reverse source derivation mechanism based on semantic decay gradients, this invention deeply couples content-level semantic information with propagation topology information to form a dual-channel source tracing path, solving the problem of insufficient source tracing accuracy caused by existing graph neural network-based propagation analysis techniques relying solely on topological features while ignoring the semantic evolution patterns of content. Even further, by designing a specialized enhancement processing module for adversarial mutation propagation scenarios, this invention can penetrate intentionally applied surface mutation interference from propagators for deep source tracing, addressing the lack of effective source tracing methods in the face of malicious variant propagation. Furthermore, this invention introduces a platform coding degradation estimation submodule to distinguish between platform-inherent coding degradation and user-initiated edit variations, filtering out interference from platform-inherent coding noise during cross-platform propagation and improving the accuracy of cross-platform source tracing. In another embodiment of this invention, by modeling the propagation process as a forward diffusion process and the source tracing process as a reverse denoising process, source localization can be achieved without pre-constructing a complete propagation dynamic graph, avoiding erroneous edge connections that may be introduced during candidate link screening in the graph construction stage, and inherently supporting the quantification of uncertainty in source tracing results. Attached Figure Description

[0019] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] Figure 1 This is a schematic diagram of the overall process of the multimodal content tracing and analysis method for social media propagation links provided in Embodiment 1 of the present invention.

[0021] Figure 2This is a comparison diagram of the variations of multimodal content provided in Embodiment 1 of the present invention after undergoing various information transformation operations during the dissemination of social media.

[0022] Figure 3 This is a schematic diagram illustrating the effects of homologous content aggregation and non-homologous content dispersion in the cross-modal semantic alignment space provided in Embodiment 1 of the present invention.

[0023] Figure 4 This is a schematic diagram of the dynamic weight allocation of the modal adaptive attention mechanism under different information transformation types provided in Embodiment 1 of the present invention.

[0024] Figure 5 This is a schematic diagram of the lightweight attention network structure of the modality adaptive attention mechanism provided in Embodiment 1 of the present invention.

[0025] Figure 6 This is a data flow diagram of multimodal semantic fidelity attenuation quantification modeling and platform coding degradation discriminative modeling provided in Embodiment 1 of the present invention.

[0026] Figure 7 This is a schematic diagram of the algorithm flow for reverse semantic gradient tracing and source node localization provided in Embodiment 1 of the present invention.

[0027] Figure 8 This is a schematic diagram of the semantic fidelity decay gradient field and reverse tracing path in the propagation dynamic graph provided in Embodiment 1 of the present invention.

[0028] Figure 9 This is a device structure block diagram of the multimodal content tracing and analysis system for social media propagation links provided in Embodiment 4 of the present invention.

[0029] Figure 10 This is a curve comparing the source tracing accuracy of the method of the present invention provided in Embodiment 5 with that of various existing methods under different propagation hop numbers.

[0030] Figure 11 This is a heatmap comparing the path reconstruction accuracy of the method of the present invention provided in Embodiment 5 of the present invention with that of various existing methods in six propagation scenarios.

[0031] Figure 12 This is the ROC curve for detecting adversarial mutation propagation provided in Embodiment 5 of the present invention.

[0032] Figure 13 This is a graph showing the changes in precision, recall, and F1 score of the adversarial mutation propagation detection as a function of the confidence threshold, as provided in Embodiment 5 of the present invention. Detailed Implementation

[0033] To make the objectives and technical solutions of this invention clearer, the embodiments of this invention will be further described in detail below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of this invention and should not be construed as limiting the scope of protection of this invention.

[0034] Example 1 This embodiment provides a multimodal content source tracing analysis method for social media dissemination links, such as... Figure 1 As shown, the method includes the following steps: Step 1: Multimodal social media data collection and preprocessing.

[0035] The analysis involves collecting a multimodal propagation data set from a target social media platform group. This group encompasses multiple mainstream social media platforms, and data collection is achieved through the open application programming interfaces (APIs) provided by each platform or through web scraping techniques. The propagation data includes image data, video data, and their associated text data, as well as propagation metadata for each data entry. This metadata includes the posting timestamp, posting platform identifier, posting user identifier, and forwarding / referencing relationships. In practice, the data volume of a single collection task typically ranges from several thousand to hundreds of thousands of data entries, depending on the scope and duration of the event being analyzed.

[0036] Standardized preprocessing is performed on the collected raw data. In image data preprocessing, images from different sources and with different resolutions are uniformly scaled to a preset standard size. Preferably, the standard size is set to 224 pixels by 224 pixels or 336 pixels by 336 pixels to match the input specifications of the subsequent visual encoder. The scaling process uses bilinear interpolation to maintain the integrity of the image's semantic information. For video data preprocessing, frame sampling is performed first, employing an adaptive keyframe extraction strategy. This strategy detects abrupt changes based on the statistical distribution of pixel differences between adjacent frames. When the average pixel difference between adjacent frames exceeds a preset abrupt change threshold, the frame is determined to be a keyframe and retained. In specific implementations, this abrupt change threshold is set to 1.5 times the standard deviation of the average difference between adjacent frames. This adaptive strategy can filter out a large number of redundant frames while retaining semantically representative frames, compressing each video segment into 5 to 20 keyframes to reduce the computational burden. For the preprocessing of text data, word segmentation and noise reduction are performed to remove noise information such as special symbols, emoji codes, hyperlinks and platform format marks from the text, and the processed text is uniformly converted into a text sequence with sentences as the basic unit.

[0037] like Figure 2As shown, original multimodal content (including images and text) may undergo six typical information transformation operations during its dissemination on social media, leading to varying degrees of content variation. Screenshot compression reduces image resolution and blurs details; image cropping alters the composition and local areas of the content; watermarks are overlaid on images to introduce text or logos that obscure the image; rewriting text leads to semantic drift and wording changes; splicing video clips causes temporal reorganization; and adversarial variation involves intentional local modifications by the disseminator to evade detection, indicated by a dashed border. The semantic fidelity scale at the bottom of the figure shows a gradient from high to low, visually reflecting the different degrees of impact of various transformation operations on the semantic fidelity of the content.

[0038] Furthermore, the collected propagation metadata undergoes timestamp alignment. Since different social media platforms may use different time precisions and time zone standards, all timestamps need to be uniformly converted to Coordinated Universal Time (UTC) format and stored in a standardized manner in seconds to ensure the consistency and accuracy of subsequent propagation time series analysis.

[0039] Step 2: Multimodal semantic feature extraction and cross-modal aligned embedding.

[0040] Deep feature extraction networks are used to encode features for both visual and semantic modal data. For the visual modality, a pre-trained visual encoder extracts visual semantic feature vectors from keyframes of images and videos. This pre-trained visual encoder employs a model based on the Vision Transformer (ViT) architecture, which is pre-trained on large-scale image datasets and capable of extracting visual features containing rich high-level semantic information. In this specific implementation, the visual encoder divides each input image into several fixed-size image patches. Preferably, the size of each patch is set to 16 pixels by 16 pixels. After linearly mapping each patch into an embedding vector, it is fed into a multi-layer Transformer encoder for global self-attention encoding, ultimately outputting a visual semantic feature vector with a dimension of 768 or 1024.

[0041] For the semantic modality, a pre-trained language encoder is used to extract semantic feature vectors from the text data. The pre-trained language encoder employs a Transformer-based pre-trained language model, which is pre-trained on large-scale text corpora and can capture deep semantic representations of the text. The language encoder converts the input text into a sequence of tokens after word segmentation. After passing through multiple Transformer encoding layers, the output corresponding to the CLS marker (i.e., the classification flag at the beginning of the input sequence) in the final hidden layer is taken as the semantic feature vector of the text. The dimension of the semantic feature vector is consistent with that of the visual semantic feature vector, specifically set to 768 dimensions.

[0042] Furthermore, the visual semantic feature vector and the semantic feature vector are projected into a unified cross-modal semantic alignment space to generate a multimodal fused semantic representation vector. Specifically, two independent linear projection layers are used to map the visual semantic feature vector and the semantic feature vector to a shared alignment embedding space, the dimension of which is set to 512 in this implementation. In this alignment space, the multimodal fused semantic representation vector is obtained by performing weighted concatenation or element-wise addition operations on the projected visual feature vector and the semantic feature vector.

[0043] The cross-modal semantic alignment space is trained using a contrastive learning objective function. The training dataset is constructed by extracting positive sample pairs (multimodal pairs of content from the same source after different transformations) and negative sample pairs (multimodal content pairs of non-same source) from historical data of known propagation paths. Preferably, the ratio of positive to negative samples is set between 1:5 and 1:10. The loss function for contrastive learning uses InfoNCE (Info Noise Contrastive Estimation) loss, which uses all negative samples within a batch as the comparison benchmark and optimizes the projection layer parameters by maximizing the cosine similarity of positive sample pairs in the alignment space and minimizing the cosine similarity of negative sample pairs. The training process uses the Adam (Adaptive Moment Estimation) optimizer, with a learning rate preferably set between 1e-4 and 5e-4, a batch size of 128 or 256, and 50 to 100 training epochs, employing a cosine annealing learning rate decay strategy. The training hardware environment is preferably configured with a GPU equipped with 24GB or more of video memory.

[0044] Preferably, the contrastive learning training process incorporates data augmentation strategies tailored to social media dissemination scenarios. These strategies generate training samples by simulating common information transformation operations in social media dissemination. Specifically, they include: performing JPEG compression with random quality coefficients on images to simulate screenshot compression; performing random region cropping on images to simulate image cropping; overlaying text watermarks with random positions and transparency to simulate watermark overlay; performing synonym replacement and sentence structure transformation on text to simulate copywriting; and performing random truncation on text to simulate platform character limits. Through these data augmentation strategies, the robustness of the fused semantic representation to various mutation operations during dissemination is enhanced.

[0045] like Figure 3As shown, in the cross-modal semantic alignment space trained by contrastive learning, the fused semantic representation vectors of multimodal content that are from the same source but have undergone different transformation operations exhibit obvious clustering characteristics. The figure shows five clusters of content from the same source. Within each group, the source node (large marker) and its multiple variant nodes (small markers) generated through transformation operations such as screenshot compression, text rewriting, and image cropping are closely clustered in the same region, indicating that the contrastive learning objective function effectively maintains the spatial clustering of the representations of variant content from the same source. Meanwhile, the representation vectors of non-originating content (diamond markers) are dispersed in the alignment space, far from the centers of each cluster of content from the same source, verifying the effect of contrastive learning on constraining the dispersion of non-originating content.

[0046] Step 3: Quantitative modeling of multimodal semantic fidelity decay.

[0047] A multimodal semantic fidelity attenuation index D(vi, vj) is defined to quantify the semantic deviation between the content carried by any two propagation nodes vi and vj. The semantic fidelity attenuation is obtained by calculating the weighted cosine distance between the multimodal fused semantic representation vectors of the two propagation nodes, and its calculation formula is as follows: D(vi, vj) = alpha_vis * (1 - cos(f_vis_i, f_vis_j)) + alpha_sem * (1- cos(f_sem_i, f_sem_j)) Where f_vis_i and f_vis_j are the projections of the visual modal feature vectors of nodes vi and vj into the alignment space, respectively; f_sem_i and f_sem_j are the projections of the semantic modal feature vectors of nodes vi and vj into the alignment space, respectively; cos(a, b) represents the cosine similarity between vector a and vector b; and alpha_vis and alpha_sem are the dynamic weight coefficients of the visual modality and semantic modality, respectively, and satisfy alpha_vis + alpha_sem = 1.

[0048] The weighted cosine distance weighting strategy employs a modality-adaptive attention mechanism for dynamic adjustment. Specifically, when the loss in information transformation of the visual modality is greater than that of the semantic modality, the weight coefficient alpha_vis of the visual modality distance component is automatically increased; conversely, when the information loss of the semantic modality is more significant, the weight coefficient alpha_sem of the semantic modality distance component is increased. This modality-adaptive attention mechanism is implemented through a lightweight attention network, which consists of a concatenation layer, two fully connected layers, and a Softmax normalization layer. The concatenation layer concatenates the visual modality feature difference vector delta_vis = f_vis_i - f_vis_j and the semantic modality feature difference vector delta_sem = f_sem_i - f_sem_j between two nodes to form the input vector. The first fully connected layer maps the input vector to an intermediate hidden layer, preferably with a 128-dimensional dimension and ReLU activation function. The second fully connected layer maps the hidden layer output to a two-dimensional vector. The Softmax normalization layer converts the two-dimensional vector into two weight coefficients, alpha_vis and alpha_sem, ensuring that their sum is 1. Through end-to-end training, this attention network can automatically adjust the modality weight allocation according to the specific transformation characteristics of the content of each pair of nodes, so that the semantic fidelity decay more accurately reflects the actual semantic deviation of the content.

[0049] like Figure 4 As shown in the chart, the adaptive attention mechanism assigns visual and semantic modality weight coefficients under six information transformation types. When the information transformation type is screenshot compression, the visual modality weight coefficient is automatically set to a higher value because this transformation mainly affects visual modality features; when the transformation type is text rewriting, the semantic modality weight coefficient is increased accordingly because this transformation mainly affects semantic modality features. Figure 5 As shown in the diagram, the lightweight attention network concatenates the visual modal feature difference vectors and semantic modal feature difference vectors of two nodes. After processing through two fully connected layers and a Softmax normalization layer, it outputs visual modal weight coefficients and semantic modal weight coefficients. This attention mechanism can automatically adjust the weight allocation according to the specific transformation characteristics of the content of each pair of propagation nodes, making the semantic fidelity attenuation more accurately reflect the actual semantic deviation of the content.

[0050] Preferably, the calculation of the semantic fidelity attenuation also incorporates an information transformation type perception factor, gamma. An information transformation type classifier is trained to automatically identify the main information transformation types experienced by the content between two adjacent propagation nodes. This classifier takes the multimodal feature difference vectors of the two nodes as input and outputs the probability distribution of the respective information transformation types. The information transformation types include categories such as screenshot compression, text rewriting, image cropping, watermark overlay, video clip splicing, and hybrid transformation. The classifier is trained using a cross-entropy loss function, and the training data is obtained by labeling historical propagation data of known transformation types. The semantic fidelity attenuation value is calibrated according to different transformation types: D_cal(vi, vj) = D(vi, vj) * gamma(t), where t is the identified information transformation type, gamma(t) is the calibration coefficient corresponding to that transformation type, and t belongs to the information transformation type set {screenshot compression, text rewriting, image cropping, watermark overlay, video clip splicing, hybrid transformation}. The calibration coefficient is obtained by normalizing the statistical distribution of attenuation for each transformation type. This calibration makes the attenuation caused by the same type of transformation comparable across different content pairs, thereby improving the accuracy of subsequent reverse tracing.

[0051] like Figure 6 As shown, a platform encoding degradation estimation submodule is further introduced into the semantic fidelity decay calculation process. This submodule is used to solve the aliasing problem between platform-inherent encoding degradation and user-initiated editing variations in cross-platform propagation. Specifically, a platform encoding degradation model M_p is pre-trained for each target social media platform, where p is the platform identifier. The training data for this model is constructed as follows: a batch of high-fidelity original multimodal content samples is collected, each sample is uploaded to the target platform and then downloaded back to obtain a content copy encoded and compressed by the platform. The pairing of the original content and the compressed content is used as training data. The platform encoding degradation model M_p takes the fused semantic representation vector of the original high-fidelity multimodal content as input and outputs the expected semantic loss estimate D_platform(p) after platform encoding degradation. The model preferably adopts a three-layer fully connected network with hidden layer dimensions of 256 and 128, respectively, ReLU activation function, and single-neuron linear output. During training, the mean squared error (MSE) loss function was used, the Adam optimizer was used, and the learning rate was set to 1e-3.

[0052] When calculating the semantic fidelity attenuation between two adjacent nodes in the propagation link, the estimated expected semantic loss due to encoding degradation of the target platform is first subtracted to obtain the net semantic attenuation D_net(vi, vj): D_net(vi, vj) = D_cal(vi, vj) - D_platform(p_j) Here, p_j represents the social media platform identifier where node vj resides. This net semantic decay excludes semantic loss interference introduced by the target platform's inherent encoding compression operations, retaining only the semantic changes caused by the propagator's editing behavior. This net semantic decay, as the actual edge weight of the propagation dynamic graph, enables the reverse gradient tracing mechanism to filter out inherent platform encoding noise and focus on semantic changes caused by the propagator's intentional actions, thereby improving the accuracy of cross-platform tracing.

[0053] Step 4: Propagation dynamic graph construction and edge weight assignment.

[0054] Based on the collected propagation metadata, all multimedia content nodes participating in the propagation are constructed into a propagation dynamic graph G = (V, E, W). In this propagation dynamic graph, V is a set of nodes, and each node vi belongs to V, representing a propagation event, i.e., a piece of multimodal content published or forwarded by a user on a platform at a certain time. The feature vector of each node is the multimodal fusion semantic representation vector of the content corresponding to the propagation event. E is a set of directed edges, and the directed edge eij between nodes represents the propagation direction of content from node vi to node vj. W is a set of edge weights, and the weight value wij of each edge is set as the net semantic decay value D_net(vi, vj) connecting node vi and node vj.

[0055] The initial construction of the propagation dynamic graph is based on explicit reference relationships in the propagation metadata. These explicit reference relationships include forwarding relationships, referencing relationships, and reply relationships recorded by the platform. For node pairs with explicit reference relationships, directed edges are directly established in the propagation dynamic graph, and the corresponding edge weights are calculated.

[0056] Furthermore, for implicit propagation node pairs whose explicit referencing relationships cannot be directly determined through propagation metadata, a candidate link filtering strategy is adopted to supplement the edges. This strategy consists of three stages. The first stage is candidate node retrieval, which uses an approximate nearest neighbor retrieval algorithm based on multimodal fusion semantic representation vectors to retrieve the K closest candidate neighbor nodes in the semantic space for each node. The retrieval algorithm preferably uses an approximate nearest neighbor index based on a hierarchical navigable small world (HNSW) graph, and the value of K is preferably set between 10 and 50. The second stage is time series constraint filtering, which verifies the candidate node pairs retrieved in the first stage by time constraints, retaining only candidate links whose publication timestamps satisfy the chronological order relationship (i.e., the publication time of the potential source node is earlier than the publication time of the potential target node). The third stage is threshold filtering and edge connection, which calculates the cosine similarity of the fusion semantic representation vectors between candidate node pairs that pass the time constraint filtering, retains node pairs with similarity exceeding a preset threshold theta as supplementary propagation links, and calculates the corresponding net semantic decay as the edge weight. The preset threshold theta is adaptively set based on the statistical distribution of the fusion semantic similarity between adjacent nodes in the known propagation link, and preferably the 25th percentile value of the statistical distribution is taken as the threshold.

[0057] Step 5: Aggregation and enhancement of propagation features based on graph convolutional networks.

[0058] A Graph Convolutional Network (GCN) is deployed on the constructed propagation dynamic graph, aggregating the neighborhood propagation features of each node through a multi-layer message passing mechanism. The design of the GCN introduces an edge-weight-aware attention aggregation mechanism based on standard graph convolution operations. Specifically, in each layer of graph convolution operations, the formula for calculating the updated feature h_vi^(l+1) of node vi is: h_vi^(l+1) = sigma(sum_j(a_ij * W^(l) * h_vj^(l)) + h_vi^(l)) Where h_vj^(l) is the feature vector of node vj in layer l, W^(l) is the learnable weight matrix of layer l, sigma is the activation function (preferably LeakyReLU), and a_ij is the attention weight coefficient. The attention weight coefficient a_ij comprehensively considers the correlation between edge weights and node features: neighboring edges with smaller semantic fidelity decay (i.e., propagation links with stronger semantic relevance) receive higher attention weights, thus playing a dominant role in feature aggregation. The attention weight coefficient is calculated as follows: first, the feature vectors of nodes vi and vj are concatenated, and then subjected to a linear transformation and LeakyReLU activation to obtain the original attention score. Then, this score is multiplied by the inverse of the edge weight to introduce edge weight awareness. Finally, the attention scores of all neighboring nodes are normalized using Softmax.

[0059] The h_vi^(l) term in the above formula implements a residual skip connection structure to alleviate the problem of excessive feature smoothing in deep graph convolutions. The number of layers in the graph convolutional network is preferably set to 2 to 4, with the feature dimension of each layer remaining consistent with the input fused semantic representation vector. During the training phase, a multi-task learning strategy is adopted, simultaneously optimizing the objective functions for both the propagation link prediction task and the source node classification task. Training uses the Adam optimizer, with a learning rate set between 1e-3 and 5e-3, and the number of training epochs set to 200 to 500.

[0060] Preferably, after each convolutional layer of the graph convolutional network, a feature fusion operation is performed between the node features and the original multimodal fused semantic representation. Specifically, the node features h_vi^(l+1) output by the l-th graph convolutional layer are integrated with the original fused semantic representation vector f_vi of that node through a gated fusion mechanism: h_vi_fused^(l+1) = g * h_vi^(l+1) + (1 - g) * f_vi Here, g is the gating coefficient, dynamically calculated using a sigmoid-activated linear layer based on h_vi^(l+1) and f_vi. This feature fusion operation maintains the semantic anchoring of the node's content, preventing the node features from deviating too far from their original semantic meaning after multiple layers of message passing.

[0061] Step 6: Reverse semantic gradient tracing and source node localization.

[0062] like Figure 7As shown, based on the propagation dynamic graph enhanced by a graph convolutional network, a reverse semantic gradient tracing operation is performed to locate the original propagation source node. The core mechanism of the reverse semantic gradient tracing operation is as follows: starting from any target node to be traced in the propagation dynamic graph, iterative traversal is performed along the reverse direction of the directed edges; in each iteration, the path direction that minimizes the cumulative value of semantic fidelity decay is selected as the backtracking direction, that is, moving backward along the gradient direction from low to high semantic fidelity.

[0063] The iterative traversal process includes the following sub-steps: Sub-step 1: Initialize the target node v_target to be traced to the current node v_current, initialize the cumulative semantic fidelity recovery path P to an empty set, and initialize the cumulative source likelihood score sum S_total to 0.

[0064] Sub-step 2: Obtain the set N_in(v_current) of all incoming edge neighbors of the current node v_current, which is all nodes pointing to v_current in the propagation dynamic graph. For each neighbor node v_k in the set, obtain the net semantic decay value D_net(v_k, v_current) between the current node and that neighbor node from the edge weights of the propagation dynamic graph.

[0065] Sub-step 3 involves calculating the likelihood score S(v_k) of each incoming neighbor node as a closer source candidate node based on the enhanced node features output by the graph convolutional network, using a source likelihood scoring network. The source likelihood scoring network is a multi-layer perceptron (MLP), whose input features are composed of three concatenated components: the first component is the encoded value of the relative magnitude of semantic fidelity decay, obtained by normalizing the D_net(v_k, v_current) value and converting it into a feature vector through a small embedding layer; the second component is the encoded value of the propagation timestamp sequence, obtained by logarithmically transforming the difference between the publication time of candidate node v_k and the publication time of the current node v_current; and the third component is the Mahalanobis distance between the graph convolutional enhanced features of the candidate node and the typical source node feature distribution.

[0066] In the calculation of the Mahalanobis distance, the typical source node feature distribution is obtained by statistically analyzing the graph convolutional enhancement features of known source nodes in the training set, and calculating their mean vector mu_src and covariance matrix Sigma_src. The Mahalanobis distance MD(v_k) of candidate node v_k is calculated as follows: MD(v_k) = sqrt((h_vk - mu_src)^T * Sigma_src^(-1) * (h_vk - mu_src)) Where h_vk is the graph convolutional enhanced feature vector of candidate node v_k. The smaller the Mahalanobis distance value, the closer the features of the candidate node are to the feature distribution of the typical source node, and the higher the probability that it is the source node.

[0067] The MLP structure of the source likelihood scoring network contains two hidden layers with dimensions of 256 and 128, respectively. The ReLU activation function is used, and the output layer is a single-neuron sigmoid activation, outputting a likelihood score between 0 and 1. The network is trained on a training set labeled with known source nodes using a binary cross-entropy loss function.

[0068] Sub-step 4: Select the incoming neighbor node v_best with the highest source likelihood score S(v_k) as the next hop backtracking node, add this node to the cumulative semantic fidelity recovery path P = P union v_best, update the cumulative source likelihood score sum S_total = S_total + S(v_best), and update the current node v_current = v_best.

[0069] Sub-step 5: Determine if the current node meets the source node termination condition. The termination condition includes any one of the following three cases: First, the current node has no incoming edge neighbors, i.e., N_in(v_current) is an empty set, indicating that the upstream node of the propagation dynamic graph has been reached; Second, the source likelihood score S(v_current) of the current node exceeds the preset source confirmation threshold theta_src, which is preferably set between 0.85 and 0.95, indicating that the current node's height may be the original source; Third, the net semantic decay increment of consecutive multi-hop backtracking is lower than the preset convergence threshold epsilon_conv, which is preferably set between 0.01 and 0.05, indicating that further backtracking can no longer obtain meaningful semantic fidelity improvement. If the termination condition is met, the current node is determined to be the original propagation source node v_source, and the accumulated semantic fidelity recovery path P is output as the restored complete propagation path; otherwise, return to sub-step 2 to continue iteration.

[0070] Preferably, to improve the robustness of source localization, the reverse tracing operation employs a multi-path parallel backtracking strategy. In sub-step 4, not only is the single candidate node with the highest source likelihood score retained, but also the candidate backtracking paths with the highest source likelihood scores (K_beam), where K_beam is preferably set to 3 to 5. Subsequent iterations are performed in parallel for each candidate path, and finally, the path with the highest cumulative source likelihood score (S_total) is selected as the optimal source tracing result. This multi-path parallel backtracking strategy can effectively avoid tracing errors caused by local optima traps in single-path backtracking.

[0071] From a principle analysis perspective, the effectiveness of the aforementioned reverse semantic gradient tracing mechanism is based on the following technical principles. During social media dissemination, each time content undergoes a propagation hop (forwarding, screenshotting, rewriting, etc.), its multimodal fused semantic representation inevitably attenuates in semantic fidelity relative to the original source content. This attenuation exhibits a monotonically increasing trend along the propagation direction; that is, the farther a node is from the source, the greater its cumulative semantic attenuation. Therefore, traversing in the reverse direction along directed edges from any downstream node is essentially moving in the reverse direction of the semantic fidelity gradient, inherently tending towards the source node with the highest semantic fidelity. The source likelihood scoring network, by integrating three factors—semantic attenuation, timestamp order, and Mahalanobis distance—can accurately identify the most likely upstream propagation node among multiple incoming neighbors: semantic attenuation corresponds to the semantic propagation distance, timestamp constraints ensure the causal rationality of the tracing direction, and Mahalanobis distance provides prior guidance using the statistical distribution characteristics of known source nodes. The synergistic effect of these three factors ensures that the choice of each hop backtracking is both optimal at the local level and maintains directional consistency across the global propagation topology.

[0072] like Figure 8As shown, the nodes in the propagation dynamic graph are arranged hierarchically according to the number of propagation hops. The source node is located at the top of the graph, and downstream propagation nodes extend downwards sequentially. The graph labels each node according to the number of propagation hops between it and the source node. The 0th hop layer represents the original propagation source node, and the kth hop layer (k=1,2,3,…) represents the set of nodes that are k propagation operations away from the source node—that is, all nodes reachable from the source node along the propagation direction via k directed edges. A larger propagation hop number indicates that the node is farther from the source in the propagation chain. The grayscale color of the nodes represents the degree of cumulative semantic fidelity decay; nodes farther from the source are darker, reflecting the monotonically decreasing semantic fidelity along the propagation direction. The thickness of the connecting edges represents the magnitude of the net semantic decay between adjacent nodes; the smaller the decay, the thicker the edge, indicating a stronger semantic correlation. The background gradient field, from light to dark from top to bottom, visually demonstrates the gradient direction of semantic fidelity decay. The dashed arrow in the figure marks a reverse tracing path from the downstream node to the source node. This path travels in reverse along the upward direction of the semantic fidelity gradient, verifying the core technical principle of the reverse semantic gradient tracing mechanism, which inherently tends to the source node with the highest semantic fidelity.

[0073] Step 7: Specialized source tracing for the spread of resistant variants.

[0074] To address adversarial mutation propagation scenarios where propagators intentionally modify local areas to evade detection, a dedicated enhancement processing module is introduced on top of the aforementioned basic process. This module is triggered when: if the net semantic decay value of a hop on the reverse tracing path in step six is ​​abnormally high (exceeding twice the standard deviation of the path's average decay value), and simultaneously the mixed transformation probability output by the information transformation type classifier for that hop is lower than a preset threshold, it is determined that the hop may exhibit adversarial mutation propagation behavior, triggering the dedicated processing.

[0075] The specialized enhancement processing module comprises two core sub-modules. The first sub-module is the semantically invariant region detection module, which performs multi-granularity alignment analysis on the content of adjacent nodes in the propagation chain. At the visual modality level, the images of the two nodes are divided into N×N grid regions, with N preferably set between 4 and 8. Local visual feature vectors are extracted for each grid region. By calculating the cosine similarity between corresponding grid regions, regions with similarity higher than a local threshold are identified as visually semantically invariant regions, while regions with similarity lower than the local threshold are identified as variant regions. At the semantic modality level, the text of the two nodes is segmented into sentences, and the semantic similarity between each pair of sentences is calculated (based on the cosine distance of semantic feature vectors). Sentence pairs with similarity exceeding a sentence-level threshold are identified as semantically preserved text segments. Based on the distribution ratio and spatial relationship of the identified semantically invariant and variant regions, the adversarial mutation confidence index C_adv is calculated. The confidence index C_adv for adversarial mutation is calculated as follows: C_adv = (1 - R_var) * S_spatial, where R_var is the proportion of the mutated region to the total content area or text length, and S_spatial is the spatial concentration score of the mutated region. This spatial concentration score is obtained by normalizing the inverse of the standard deviation of the centroid distribution of the mutated region; the more concentrated the mutated region is locally, the higher the value of S_spatial. The value of C_adv ranges from 0 to 1; a higher value indicates a more targeted adversarial characteristic of the propagation jump. When the mutation is concentrated in a specific local area (e.g., the proportion of the mutated region to the total area or total text length is less than 30%) while the overall semantic structure remains consistent, the C_adv value is higher, indicating that the propagation jump has targeted adversarial characteristics.

[0076] The second submodule is the deep semantic structure correlation mining module. When the adversarial mutation confidence index C_adv exceeds the preset threshold theta_adv (preferably set to 0.53), deep semantic structure correlation analysis is triggered. This analysis no longer relies on direct matching of surface features, but instead extracts deep semantic structure features of the content. These deep semantic structure features include three dimensions: narrative logic chain features, obtained by performing event extraction and causal relationship identification on the text content to obtain the narrative structure topology of the text; visual composition grammar features, obtained by performing scene graph parsing on images to obtain the spatial layout and relational structure of visual elements; and multimodal element combination pattern features, obtained by analyzing the semantic association patterns between visual elements and text elements in the image and text content. The above three dimensions of deep semantic structure features are encoded into structured feature vectors. By calculating the structural similarity S_struct between the deep semantic structure features of two nodes, a deep semantic association edge across surface mutations is established. The edge weight of this deep semantic association edge is set to 1 - S_struct, and the edge weights in the propagation dynamic graph are updated after weighted fusion with a preset fusion coefficient beta and the original net semantic decay edge weights. The updated edge weights enable the reverse semantic gradient tracing mechanism to penetrate surface interference from adversarial mutations and accurately reconstruct the underlying propagation path.

[0077] From a principle analysis perspective, the effectiveness of the aforementioned adversarial variation propagation processing is based on the following technical principles. Adversarial variation operations by propagators are typically localized and purposeful—their goal is to circumvent surface feature matching detection systems by modifying the surface features of content (such as replacing local image areas or rewriting some text wording). However, to maintain the content's propagation value and informational purpose, propagators usually do not alter the deep semantic structure and narrative logic of the content. Therefore, when variations are concentrated in specific local areas while the overall semantic structure remains consistent, it can be determined that the propagation jump exhibits characteristics of adversarial variation rather than natural propagation variation. The semantically invariant region detection module quantifies the spatial distribution characteristics of variations through multi-granularity alignment analysis, enabling the distinction between locally concentrated deliberate variations and globally uniform natural decay. The narrative logic chain, visual composition grammar, and multimodal element combination patterns extracted by the deep semantic structure correlation mining module belong to the high-order abstract features of the content. These features possess inherent invariance under surface modification operations—modifying local colors of an image or replacing individual words will not change the narrative causal chain and visual spatial layout relationship of the content. The deep semantic association edges established based on this invariance can maintain the semantic connectivity of the propagation link even when the surface features have been deliberately destroyed, thus enabling the reverse tracing mechanism to penetrate surface interference and restore the underlying propagation path.

[0078] Step 8: Output and visualize the source tracing results and propagation path.

[0079] The original propagation source node information, the reconstructed complete propagation path information, the net semantic attenuation value of each propagation hop, and the information transformation type identification results obtained from the location are structured and encapsulated for output. The output data structure includes: the platform identifier, user identifier, publication timestamp, and original content of the source node; the above information for each intermediate node in the propagation path, as well as the net semantic attenuation value and information transformation type between it and the predecessor node; the total number of propagation hops in the path, and the total cumulative semantic fidelity attenuation.

[0080] Preferably, the system simultaneously outputs a visual map of the propagation path. This visual map is drawn using a directed graph layout algorithm. The color intensity of a node is achieved by mapping the cumulative semantic fidelity attenuation value between that node and the source node to a continuous color gradient; nodes with higher attenuation are darker, indicating a greater semantic deviation from the original source content. The thickness of an edge is proportional to the semantic association strength between two nodes; that is, edges with smaller net semantic attenuation are thicker. Nodes identified as propagating adversarial mutations are highlighted with special markers (such as special shapes or highlighted borders) and their adversarial mutation confidence index values ​​are indicated. This visual map provides intuitive quantitative support for the correlation analysis of electronic evidence.

[0081] In this embodiment, the evaluation metrics used include: source tracing accuracy, defined as the proportion of queries that correctly locate the true source node out of the total number of queries; path reconstruction accuracy, defined as the normalized value of the edit distance between the reconstructed path and the true propagation path; and the F1 score for adversarial mutation detection, defined as the harmonic mean of precision and recall for adversarial mutation propagation jump detection. In constructing the training dataset, the total data is divided into a training set, a validation set, and a test set in a 7:1.5:1.5 ratio. An early stopping mechanism is used during training; training stops when the source tracing accuracy on the validation set no longer improves after 10 consecutive training epochs to prevent overfitting.

[0082] Example 2 In a preferred embodiment of the present invention, the modality adaptive attention mechanism and information transformation type-aware calibration strategy in the quantitative modeling of multimodal semantic fidelity attenuation are further refined.

[0083] The lightweight attention network of the modality adaptive attention mechanism is trained using a specially constructed propagation transformation pair dataset. This dataset is constructed as follows: large-scale known propagation pair data are collected from multiple social media platforms. Each propagation pair contains content from an upstream node and a downstream node, and the main information transformation types between the propagation pairs are labeled. The dataset size ranges from hundreds of thousands to millions of pairs, and is divided into training, validation, and test sets in a 7:1.5:1.5 ratio. For each propagation pair, the visual modality feature vectors and semantic modality feature vectors of the upstream and downstream nodes are extracted, and the independent cosine distance between each modality is calculated as the ground truth value of the modality-level attenuation of the propagation pair.

[0084] The concatenation layer of the attention network concatenates the visual modality feature difference vector delta_vis and the semantic modality feature difference vector delta_sem into a 2d-dimensional vector, where d is the dimension of the single-modality feature vector. The weight matrix W1 of the first fully connected layer has a dimension of 2d x h, where h is the hidden layer dimension, preferably set to 128. The output of the hidden layer after ReLU activation is mapped to a two-dimensional vector through the second fully connected layer. The Softmax normalization layer ensures that the sum of the two weight coefficients alpha_vis and alpha_sem is strictly 1. During training, the attention network and the semantic fidelity decay calculation network are jointly trained end-to-end. The training objective function uses a combination loss L_att = L_rank + lambda * L_reg, where L_rank is the ranking loss, ensuring that alpha_vis is greater than alpha_sem in pairings where information transformation mainly affects the visual modality, and alpha_sem is greater than alpha_vis in pairings where information transformation mainly affects the semantic modality; L_reg is a regularization term, constraining the distribution of weight coefficients to avoid excessive bias towards any one modality; and lambda is a balancing coefficient, preferably set between 0.1 and 0.3. Training uses the Adam optimizer, with a learning rate of 5e-4, a batch size of 256, 100 training epochs, and a cosine annealing learning rate decay strategy.

[0085] Furthermore, the ranking loss L_rank takes the following form: For a set of training samples (vi, vj), given their true visual modal decay D_vis_gt and semantic modal decay D_sem_gt, when D_vis_gt is greater than D_sem_gt (i.e., the visual modal loss is greater), the ranking loss is max(0, alpha_sem - alpha_vis + margin), where margin is a preset interval value, preferably set to 0.1; and vice versa. This loss function ensures that the output weights of the attention mechanism are consistent with the direction of the actual modal-level decay.

[0086] The calibration process for the information transformation type perceptual factor includes the following steps. First, the information transformation type classifier adopts a three-layer fully connected network structure with an input dimension of 4d (concatenated feature difference vectors of two modalities from each node), a first hidden layer dimension of 512, a second hidden layer dimension of 256, and an output layer dimension equal to the number of information transformation type categories T. The classifier is trained using the cross-entropy loss function, and the training data is a subset of the propagation transformation pairing dataset labeled with transformation types. Training also uses the Adam optimizer, with a learning rate set to 1e-3, a batch size set to 128, and 50 training epochs.

[0087] After the classifier is trained, the distribution of the original semantic fidelity attenuation values ​​for each information transformation type in the training dataset is statistically analyzed. Let the mean attenuation of the t-th transformation be mu_t, and the standard deviation be sigma_t. The calibration coefficient gamma(t) is calculated as: gamma(t) = mu_ref / mu_t, where mu_ref is the median of the mean attenuation values ​​{mu_1, mu_2, ..., mu_T} for each transformation type. This calibration brings the attenuation caused by different transformation types back to a unified reference scale.

[0088] For the in-depth design of the platform encoding degradation estimation submodule, the training process of each platform encoding degradation model M_p is as follows: Collect no fewer than 5,000 high-fidelity multimodal content samples, and obtain a content copy of the encoded degradation for each sample through the platform's upload and download process. Extract multimodal fused semantic representation vectors from both the original and degraded content, and calculate the cosine distance between them as the platform encoding degradation ground truth for that sample. Model M_p takes the fused semantic representation vector of the original content as input, processes it through a three-layer fully connected network (dimensions of 512, 256, and 128, with ReLU activation function), and outputs a single-neuron linear output to predict the encoding degradation semantic loss value of the content on the target platform. Training uses the mean squared error loss function and the Adam optimizer, with a learning rate set to 1e-3 and 200 training epochs.

[0089] Preferably, the platform encoding degradation model also accepts content type labels as auxiliary input (such as pure image content, mixed image and text content, video content, etc.). After encoding the content type into a feature vector through the embedding layer, it is concatenated with the fused semantic representation vector as network input to improve the degradation prediction accuracy for different content types.

[0090] In the calculation of net semantic decay, an outlier detection mechanism is further introduced. When the calculated net semantic decay D_net(vi, vj) is negative (i.e., the calibrated decay is lower than the platform coding degradation estimate), it indicates that the platform coding degradation model overestimates the degradation of that specific content. In this case, the value of D_net(vi, vj) is truncated to zero, and the edge is marked as an outlier edge in the platform degradation estimate. In the subsequent graph convolution feature aggregation process, the attention weight of such outlier edges will be appropriately reduced to minimize their interference with the overall source tracing results.

[0091] The beneficial effects of the above implementation methods include the following aspects. Regarding the adaptability of weight allocation, through end-to-end training driven by ranking loss, the modality adaptive attention mechanism can automatically adjust the weight contributions of the visual and semantic modalities according to the specific transformation characteristics of the content of each pair of propagation nodes. Compared with a fixed weight allocation method, it can maintain the accuracy of attenuation measurement in different types of information transformation scenarios. Regarding cross-type comparability, the calibration processing of the information transformation type perception factor unifies the attenuation caused by different transformation types to the same reference scale, enabling fair comparison and selection of propagation jumps of different transformation types during the back-gradient tracing process. Regarding cross-platform accuracy, the platform coding degradation estimation submodule filters out platform-inherent coding noise, allowing the net semantic attenuation to more accurately reflect the actual editing behavior of the propagator. In cross-platform propagation links involving three or more platforms, the source tracing accuracy can achieve a considerable improvement compared to schemes without degradation decoupling.

[0092] From a principle analysis perspective, the effectiveness of the aforementioned techniques is based on the following technical principles: Information transformation operations in social media dissemination have an asymmetric impact on different modalities—screenshot compression and image cropping primarily affect visual modal features, while text rewriting and translation primarily affect semantic modal features. Fixed-weight distance metrics cannot capture this asymmetry, leading to a systematic overestimation or underestimation of attenuation under certain transformation types. Modality-adaptive attention mechanisms learn the modality-level loss distribution between each pair of content and dynamically adjust weights to eliminate this systematic bias. The principle of information transformation type-aware calibration lies in the inherent differences in the magnitude of attenuation caused by different types of information transformations in the semantic space—screenshot compression typically causes smaller attenuation, while text rewriting typically causes larger attenuation. Without calibration, the backtracking process systematically biases towards propagation paths undergoing screenshot compression (due to its smaller surface attenuation value), ignoring potentially correct paths undergoing text rewriting. Calibration maps the attenuation of different transformation types to a uniform scale, eliminating this selection bias. The principle of decoupling platform coding degradation is that the observed attenuation in cross-platform propagation is the result of the superposition of two factors: platform coding degradation and user editing behavior. If decoupling is not performed, the reverse tracing will incorrectly attribute platform coding degradation to user editing behavior, resulting in a systematic deviation in the path selection of cross-platform propagation jumps.

[0093] Example 3 In another embodiment of the present invention, a propagation link tracing method based on a temporal diffusion probability model is provided. This method does not rely on explicitly constructing a propagation dynamic graph and graph convolutional network, but instead utilizes the temporal diffusion law of content semantic features during the propagation process to locate the source.

[0094] This method treats the multimodal fused semantic representation vector of content along the propagation path as a temporal signal performing a random walk in the semantic space. Based on the nonlinear characteristics of semantic fidelity decay during propagation, a conditional diffusion probability model is established. This model models the propagation process as a forward diffusion process: the high-fidelity semantic representation of the source node undergoes several diffusion steps (each step corresponding to a propagation hop), gradually accumulating information transformation noise and degenerating into a low-fidelity semantic representation of the downstream node. The reverse tracing process is modeled as a reverse denoising process: starting from the low-fidelity semantic representation of the downstream node, a trained denoising network gradually removes the information transformation noise introduced by each hop, recovering a high-fidelity semantic representation that increasingly approximates the source node.

[0095] Specifically, let the number of hops in the propagation link be T. The forward diffusion process is defined as: f_t = sqrt(1 - beta_t) * f_(t-1) + sqrt(beta_t) * epsilon_t, where f_0 is the fused semantic representation vector of the source node, f_t is the representation vector of the downstream node after the t-th hop, beta_t is the noise scheduling coefficient of the t-th hop (related to the information transformation intensity of that hop), and epsilon_t is the standard normally distributed noise. The noise scheduling coefficient beta_t is estimated by parameter analysis of known propagation links. The value of beta_t is different at different hop positions, reflecting the non-uniformity of attenuation intensity during propagation.

[0096] The reverse denoising process employs a conditional denoising network, theta, which takes the noisy representation f_t of the current step and the step number t as input to predict the noise components to be removed in the current step. The denoising network uses a Transformer-based sequence denoising architecture, where the step number t is converted into a time-step embedding vector through sinusoidal position encoding, concatenated with the noisy representation f_t, and then fed into a multi-layer Transformer encoder for processing. The denoising network is trained on training data from a propagation path with known sources, and the loss function is the mean square error between the predicted noise and the actual noise.

[0097] During the source tracing execution phase, given the fused semantic representation vector f_T of the target node to be traced, the reverse denoising process starts from f_T and iteratively executes T denoising steps. Each denoising step yields an intermediate estimated representation f_(Tk), which corresponds to the predicted fused semantic representation of the Tk-th hop node in the propagation path. By searching the known node library for the actual node closest to each intermediate estimated representation, the complete propagation path is reconstructed. When the denoising iteration converges to f_0, the actual node closest to f_0 is the inferred source node.

[0098] Preferably, in a variant implementation of this method, the noise scheduling coefficient beta_t is not used with a preset value, but is adaptively predicted through an auxiliary noise intensity estimation network. This estimation network takes the fused semantic representation vectors of two adjacent nodes as input and outputs an estimated noise intensity value for the propagation hop. This variant allows the model to adapt to the differences in noise intensity patterns across different propagation paths.

[0099] This implementation method eliminates the need to pre-construct a complete propagation dynamic graph during source localization, avoiding erroneous edge connections that may be introduced during candidate link selection in the graph construction phase. Furthermore, the reverse denoising process based on the diffusion probability model inherently supports uncertainty quantification—by sampling the denoising process multiple times, confidence distribution information of the source tracing results can be obtained, providing support for the reliability assessment of the source tracing results.

[0100] From a principle analysis perspective, the effectiveness of the aforementioned source tracing method based on the temporal diffusion probability model is based on the following technical principles. The hop-by-hop decay process of content semantic features during social media propagation shares a structural similarity in mathematical form with the forward noise addition process in the diffusion model—the high-fidelity semantic representation of the source node degenerates into a low-fidelity representation of the downstream node after being progressively superimposed with propagation noise, similar to how the original data in the diffusion model degenerates into pure noise after being progressively added with Gaussian noise. Based on this correspondence, the reverse source tracing process can be naturally modeled as a reverse denoising process: the trained denoising network learns the statistical regularity of the propagation noise at each hop, and can progressively remove propagation noise from the noisy representation of the downstream node, recovering a high-fidelity representation that increasingly approximates the source. The adaptability of the noise scheduling coefficient beta_t reflects the difference in attenuation intensity at different hop positions during propagation, enabling the model to adapt to non-uniform decay patterns. Compared to methods based on propagation dynamic graphs, the core advantage of this method lies in transforming the source tracing problem from a graph search paradigm to a sequence generation paradigm, avoiding dependence on the complete propagation topology. At the same time, the random sampling characteristics of the reverse denoising process inherently provide confidence distribution information of the source tracing results.

[0101] Example 4 like Figure 9 As shown, this embodiment provides a multimodal content tracing and analysis system for social media dissemination links. The system includes the following modules: The multimodal data acquisition and preprocessing module is used to collect multimodal propagation datasets from the target social media platform group and perform standardized preprocessing operations. The multimodal propagation dataset includes image data, video data, and associated text data, as well as propagation metadata for each data entry. This metadata includes the posting timestamp, posting platform identifier, posting user identifier, and forwarding / referencing relationships. The preprocessing operations include image size normalization, video frame sampling and keyframe extraction, and text segmentation and denoising. This module outputs the preprocessed standardized multimodal data and aligned propagation metadata to the multimodal semantic feature extraction and cross-modal alignment module.

[0102] The multimodal semantic feature extraction and cross-modal alignment module utilizes a deep feature extraction network to encode features for both visual and semantic modal data. For the visual modality, this module employs a pre-trained visual encoder to extract visual semantic feature vectors from keyframes of images and videos; for the semantic modality, a pre-trained language encoder extracts semantic feature vectors from text data. This module further projects the visual and semantic feature vectors into a unified cross-modal semantic alignment space to generate a multimodal fused semantic representation vector. The cross-modal semantic alignment space is trained using a contrastive learning objective function, ensuring that the representations of multimodal content originating from the same source but undergoing different transformations remain clustered, while the representations of content originating from different sources remain dispersed. This module outputs the multimodal fused semantic representation vectors from each node to a semantic fidelity decay quantification module and a propagation dynamic graph construction module.

[0103] The semantic fidelity decay quantification module obtains the semantic fidelity decay by calculating the weighted cosine distance between the multimodal fused semantic representation vectors of two propagation nodes. The weighting strategy for the weighted cosine distance employs a modality-adaptive attention mechanism for dynamic adjustment, and an information transformation type-aware factor is introduced to calibrate the decay value. This module also includes a platform encoding degradation estimation submodule, which estimates the expected semantic loss caused by encoding degradation for each target social media platform and calculates the net semantic decay by subtracting this estimate. This module outputs the calculated net semantic decay value to the propagation dynamic graph construction module as edge weights.

[0104] The propagation dynamic graph construction module is used to construct a propagation dynamic graph from all multimedia content nodes participating in the propagation based on propagation metadata and assign edge weights. In the propagation dynamic graph, the node features are the multimodal fusion semantic representation vectors of the corresponding content, the directed edges represent the propagation direction of the content, and the edge weights are the net semantic decay values ​​between the two nodes. This module also includes a candidate link filtering submodule, used to establish supplementary edges for implicit propagation node pairs through approximate nearest neighbor retrieval and time series constraints. This module outputs the constructed propagation dynamic graph to the graph convolution feature aggregation module.

[0105] The graph convolutional feature aggregation module deploys a graph convolutional network (GCNN) with an edge-weight-aware attention aggregation mechanism on the propagation dynamic graph. It aggregates the neighborhood propagation features of each node through a multi-layer message passing mechanism. The GCNN in this module employs a residual skip connection structure and performs a gated feature fusion operation between node features and the original multimodal fused semantic representation after each convolutional layer to maintain the semantic anchoring of node content. This module outputs the enhanced node features from the graph convolution to the inverse semantic gradient tracing module.

[0106] The reverse semantic gradient tracing module iteratively traverses the directed edges from the target node to be traced. It uses a source likelihood scoring network to calculate the source likelihood score of each incoming neighbor node based on three factors: the relative magnitude of semantic fidelity decay, the order of propagation timestamps, and the Mahalanobis distance between the graph convolutional enhancement features of candidate nodes and the feature distribution of typical source nodes. The module selects the node with the highest score for backtracking until the source node termination condition is met, thus locating the original propagation source node and outputting the complete propagation path. This module preferably employs a multi-path parallel backtracking strategy to improve tracing robustness.

[0107] The adversarial mutation propagation processing module provides specialized enhancements for adversarial mutation propagation scenarios. This module comprises a semantically invariant region detection submodule and a deep semantic structure correlation mining submodule. The semantically invariant region detection submodule identifies visually semantically invariant regions and semantically preserved text fragments through multi-granularity alignment analysis and calculates the adversarial mutation confidence index. The deep semantic structure correlation mining submodule is triggered when the adversarial mutation confidence exceeds a threshold. It extracts deep semantic structure features such as narrative logic chains, visual composition grammar, and multimodal element combination patterns to establish deep semantic correlation edges across surface mutations, enabling the reverse semantic gradient tracing mechanism to penetrate the surface interference of adversarial mutations and reconstruct the underlying propagation path.

[0108] The source tracing result output module is used to structurally encapsulate and output the original propagation source node information obtained from the location, the restored complete propagation path information, the net semantic attenuation value of each propagation hop and the information transformation type identification results, and generate a visual map of the propagation path.

[0109] The hardware deployment environment of the above system adopts a distributed server cluster architecture. The multimodal semantic feature extraction and cross-modal alignment module and the graph convolutional feature aggregation module are deployed on computing nodes equipped with GPU computing acceleration cards, while the remaining modules are deployed on general-purpose computing nodes. The modules transmit data asynchronously through message queues to support the real-time processing requirements of large-scale data propagation.

[0110] Example 5 In this embodiment, the task of tracing the spread of multimodal content in a group of social media platforms is used as an application case to illustrate the implementation process and technical effects of the present invention.

[0111] The application scenario is described as follows: A piece of multimodal content containing images and text was first published on social media platform A, and then continued to spread on platforms B, C, and D within 72 hours through user forwarding, screenshot sharing, and text rewriting. During the spread, some disseminators performed editing operations on the original content, such as image cropping, text summary rewriting, and watermark overlay, while others made targeted local modifications to the content in an attempt to circumvent content detection. After system collection, a total of 4,376 relevant multimodal dissemination data were obtained, involving four social media platforms.

[0112] In step one, the multimodal data acquisition and preprocessing module standardized 4376 pieces of propagation data. Image data were uniformly normalized to a standard size of 224 pixels by 224 pixels; 238 pieces of data containing video were processed using an adaptive keyframe extraction strategy, resulting in an average of 8 keyframes extracted from each video segment; text data were converted into text sequences in sentence units after word segmentation and noise reduction. Timestamps were uniformly converted to UTC format and stored in seconds.

[0113] In step two, the multimodal semantic feature extraction and cross-modal alignment modules encode features into the processed data. The visual encoder extracts a 768-dimensional visual semantic feature vector using the ViT-B / 16 architecture (where B represents the base size and 16 represents an image patch size of 16 pixels), and the language encoder extracts a 768-dimensional semantic feature vector. The feature vectors from both modalities are mapped to a 512-dimensional cross-modal semantic alignment space via a linear projection layer, generating multimodal fused semantic representation vectors for each node.

[0114] In step three, the semantic fidelity attenuation quantification module calculates the net semantic attenuation for each node pair. The modality adaptive attention mechanism automatically assigns weight coefficients for visual and semantic modalities based on the specific content transformation characteristics of each node pair. The information transformation type classifier identifies five types of transformation operations involved in the propagation process: screenshot compression (approximately 35%), text rewriting (approximately 28%), image cropping (approximately 18%), watermark overlay (approximately 12%), and hybrid transformation (approximately 7%), and calibrates the attenuation values ​​accordingly. The platform encoding degradation estimation submodule calculates estimated encoding degradation semantic loss values ​​for the four platforms: platform A has an average encoding degradation loss of 0.023, platform B 0.031, platform C 0.018, and platform D 0.027. Subtracting the platform degradation loss yields the net semantic attenuation value for each edge. The data above show that the modal adaptive attention mechanism can automatically adjust the weight allocation according to the different content transformation characteristics, the information transformation type calibration factor realizes the comparability of the attenuation degree of different transformation types, and the platform coding degradation estimation submodule can effectively distinguish the inherent coding degradation characteristics of different platforms.

[0115] In step four, the propagation dynamic graph construction module established 2841 directed edges based on explicit references. A candidate link filtering strategy further identified 763 implicit propagation links, and after time-series constraint filtering and threshold filtering, 412 supplementary edges were retained. The final constructed propagation dynamic graph contains 4376 nodes and 3253 weighted directed edges.

[0116] In step five, a three-layer graph convolutional network is deployed on the propagation dynamic graph for feature aggregation and enhancement. After multi-layer edge-weight-aware attention aggregation, the enhanced features of each node integrate their neighborhood propagation context information.

[0117] In step six, 100 nodes are randomly selected from the leaf nodes of the propagation dynamic graph as target nodes to be traced, and reverse semantic gradient tracing is performed on each. The source likelihood scoring network integrates semantic decay, time series, and Mahalanobis distance for hop-by-hop backtracking. 87 target nodes were accurately located to the correct source node through single-path backtracking, while 13 target nodes triggered a multi-path parallel backtracking strategy (K_beam set to 3), of which 9 were accurately located to the source node after multi-path backtracking, achieving a final source tracing accuracy of 96%. The mean path reconstruction accuracy (measured by edit distance normalization) was 0.92. These source tracing accuracy and path reconstruction accuracy demonstrate that the reverse semantic gradient tracing mechanism combined with the source likelihood scoring network can effectively achieve accurate location of the propagation source, and the multi-path parallel backtracking strategy further improves the robustness of source tracing in complex propagation scenarios.

[0118] Furthermore, to verify the source tracing performance of the method of the present invention under different propagation depths, comparative experiments were designed on the aforementioned test set. The comparative methods included three existing source tracing techniques: a propagation topology analysis method based on graph neural networks (utilizing only the structural features of the propagation graph without incorporating content semantic information), a passive evidence collection method based on digital watermarking (matching the source by extracting the embedded digital watermark identifier), and a content matching method based on perceptual hash fingerprinting (performing precise matching by calculating the perceptual hash value of the content). All comparative methods were executed on the same test set, and the evaluation metric adopted the aforementioned source tracing accuracy definition. Tests were grouped by the number of propagation hops from 1 to 10, with each group containing no fewer than 50 query samples. Each method was independently executed 5 times, and the mean and standard deviation were calculated. Figure 10As shown, the source tracing accuracy of the method of this invention is 0.99 at 1 hop, decreasing to 0.82 as the number of propagation hops increases to 10 hops; the graph neural network topology analysis method has an accuracy of 0.95 at 1 hop, decreasing to 0.40 at 10 hops; the accuracy of the digital watermarking passive evidence collection method and the hash fingerprint matching method decreases to below 0.35 and 0.28 respectively at 5 hops. The standard deviation range of 5 experiments is marked around the curves of each method. The standard deviation of the method of this invention is smaller than that of the comparative method in each hop count group. The above experimental data show that the method of this invention can maintain a high source tracing accuracy in scenarios with long propagation links, and the consistency of the source tracing results is better than that of the comparative method.

[0119] Based on the comparative experiments described above, the accuracy of path reconstruction was further evaluated across different scenarios. The source tracing queries in the test set were divided into six groups according to propagation scenarios: single-platform short links (propagation hops not exceeding 3 and limited to a single platform), single-platform long links (propagation hops exceeding 3 and limited to a single platform), cross-platform short links (involving two or more platforms and hops not exceeding 3), cross-platform long links (involving two or more platforms and hops exceeding 3), links with adversarial mutations (the propagation path contains at least one propagation hop identified by the adversarial mutation processing module), and mixed transformation scenarios (the propagation path contains three or more information transformation types). The evaluation metrics adopted the path reconstruction accuracy definition described above. Figure 11 As shown, the heatmap displays the path reconstruction accuracy values ​​of the four methods in the six propagation scenarios using grayscale levels. The path reconstruction accuracy of the method of this invention is no less than 0.85 in all six scenarios. In scenarios with adversarial mutations, the path reconstruction accuracy of the method of this invention is 0.87, the graph neural network topology analysis method is 0.45, the digital watermarking passive forensics method is 0.15, and the hash fingerprint matching method is 0.10. In cross-platform long-link scenarios, the path reconstruction accuracy of the method of this invention is 0.89, while the other three comparative methods are all below 0.62. The above scenario-specific evaluation results show that the method of this invention has a significant improvement in path reconstruction accuracy compared to the comparative methods in propagation scenarios involving adversarial mutations and cross-platform long-link propagation.

[0120] In step seven, the system detected 37 propagation hops in the propagation dynamic graph whose net semantic decay values ​​exceeded twice the standard deviation of the path mean. Of these, 23 were identified by the adversarial mutation propagation processing module as having targeted adversarial mutation characteristics. The semantically invariant region detection module confirmed that the mutated regions in these propagation hops were concentrated only within a local range of 15% to 25%. After performing deep semantic analysis on these 23 propagation hops, the deep semantic structure correlation mining module successfully established deep semantic correlation edges, enabling the reverse tracing of related paths to penetrate the interference of surface mutations. The F1 score for adversarial mutation propagation detection reached 0.85. The above detection and tracing results demonstrate that the adversarial mutation propagation processing module can accurately identify targeted adversarial mutation characteristics, and the deep semantic structure correlation mining can penetrate the interference of surface mutations to establish effective propagation correlations, verifying the practicality of this invention in malicious variant propagation tracing scenarios.

[0121] Furthermore, a detailed classification performance evaluation was conducted on the aforementioned adversarial variant propagation detection results. The evaluation method employed Receiver Operating Characteristic (ROC) curve analysis, plotting the true positive rate on the vertical axis and the false positive rate on the horizontal axis, and calculating the area under the curve (AUC) as a quantitative indicator of overall discriminative ability. Comparison methods included: a detection method based on joint modeling of graph neural network topological features and content features (hereinafter referred to as the GNN topology plus content detection method), and a detection method based on direct matching of surface visual features and text features (hereinafter referred to as the surface feature matching method). The test data came from manually labeled adversarial variant propagation skip samples in the aforementioned test set, including 37 positive samples (confirmed as adversarial variants) and 163 negative samples (naturally propagated variants). Figure 12 As shown, the AUC value of the method of this invention is 0.777, the AUC value of the GNN topology plus content detection method is 0.643, and the AUC value of the surface feature matching method is 0.545. Figure 13 The figure shows the curves of precision, recall, and F1 score of the method of the present invention as a function of the adversarial mutation confidence threshold theta_adv. As the threshold increases from 0.50 to 0.90, precision gradually increases from 0.72 to 0.97, recall gradually decreases from 0.96 to 0.62, and the F1 score reaches its maximum value of 0.85 when the threshold is 0.53, corresponding to the preferred setting value of the adversarial mutation confidence threshold theta_adv described in step seven. The above ROC curve analysis and threshold sensitivity analysis results provide a quantitative basis for the selection of threshold parameters in the adversarial mutation propagation detection module.

[0122] The above application examples demonstrate that the technical solution of the present invention can effectively locate the original propagation source and restore the complete propagation path in complex scenarios involving multi-platform, multi-type information transformation and adversarial mutation propagation.

[0123] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for multimodal content tracing and analysis of social media dissemination links, characterized in that, Includes the following steps: Step S1: Perform standardization preprocessing on the acquired multimodal propagation data. The multimodal propagation data includes visual modal data, semantic modal data, and propagation metadata corresponding to each data item. The propagation metadata includes publication timestamp and forwarding reference relationship. The standardization preprocessing includes size normalization and keyframe extraction for visual modal data, and word segmentation and noise reduction for semantic modal data. Step S2: Use a deep feature extraction network to extract visual semantic feature vectors from the preprocessed visual modal data and semantic feature vectors from the preprocessed semantic modal data. Project the visual semantic feature vectors and the semantic feature vectors into the cross-modal semantic alignment space to generate a multimodal fusion semantic representation vector. Step S3: The semantic fidelity attenuation is obtained by calculating the weighted cosine distance between the multimodal fusion semantic representation vectors of the two propagation nodes. The weight of the weighted cosine distance is dynamically adjusted using a modality adaptive attention mechanism. Step S4: Determine the propagation direction and construct a propagation dynamic graph based on the forwarding and referencing relationships in the propagation metadata, and assign values ​​to the edge weights calculated based on the semantic fidelity attenuation. Step S5: Aggregate the neighborhood propagation features of each node in the propagation dynamic graph through a graph convolutional network; Step S6: Iterate along the directed edge in the opposite direction from the target node, calculate the source likelihood score of the incoming edge neighbor node through the source likelihood scoring network. The source likelihood score integrates three factors: semantic fidelity decay, propagation timestamp order and Mahalanobis distance. The Mahalanobis distance is the Mahalanobis distance between the graph convolution enhancement feature of the candidate node and the feature distribution of the typical source node. Select the node with the highest score and backtrack until the termination condition is met to locate the source node. The propagation timestamp is derived from the propagation metadata obtained in step S1.

2. The method according to claim 1, characterized in that, In step S2, the cross-modal semantic alignment space is trained by a contrastive learning objective function, so that the representations of multimodal content that are from the same source but have undergone different transformation operations remain clustered in the cross-modal semantic alignment space, while the representations of non-same-source content remain dispersed. The contrastive learning training process introduces data augmentation strategies such as simulated screenshot compression, text rewriting, and image cropping.

3. The method according to claim 1, characterized in that, In step S3, the modality adaptive attention mechanism is implemented through a lightweight attention network. The lightweight attention network takes the concatenation of the visual modality feature difference vector and the semantic modality feature difference vector of two nodes as input, and outputs the visual modality weight coefficient and the semantic modality weight coefficient after passing through two fully connected layers and a Softmax normalization layer. The calculation of the semantic fidelity attenuation also introduces an information transformation type perception factor. The information transformation type classifier identifies the information transformation type that the content between adjacent propagation nodes undergoes, and the semantic fidelity attenuation value is calibrated according to the information transformation type.

4. The method according to claim 3, characterized in that, Step S3 also introduces a platform coding degradation estimation submodule, which pre-trains a platform coding degradation model for each target social media platform. The platform coding degradation model outputs the expected semantic loss estimate after platform coding degradation. When calculating the semantic fidelity decay, the expected semantic loss estimate is subtracted to obtain the net semantic decay, which is used as the edge weight of the propagation dynamic graph.

5. The method according to claim 1, characterized in that, In step S4, for implicit propagation node pairs whose explicit reference relationships cannot be determined through propagation meta-information, a candidate link filtering strategy is adopted to supplement the edges. The candidate link filtering strategy includes: filtering node pairs with semantic similarity exceeding a preset threshold based on the approximate nearest neighbor retrieval of multimodal fusion semantic representation vectors, filtering unreasonable candidate links in combination with time series constraints, calculating the semantic fidelity decay degree of the candidate links that pass the filtering and assigning edge weights.

6. The method according to claim 1, characterized in that, In step S5, the graph convolutional network introduces an edge-weight-aware attention aggregation mechanism, where neighboring edges with smaller semantic fidelity decay receive higher attention weights. The inter-layer connections of the graph convolutional network adopt a residual skip connection structure, and after each convolutional layer, a gated feature fusion operation is performed between node features and the original multimodal fusion semantic representation vector.

7. The method according to claim 1, characterized in that, In step S6, the source likelihood scoring network is a multilayer perceptron, and its input features include: the encoded value of semantic fidelity decay obtained by normalization and embedding layer transformation, the temporal feature obtained by logarithmic transformation of propagation timestamp difference, and the Mahalanobis distance between the graph convolutional enhancement features of candidate nodes and the feature distribution of typical source nodes. The feature distribution of typical source nodes is obtained by statistical analysis of the graph convolutional enhancement features of known source nodes. The termination conditions include the current node having no incoming edge neighbor nodes, or the source likelihood score exceeding the source confirmation threshold, or the semantic fidelity decay increment of continuous multi-hop backtracking being lower than the convergence threshold.

8. The method according to claim 1, characterized in that, Step S6 also employs a multi-path parallel backtracking strategy, while retaining the top K candidate backtracking paths ranked by source likelihood score. Subsequent iterations are performed in parallel for each path, and the path with the highest cumulative source likelihood score is selected as the source tracing result.

9. The method according to claim 1, characterized in that, It also includes steps S7 and S8; step S7 is a special source tracing process for adversarial mutation propagation, including: performing multi-granularity alignment analysis on the content of adjacent nodes on the propagation link in the propagation dynamic graph through the semantic invariant region detection module, identifying visual semantic invariant regions through region-level feature matching at the visual modality level, identifying semantically preserved text fragments through sentence-level semantic similarity comparison at the semantic modality level, and calculating the adversarial mutation confidence index; when the adversarial mutation confidence index exceeds a preset threshold, extracting deep semantic structure features of narrative logic chain, visual composition grammar and multimodal element combination pattern through the deep semantic structure correlation mining module, establishing deep semantic correlation edges across surface mutations and integrating them into the edge weights of the propagation dynamic graph; Step S8 outputs the source tracing results and visualizes the propagation path. The original propagation source node information, the restored complete propagation path information, the semantic fidelity attenuation value of each propagation hop, and the information transformation type identification results are structured and encapsulated for output, and a propagation path visualization map is generated. The color intensity of the node indicates the degree of semantic fidelity attenuation with the source node, and the thickness of the edge indicates the semantic association strength of the propagation link. Nodes identified as adversarial mutation propagation are highlighted with special markers.

10. A multimodal content tracing and analysis system for social media dissemination links, characterized in that, include: The multimodal data acquisition and preprocessing module is used to perform standardized preprocessing operations on the acquired multimodal propagation data. The multimodal propagation data includes visual modal data, semantic modal data, and propagation meta-information corresponding to each data item. The propagation meta-information includes publication timestamp and forwarding reference relationship. The standardized preprocessing operations include size normalization and keyframe extraction for visual modal data, and word segmentation and noise reduction for semantic modal data. The multimodal semantic feature extraction and cross-modal alignment module is used to extract visual semantic feature vectors from visual modal data and semantic feature vectors from semantic modal data using a deep feature extraction network, and project the visual semantic feature vectors and the semantic feature vectors into the cross-modal semantic alignment space to generate a multimodal fusion semantic representation vector. The semantic fidelity decay quantification module is used to obtain the semantic fidelity decay by calculating the weighted cosine distance between the multimodal fusion semantic representation vectors of two propagation nodes. The weight of the weighted cosine distance is dynamically adjusted using a modality adaptive attention mechanism. The propagation dynamic graph construction module is used to determine the propagation direction and construct the propagation dynamic graph based on the forwarding and referencing relationships in the propagation metadata, and to assign values ​​to the edge weights calculated based on the semantic fidelity attenuation. The graph convolutional feature aggregation module is used to aggregate the neighborhood propagation features of each node through a graph convolutional network. The reverse semantic gradient tracing module is used to iteratively traverse the target node along the reverse direction of the directed edge. It calculates the source likelihood score of the incoming edge neighbor node through the source likelihood scoring network. The source likelihood score integrates three factors: semantic fidelity decay, propagation timestamp order, and Mahalanobis distance. The Mahalanobis distance is the Mahalanobis distance between the graph convolution enhancement feature of the candidate node and the feature distribution of the typical source node. The node with the highest score is selected to backtrack until the termination condition is met, and the source node is located. The adversarial mutation propagation processing module is used to perform multi-granularity alignment analysis on the content of adjacent nodes on the propagation link in the propagation dynamic graph through the semantically invariant region detection submodule. At the visual modality level, it identifies visually semantically invariant regions through region-level feature matching, and at the semantic modality level, it identifies semantically preserved text fragments through sentence-level semantic similarity comparison, and calculates the adversarial mutation confidence index. When the adversarial mutation confidence index exceeds a preset threshold, the deep semantic structure features of the narrative logic chain, visual composition grammar, and multimodal element combination patterns are extracted through the deep semantic structure correlation mining submodule. Deep semantic correlation edges that cross surface mutations are established and integrated into the edge weights of the propagation dynamic graph. The source tracing result output module is used to structurally encapsulate and output the original propagation source node information obtained from the location, the restored complete propagation path information, the semantic fidelity attenuation value of each propagation hop, and the information transformation type identification results, and generate a propagation path visualization map. The color of the node indicates the degree of semantic fidelity attenuation with the source node, the thickness of the edge indicates the semantic association strength of the propagation link, and the nodes identified as adversarial mutation propagation are highlighted with special markers.

Images