Network Content Dissemination Method and System Based on Integrated Cognitive Comprehension and Intelligent Governance

Abstract

The present application discloses a network content dissemination method and system based on integrated cognitive understanding and intelligent governance. The method includes constructing a subgraph network for each modality to generate a cross-modal feature graph representation; performing topic modeling on the texts in the text knowledge base to extract potential topic words and construct an event graph; retrieving nodes related to hotspots from the event graph to form a hotspot pattern, and matching content features similar to the hotspot pattern in the cross-modal feature graph representation to identify potential hotspot content; constructing a cross-modal content graph, learning an anomaly pattern using the labeled abnormal data, training an anomaly detection model according to the anomaly pattern, and identifying abnormal content in the cross-modal content graph; predicting the content dissemination links of the hotspot content and the abnormal content; and determining the time, location, and target audience of content placement by adjusting the weights of the key nodes in the content dissemination link. The present application achieves precise personalized recommendation and maximization of the dissemination effect.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and particularly to a network content dissemination method and system based on the integration of cognitive understanding and intelligent governance. Background Art

[0002] In the current all-media era, the speed and breadth of information dissemination have increased unprecedentedly, which poses higher requirements for the accurate perception, intelligent dissemination, and effective governance of content. First, the heterogeneity and complex interactivity of all-media content are the main technical challenges currently faced. There are significant differences in the structure and semantics of different modalities of data such as audio, video, images, and text. Traditional single-modal processing methods cannot effectively integrate this multi-modal information, resulting in incomplete and inaccurate information. In addition, the dynamic spatio-temporal coupling of the information dissemination network also increases the uncertainty of content dissemination, making it difficult to predict and control the dissemination path and effect. Second, existing content dissemination technologies have deficiencies in the identification of hot and abnormal content. Although traditional keyword matching and frequency statistics methods are simple, their effectiveness is limited when dealing with complex text and deep semantic information. For example, on social media, the formation of hot topics is often the result of the combined action of multiple factors, while abnormal content such as false information and sensitive images has concealment and diversity, making it difficult to accurately identify through traditional methods. In addition, the prediction of the dissemination effect of network content and the distribution guidance also face many challenges. Existing dissemination models mostly adopt static analysis and cannot capture the dynamic changes in the dissemination process in real time, resulting in insufficient accuracy and reliability of the prediction results. At the same time, the optimization of content distribution strategies also lacks effective technical support and it is difficult to achieve accurate personalized recommendations and maximize the dissemination effect. Summary of the Invention

[0003] To solve the above problems, this application proposes a network content dissemination method and system based on the integration of cognitive understanding and intelligent governance, which can effectively improve the dissemination efficiency and quality of all-media content.

[0004] This application discloses a network content dissemination method based on the integration of cognitive understanding and intelligent governance, which includes:

[0005] Step 1: Decouple the data features of different modalities and output the decoupled data features; different modalities include text, video, audio, and images;

[0006] Step 2: According to the decoupled data features of audio, video, images, and text, construct a subgraph network for each modality, and perform convolutional operations on cross-modal features through multiple subgraph networks to generate cross-modal feature graph representations;

[0007] Step 3: Conduct topic modeling on the texts in the text knowledge base, extract potential topic words, extract event triples from the potential topic words, and construct an event graph; retrieve nodes related to hotspots from the event graph to form a hotspot pattern, match content features similar to the hotspot pattern in the cross-modal feature chart representation, and identify potential hotspot content;

[0008] Step 4: Combine the cross-modal feature chart representation to construct a cross-modal content graph, learn abnormal patterns using the labeled abnormal data, train an anomaly detection model according to the abnormal pattern, and identify abnormal content in the cross-modal content graph through the trained anomaly detection model; the abnormal content includes false information, sensitive images, and dense videos;

[0009] Step 5: Simulate the process of information dissemination among users, and predict the content dissemination link of hotspot content and the content dissemination link of abnormal content;

[0010] Step 6: Adjust the weights of the key nodes in the content dissemination link to determine the best time, location, and target audience for content placement.

[0011] Furthermore, the said Step 1 includes:

[0012] Step 11: Extract the data features of text, video, audio, and images and input them into the encoder, extract the features of the corresponding modalities, and use dynamic time warping to align the video features;

[0013] Step 12: Model the extracted cross-modal data features;

[0014] Step 13: Extract the decoupled cross-modal data features in the modeled cross-modal features through a method based on the attention mechanism.

[0015] Furthermore, the said Step 11 includes:

[0016] Extract text data features, video features, audio data features, and image data features from text, video, audio, and images respectively;

[0017] Calculate the distance between different time steps of the video feature and the audio data feature A to obtain the cumulative cost matrix:

[0018]

[0019] According to this distance, obtain the cost of the cumulative cost matrix through the following formula:

[0020]

[0021] where, Denote the distance between the i-th time step of the video and the j-th time step of the audio, Denote the feature of the i-th time step of the video, Denote the i-th element in Denote the feature of the j-th time step of the audio, Denote the j-th element in A, Denote the Euclidean distance, Denote the cumulative cost matrix, Refer to the minimum cumulative cost from the starting point to ;

[0022] From the end point of the cost in the cumulative cost matrix , that is, starting from the last time step of the video and audio, by backtracking to its starting point, the initial position D(0,0) of the cumulative cost matrix, calculate the optimal matching path P through the following formula:

[0023]

[0024]

[0025] where P represents an optimal path for the aligned time steps of the video and audio, represents the alignment of the last time steps of the video and audio, represents the total number of time steps of the video features, represents the total number of time steps of the audio features, is the point on the path, represents the video time step and the audio time step alignment relationship, represents choosing the path with the minimum cumulative cost, represents the path number, K represents the maximum value of the path number, represents the previous video time step and the previous audio time step cumulative cost to the current position, and at the same time considers the total number of time steps of the video and audio and as adjustment factors;

[0026] Through the optimal matching path P, the aligned video features are obtained.

[0027] Furthermore, the step 12 includes:

[0028] Integrate the data features extracted from all modalities by linear weighted summation to generate a weighted global data feature combination , the formula is:

[0029]

[0030] Among them, represents the global data feature after post-weighting combination, represents the feature vector decoded from the i-th modality, where i represents the modality type and m represents the number of modalities of cross-modal data features, represents the feature weight vector of, , and satisfies the constraint condition ;

[0031] By concatenating , the global and fine-grained features are retained, and the final cross-modal data feature vector representation is formed:

[0032]

[0033] Among them, X represents the vector of the final cross-modal data feature, and [·] is the concatenation operation;

[0034] The said step 13 includes:

[0035] The data feature vector X is feature decoupled by a method based on the attention mechanism to obtain the decoupled data feature.

[0036] Furthermore, the said step 2 includes:

[0037] Step 21: Use the cross-entropy loss function to optimize the node graph representation inside each sub-graph network;

[0038] Step 22: And use the KL divergence as the loss constraint condition between different sub-graph networks to quantify the similarity of node distributions between modalities, establish a similarity matrix of nodes between modalities, and calculate the total loss function between modalities;

[0039] Step 23: Integrate each sub-graph network according to the similarity matrix, and combine the total loss function and use graph convolution operations to integrate each sub-graph network into a global cross-modal feature graph representation.

[0040] Furthermore, the said step 21 includes:

[0041] Initialize the sub-graph network constructed for each modality i , and optimize it using the loss function and graph convolution operations ; The point set represents the nodes in modality i, and the edge set represents the connection relationship between the internal nodes of modality i, and is initially defined as the node similarity:

[0042]

[0043] Among them, is the cosine similarity, represents the feature vector of the $i$-th modality of the $j$-th node after cross-modal decoupling, represents the feature vector of the $i$-th modality of the $k$-th node after cross-modal decoupling, represents the cosine similarity of nodes $j$ and $k$ in the $i$-th modality;

[0044] Then, through the following formula, the cross-entropy loss function is used to optimize the node graph representation inside each sub-graph network:

[0045]

[0046]

[0047] The gradient descent method is used to optimize the node graph representation in each sub-graph network:

[0048]

[0049]

[0050] Among them, represents the normalized probability distribution of the $j$-th node in the $i$-th modality sub-graph network, represents the total number of nodes in the $i$-th modality sub-graph network, represents the node 's feature vector, $k$ represents the node number in the $i$-th modality sub-graph network, represents the exponential mapping, represents calculating the sum of the eigenvalues of all nodes in the $i$-th modality sub-graph network after exponential mapping, represents the total distribution loss inside the sub-graph network, $m$ represents the total number of modality types, and $\eta$ is the learning rate.

[0051] Furthermore, in step 22, the KL divergence between nodes of different modalities is obtained through the following formula:

[0052]

[0053] Among them, , represent the modality numbers, represents the KL divergence between node $j$ in modality and node $k$ in modality , represents the normalized probability distribution of node $j$ in modality , represents the normalized probability distribution of node $k$ in modality ;

[0054] Map to the corresponding matrix element to establish a similarity matrix :

[0055]

[0056] where represents the KL divergence between node j in modality and node k in modality , represents the KL divergence between node n in modality and node n in modality ;

[0057] Calculate the similarity loss between modalities through the following formula :

[0058]

[0059] Combine the similarity loss between modalities and the total distribution loss within the subgraph network to obtain the total loss function :

[0060]

[0061] where represents the similarity matrix of nodes between modalities, storing the KL divergence between node pairs, represents the KL divergence of node pair (j,k) between modalities, represents the KL divergence loss between modalities, represents the balance coefficient.

[0062] Furthermore, in step 23:

[0063] During the cross-modal association and fusion process, based on the similarity matrix judge the similarity of nodes between modalities and establish additional cross-modal connections on the basis of the original subgraph network:

[0064] where represents the adjacency relationship between node j in modality and node k in modality , is the set threshold;

[0065] For each subgraph Update by performing standard graph convolution operations through the following formula:

[0066]

[0067] Among them, is the normalized adjacency matrix, which is composed of the original connections within the modality and the cross-modal connections constrained by KL divergence; is the feature matrix of the L-th layer, is the trainable parameter of graph convolution, is the activation function;

[0068] During the optimization process, combined with the loss function and using the gradient descent algorithm, each subgraph network is integrated into a global cross-modal feature graph representation:

[0069]

[0070] Among them, represents the final cross-modal feature map, represents the subgraph network corresponding to modality i after optimization.

[0071] Furthermore, the step 3 includes:

[0072] Step 31: Combine the text knowledge base in the mainstream value vertical field, use the LDA model to perform topic modeling on the text in the text knowledge base, and extract potential topic words;

[0073] Step 32: Based on the extracted topic words, use the RoBERTa-CRF model to extract event triples, construct an event graph, and retrieve nodes related to hotspots from the event graph through a graph retrieval engine;

[0074] Step 33: Form a subgraph network with the relevant nodes and convert the subgraph network into a vector representation to form a hotspot pattern;

[0075] Step 34: By calculating the Jaccard similarity, match the content features similar to the hotspot pattern in the cross-modal feature graph representation obtained in step 2, and identify the potential hotspot content in the cross-modal feature graph.

[0076] Furthermore, in the step 31:

[0077] The following formula is used to implement topic modeling of the text in the text knowledge base using the LDA model:

[0078]

[0079] Among them, is the distribution of the term under the topic , the topic distribution of the document follows the Dirichlet prior, and the term The distribution under the theme constitutes a theme-word matrix.

[0080] Furthermore, step 32 includes:

[0081] Encoding the input text through the RoBERTa layer to obtain the vector representation of each word ;

[0082] Learning the sequence probability of word tags based on the vector representation , where is the tag sequence, and predicting the optimal tag sequence for each word according to the maximized conditional probability :

[0083]

[0084] where, is the normalization factor, is the weight of the transition feature, is the tag of the th word, is the tag of the i-th word, represents the index of the tag, is the exponential function, is the length of the word sequence, is the number of tag types;

[0085] Converting the event triple into an event graph to represent the association relationship of events in a graph structure; the subject and object in the event triple are used as nodes in the graph, and the predicate is used as the edge connecting the nodes:

[0086]

[0087] where, represents the event graph, is the node set, is the edge set;

[0088] Each entity pair in the event triple is connected by the predicate

[0089] Furthermore, step 33 includes:

[0090] For each node , is the node set, generating a node sequence of a fixed length as the context relationship of the event graph, where q is the walking length;

[0091] Perform embedding learning on the node sequence to maximize the co-occurrence probability between nodes and their contexts through the following formula:

[0092]

[0093] where is the set of context nodes of node ; is the probability of node generating context node ;

[0094] The embedding vector of each node is its representation in the d-dimensional vector space; for the entire subgraph , the overall vector representation of the subgraph is generated by aggregating the embedding vectors of all its nodes, and the subgraph vector is expressed as:

[0095]

[0096] where is the aggregation function, represents the subgraph vector, that is, the finally formed hot spot pattern.

[0097] Furthermore, the step 34 includes:

[0098] By calculating the Jaccard similarity between the subgraph vector and the cross-modal graph representation obtained in step 2, find the content features similar to the hot spot pattern in the obtained cross-modal graph representation, so as to identify potential hot spot content; by comparing the Jaccard similarity, identify potential hot spot content in the cross-modal feature map.

[0099] Furthermore, the construction of the cross-modal content graph by combining the cross-modal feature graph representation includes:

[0100] Combine the global feature graph representation obtained in step 2 , map the graph representations of all modalities to a high-dimensional semantic space; the dimension of this semantic space is d, then define a mapping function for the graph representation of each modality:

[0101]

[0102] The mapping function maps the graph representation of each modality to the high-dimensional semantic space, and the generated semantic vector is denoted as , where:

[0103]

[0104] Obtain the set of modal feature vectors after semantic alignment , that is, the cross-modal content graph.

[0105] Furthermore, the learning of the anomaly pattern using the labeled anomaly data and training the anomaly detection model according to the anomaly pattern includes:

[0106] Use graph embedding technology to semantically align the features of different modalities. The nodes of the cross-modal graph are , and each node corresponds to the feature of one modality;

[0107] By constructing a semantic similarity matrix :

[0108]

[0109] where represents the similarity between modality i and modality j, and the cosine similarity is used to calculate the similarity of each pair of modalities, and are the feature vectors of modality i and modality j mapped to the high-dimensional semantic space respectively;

[0110] The labeled anomaly data set is and use it to train the anomaly detection model: According to the semantic similarity matrix , use the graph neural network to detect the abnormal nodes in the cross-modal graph:

[0111]

[0112] where is the feature representation of node in the graph, the anomaly probability of node , indicating whether node is an abnormal node, represents the sigmoid activation function, represents the weight of the feature in the model, represents the parameter learned during the training process, represents the weighted sum of the feature of node and the features of its neighbor nodes

[0113] When an anomaly detection model is trained, the anomaly detection loss function can be defined by the following formula :

[0114]

[0115] Obtain the loss function After that, optimize the anomaly detection model by minimizing to optimize the anomaly detection model.

[0116] Furthermore, the process of identifying abnormal content in the cross-modal content graph through the trained anomaly detection model includes:

[0117] Calculate the anomaly probability of all nodes in the modal content graph, screen abnormal nodes according to a preset threshold, and process or alarm the corresponding modal content; abnormal nodes identified by the graph neural network model.

[0118] Furthermore, step 5 includes:

[0119] Step 51: Construct a SKIR model to simulate the propagation process of information among users;

[0120] Step 52: Adopt a two-factor coupled long-tail information cascade method to predict the propagation link of hot content and the propagation link of abnormal content.

[0121] Furthermore, step 51 includes:

[0122] Construct a SKIR model and introduce an informed state to expand the propagation mechanism of the model: Definition of the SKIR model: unknown information state, informed state, propagation state, unaware state:

[0123]

[0124]

[0125]

[0126]

[0127] Among them, represents the number of individuals in the unknown information state at time t, represents the number of individuals in the informed state at time t, represents the number of users in the propagation state at time t, represents the number of individuals in the unaware state at time t, represents how the number of users in the unknown information state at time t changes over time, represents how the number of informed users at time t changes over time, represents the change in the number of users in the propagation state at time t, represents the dynamic change in the number of users in the unaware state at time t, is the decay rate corresponding to different states, is the propagation intensity, is the diffusion coefficient related to the informed state, which controls the propagation rate of information from the informed state to the spreading state. is the transition rate from the informed state to the unaware state, indicating that informed users stop spreading due to loss of interest in the information. is the hesitant informed rate, representing the proportion of informed users who stop spreading due to reasons such as suspicion of the information.

[0128] Through the stability condition, it is judged whether the SKIR model reaches an equilibrium state and whether information propagation will stop; the stability condition is defined by the following formula:

[0129]

[0130] where represents the stability of the equilibrium point of the SKIR model. When Δ < 0, the model reaches equilibrium and stops information propagation. is the resistance coefficient of information propagation.

[0131] Furthermore, the said step 52 includes:

[0132] Optimize the sample selection of the model through long-tail distribution sampling and class-balanced sampling. The calculation formula for the sampling probability is:

[0133]

[0134]

[0135] And obtain the mixed sampling probability by weighted combination of long-tail distribution sampling and class-balanced sampling:

[0136]

[0137] where is the long-tail distribution sampling probability of node j. is the eigenvalue of node j. is the total number of the i-th node, R is the sampling parameter, and C represents the number of classes. is the probability of class-balanced sampling. The sampling probability of each class is equal and is , is the mixed sampling probability, and e is the balance factor used to adjust the weights of different sampling strategies.

[0138] The calculation formula for the global propagation intensity is:

[0139]

[0140]

[0141] where represents the propagation intensity at time t is the global propagation parameter is the time of the current event propagation is the time of the previous propagation event, and δ is the global attenuation factor is the sampling probability calculated through the sampling strategy, reflecting the influence of the sampling probability on the propagation intensity of different nodes is the prediction of the propagation intensity within the local time window and is a function calculated from the time difference or other input features, used to represent the influence degree of a node during the propagation process and are the weighting coefficients of the global and local propagation models affect the calculation of the local propagation intensity

[0142] Combined with the stability condition of the SKIR model , the final integrated global and local propagation intensity is obtained

[0143]

[0144] where is the final content propagation link, that is, the final propagation intensity is the propagation intensity from different modules is the weighting coefficient of each module is the stability adjustment factor, used to control the influence of the stability condition on the propagation link intensity

[0145] Furthermore, step 6 includes

[0146] Step 61: Based on the SKIR model framework, introduce the dynamic adjustment factor and the node weight , construct a control equation, and control the state transition during the information propagation process through the adjustment factor and the node weight. The state transition includes the probability changes of the susceptible state, the informed state, and the propagation state

[0147] Step 62: Monitor and update the parameters of the propagation link in step 5, and use the Kalman filter algorithm to update the model parameters. The updated model parameters include state prediction and observation correction

[0148] Step 63: Adjust the weights of the key nodes, and calculate the node adjustment weights based on the node betweenness centrality and the real-time influence index

[0149] Step 64: Optimize the propagation direction and rate, and establish an optimal control problem with the goal of maximizing the target content coverage

[0150] Step 65: Generate a dynamic distribution strategy, input the optimized parameters, divide users into multiple groups through the spectral clustering algorithm, define the group sensitivity, and calculate the optimal delivery time window.

[0151] Further, in the step 61, based on the SKIR model framework, a dynamic adjustment factor and node weights are introduced to construct a control equation, including:

[0152]

[0153] Among them, is the neighbor set of node , is the dynamic diffusion adjustment factor, ranging from [0, 1], which is used to control the information penetration intensity between nodes. is the propagation weight of node , ranging from [0, 2]. By adjusting , the propagation influence of the node can be suppressed or amplified. represents the rate of change from the unknown information state to the propagation state. is the propagation diffusion coefficient related to the informed state, which controls the propagation rate of information from the informed state to the propagation state. is the transition rate from the informed state to the unaware state, indicating that informed users stop spreading due to losing interest in the information.

[0154] Further, the step 62 includes:

[0155] State prediction is based on historical parameters to predict the current parameter :

[0156]

[0157] Among them, is the state transition matrix, is the control input matrix, is the external intervention signal, such as an artificial regulation instruction, is the process noise, is the final propagation link output in step 5;

[0158] Observation correction corrects the parameters through real-time observation data :

[0159]

[0160] Among them, is the observation matrix, is the Kalman gain matrix, Indicates the updated state prediction parameter.

[0161] Further, the step 63 includes:

[0162] Based on the node betweenness centrality and the real-time influence index , calculate the node adjustment weight:

[0163]

[0164] Wherein, is the learning rate, which is used to control the weight update amplitude. Finally, output the adjusted set of node weights .

[0165] Further, the step 64 includes:

[0166] Optimize the propagation direction and rate to maximize the target content coverage as the goal, and establish an optimal control problem:

[0167]

[0168] Wherein, is the delivery period, is the weight regularization coefficient. Solve the Hamiltonian function using the Pontryagin maximum principle, and derive the optimal control law:

[0169]

[0170] Wherein, is the co-state variable, represents the optimal output weight.

[0171] Further, the step 65 includes:

[0172] By inputting the optimized parameters , use the spectral clustering algorithm to divide users into groups , define the group sensitivity , and calculate the best delivery time window:

[0173]

[0174] Wherein, represents the best delivery time window, that is, the best content delivery time for the group, is the attenuation coefficient, represents the average value of the number of users in the propagation state at time s, and the triple represents the group at time With weights Deliver content.

[0175] This application also discloses a network content dissemination system based on integrated cognitive understanding and intelligent governance, which implements the network content dissemination method based on integrated cognitive understanding and intelligent governance described above. It includes:

[0176] A decoupling module for decoupling data features of different modalities and outputting the decoupled data features; different modalities include text, video, audio, and images;

[0177] A generation module for constructing a subgraph network for each modality based on the decoupled data features of audio, video, images, and text, and performing convolutional operations on cross-modal features through multiple subgraph networks to generate cross-modal feature graph representations;

[0178] A hot content recognition module for performing topic modeling on the text in the text knowledge base, extracting potential topic words, extracting event triples from the potential topic words, and constructing an event graph; retrieving nodes related to hotspots from the event graph to form a hot pattern, and matching content features similar to the hot pattern in the cross-modal feature graph representation to identify potential hot content;

[0179] An abnormal content recognition module for constructing a cross-modal content graph in combination with the cross-modal feature graph representation, learning an abnormal pattern using the labeled abnormal data, training an abnormal detection model according to the abnormal pattern, and identifying abnormal content in the cross-modal content graph through the trained abnormal detection model; abnormal content includes false information, sensitive images, and dense videos;

[0180] A prediction module for simulating the process of information dissemination among users and predicting the content dissemination links of hot content and abnormal content;

[0181] A determination module for adjusting the weights of key nodes in the content dissemination link and determining the best time, location, and target audience for content delivery.

[0182] Due to the adoption of the above technical solutions, this application has the following advantages:

[0183] 1. This application realizes the unified expression of multi-modal information, breaks through the mapping problem of traditional image data to high-dimensional representations in a unified space, and effectively improves the fusion ability of heterogeneous information under complex networks.

[0184] 2. The accurate perception technology for abnormal content improves the accuracy of content dissemination by accurately identifying potential hot content and abnormal information, and successfully improves the accuracy of content dissemination detection to 91.2% and 88.0%.

[0185] 3. The content distribution technology based on propagation prediction, combined with the SKIR information propagation method, realizes accurate prediction of the content propagation path and adjustment of the propagation strategy, and the prediction accuracy rate is increased by 13.5% compared with the prior art.

[0186] 4. The adaptive dynamic propagation control and optimization technology regulates the distribution of the propagation budget by optimizing the node weights, realizes the accurate content placement strategy, significantly improves the efficiency and coverage of information propagation, and ensures that the content can be propagated among the most suitable time, location and target audience. BRIEF DESCRIPTION OF THE DRAWINGS

[0187] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required to be used in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings.

[0188] Figure 1 It is a schematic flowchart of a network content propagation method based on fusion cognitive understanding and intelligent governance according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0189] The present application will be further described in conjunction with the drawings and embodiments. The described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art shall fall within the scope of protection of the embodiments of the present application.

[0190] The present application realizes the collaborative processing and deep fusion of different modal data such as audio, video, images and texts, effectively solves the inconsistency of information structures between modalities, uses the potential hot spot and abnormal content accurate perception technology to quickly identify hot topics and abnormal information in the network, provides strong support for public opinion monitoring and guidance, and through the network content propagation intelligent prediction and distribution guidance technology, accurately predicts the information propagation path and effect, optimizes the content distribution strategy, realizes accurate personalized recommendation and maximization of the propagation effect, and improves the intelligent level of all-media content propagation.

[0191] See Figure 1 , an embodiment of the present application provides a network content propagation method based on fusion cognitive understanding and intelligent governance, which includes S1 to S6:

[0192] S1: Extract the data features of text, video, audio, and images, that is: process the text data to extract its semantic information and context correlation features; for video data, extract key frame image features and temporal dynamic information; extract key information such as spectral features, pitch, and duration from audio data; extract visual features such as color, texture, and edges from image data, and then input them into a method based on the attention mechanism to decouple the data features of different modalities (i.e., text, video, audio, and images), and output the decoupled data features.

[0193] S2: Construct a subgraph network for each modality with the decoupled data features of audio, video, images, and text output in S1, and perform cross-modal feature convolution operations through multiple independent subgraph networks to generate cross-modal feature graph representations.

[0194] S3: Combine the text knowledge base in the mainstream value vertical field. This knowledge base collects and organizes data such as hot information, topic words, and event triples in different fields. Perform topic modeling on the text in the text knowledge base through the LDA model to extract potential topic words, and use the RoBERTa-CRF model to extract event triples from the potential topic words, and then construct an event graph. Retrieve nodes related to the hot topic from the event graph through the graph retrieval engine. Convert the subgraph composed of relevant nodes into a vector representation through graph embedding technology to form a hot topic pattern (i.e., a structured feature representation extracted from the event graph and related to the current hot event). By calculating the Jaccard similarity, match the content features similar to the hot topic pattern in the cross-modal feature graph representation obtained in S2 to identify potential hot content.

[0195] S4: Combine the cross-modal feature graph representation obtained in S2 to construct a cross-modal content graph. Use the labeled abnormal data to learn the abnormal pattern, optimize the anomaly detection model according to this abnormal pattern, and identify the abnormal content in the cross-modal content graph through the trained anomaly detection model; the abnormal content includes false information, sensitive images, and dense videos.

[0196] S5: Introduce the informed state (K) based on the classical propagation model SIR and the mean field theory, and propose the SKIR model. Simulate the information propagation process among users through the SKIR model, and at the same time adopt the long-tail information cascade modeling and prediction method with two-factor coupling to achieve high-precision prediction of the propagation links of the hot content and abnormal content identified in S3 and S4.

[0197] S6: Monitor and dynamically adjust the content propagation link obtained in S5 in real time through the dynamic propagation control equation. Control the propagation direction and speed by adjusting the weights of the key nodes in the content propagation link, and optimize the content propagation strategy at the same time, that is, determine the best time, location, and target audience for content placement. Realize the distribution and guidance of the hot content and abnormal content identified in S3 and S4.

[0198] Optionally, S1 includes S11 to S13:

[0199] S11: Extract the data features of text, video, audio, and images and input them into the encoder, extract the features of the corresponding modalities, and use dynamic time warping (DTW) to align the video features;

[0200] S12: Model the extracted cross-modal data features to obtain a unified representation;

[0201] S13: Extract the decoupled cross-modal data features in the modeled cross-modal features through an attention mechanism-based method.

[0202] Optionally, S11 includes:

[0203] For text data feature extraction, pre-trained language models such as BERT or GPT are usually used to convert the input text data into high-dimensional feature vectors, enabling it to effectively capture the semantic information of the text;

[0204] For image data feature extraction, a Transformer architecture image encoder with contrastive learning and fine-grained and coarse-grained fusion is used to extract its feature vectors;

[0205] For audio data feature extraction, Mel spectrogram or MFCC (Mel Frequency Cepstral Coefficients) is used to obtain audio feature vectors through their corresponding encoders;

[0206] For video feature extraction, a Transformer architecture image encoder with contrastive learning and fine-grained and coarse-grained fusion is used to encode each frame of the video in the multi-modal data, and then clustering is performed to form the video feature vectors of the key frames. Since the video usually contains audio features, it is necessary to perform feature extraction separately first and then perform dynamic time warping (DTW);

[0207] DTW calculates the distance between different time steps of the video feature and the audio data feature A to obtain the cumulative cost matrix:

[0208]

[0209] According to this distance, the cost of the cumulative cost matrix is obtained through the following formula:

[0210]

[0211] Among them, represents the distance between the i-th time step of the video and the j-th time step of the audio, represents the feature of the i-th time step of the video, represents the i-th element in represents the feature of the j-th time step of the audio, represents the j-th element in A, represents the Euclidean distance, represents the cumulative cost matrix, refers to the minimum cumulative cost from the starting point to .

[0212] From the end point of the cumulative cost matrix cost, that is, starting from the last time step of the video and audio, by backtracking to its starting point, the initial position D(0, 0) of the cumulative cost matrix, that is, the first time step of the video and audio, the optimal matching path P is calculated through the following formula:

[0213]

[0214]

[0215] Among them, P represents an optimal path for aligning the time steps of the video and audio, represents the alignment of the last time steps of the video and audio, represents the total number of time steps of the video features, represents the total number of time steps of the audio features, is the point on the path, represents the video time step and the audio time step alignment relationship, represents selecting the path with the minimum cumulative cost, represents the path number, K represents the maximum value of the path number, represents the previous video time step and the previous audio time step to the cumulative cost of the current position, and at the same time considering the total number of time steps of the video and audio and as adjustment factors.

[0216] By obtaining the optimal matching path P, the aligned video features can be obtained.

[0217] Optionally, S12 includes:

[0218] After extracting the data feature information of each single modality, the data features extracted from all modalities are integrated by linear weighted summation to generate a weighted global data feature combination , the formula is:

[0219]

[0220] Among them, represents the global data feature after post-weighting combination, represents the feature vector decoded from the i-th modality, i represents the modality type, and m represents the number of modalities of cross-modal data features, represents the feature weight vector of, , and satisfies the constraint condition ;

[0221] By concatenating , the global and fine-grained features are retained, and the final cross-modal data feature vector representation is formed:

[0222]

[0223] Among them, X represents the vector of the final cross-modal data feature, and [·] is the concatenation operation;

[0224] S13 includes:

[0225] The data feature vector X after modeling the input features is used, and the data feature vector X is feature decoupled by a method based on the attention mechanism. The implementation process of the method based on the attention mechanism is as follows:

[0226] First, X is linearly transformed to obtain the total query vector Q, key vector K, and value vector V, and then the attention weight matrix is calculated using the attention mechanism:

[0227]

[0228] Among them, represents the attention weight calculation result, represents the normalization function, represents the dimension of the key vector K, T represents the transpose calculation, represents the dot product calculation of the transpose of the query and the key;

[0229] Through the multi-head attention mechanism, parallel calculations are performed to independently calculate the weighted output of each attention head:

[0230]

[0231] Among them, represents the attention output of the i-th head;

[0232] Concatenate the outputs of multiple heads and obtain the final multi-head output through a linear transformation:

[0233]

[0234] Among them, represents the final multi-head output, represents the weight matrix of the linear transformation;

[0235] The final output is obtained through the multi-head output and the linear transformation:

[0236]

[0237] Among them, represents the output after the attention mechanism method, that is, the decoupled data features, which only contain the independent representations of each modality and do not include the global data features , in order to avoid information redundancy, which can be expressed as:

[0238]

[0239] Among them, represents the feature vector of the i-th modality after being decoupled by the attention mechanism, i represents the modality type, and m represents the number of modalities of the cross-modal data features.

[0240] Optionally, S2 includes S21 to S23:

[0241] S21: Use the cross-entropy loss function to optimize the node graph representation inside each sub-graph network;

[0242] S22: And adopt the KL divergence as the loss constraint condition between different sub-graph networks, so as to quantify the similarity of the node distributions between modalities and establish a similarity matrix of nodes between modalities , and calculate the total loss function between modalities;

[0243] S23: According to the similarity matrix Fuse each sub-graph network, and combine the total loss function and use graph convolution operations to integrate each sub-graph network into a global cross-modal feature graph representation to comprehensively capture the semantic associations and interactions between multi-modal contents.

[0244] Optionally, S21 includes:

[0245] First, the data features of the input S1 after cross-modal decoupling , m represents the total number of modalities, and initialize the sub-graph network constructed by each modality i , then optimize through the use of the loss function and graph convolution operations .

[0246] Among them, the point set represents the nodes in modality i, represents the th node in modality i, and the edge set represents the connection relationship between the internal nodes in modality i, and is initially defined as the node similarity:

[0247]

[0248] is the cosine similarity, represents the feature vector of the i-th modality of the j-th node after cross-modal decoupling, represents the feature vector of the i-th modality of the k-th node after cross-modal decoupling, represents the cosine similarity between nodes j and k in the i-th modality.

[0249] Then, through the following formula and using the cross-entropy loss function, the node graph representation inside each subgraph network is optimized:

[0250]

[0251]

[0252] Optimize the node graph representation in each subgraph network using the gradient descent method:

[0253]

[0254]

[0255] Among them, represents the normalized probability distribution of the j-th node in the i-th modality subgraph network, ni represents the total number of nodes in the i-th modality subgraph network, represents the node 's feature vector, k represents the node number in the i-th modality subgraph network, represents the exponential mapping, represents the sum of the eigenvalues of all nodes in the i-th modality subgraph network after exponential mapping, represents the total distribution loss inside the subgraph network, m represents the total number of modality types, η is the learning rate, and by continuously iterating to adjust the node feature vectors, the goal of minimizing the cross-entropy loss is finally achieved, thereby optimizing the node graph representation in each subgraph network.

[0256] Optionally, in S22, in the said step 22, the KL divergence between nodes of different modalities is obtained through the following formula:

[0257]

[0258] Among them, 、 represents the modal number, represents the mode The KL divergence between node j and mode and node k in represents the normalized probability distribution of node j in mode ; represents the normalized probability distribution of node k in mode ;

[0259] For unified representation, map to the corresponding matrix element (that is, form the similarity matrix from ), and establish the similarity matrix :

[0260]

[0261] Among them, represents the KL divergence between node j and mode and node k in mode ; represents the KL divergence between node n and mode and node n in mode ;

[0262] Calculate the similarity loss between modes through the following formula :

[0263]

[0264] Combine the similarity loss between modes and the total distribution loss within the subgraph network to obtain the total loss function :

[0265]

[0266] Among them, represents the similarity matrix of nodes between modes, storing the KL divergence between node pairs, represents the KL divergence of node pair (j,k) between modes, represents the KL divergence loss between modes, represents the balance coefficient.

[0267] Optionally, in S23,

[0268] During the cross-modal association and fusion process, first based on the similarity matrix Judge the similarity between cross-modal nodes, and based on this, establish additional cross-modal connections on the basis of the original subgraph network. Specifically:

[0269] Among them, represents the adjacency relationship between node j in modality and node k in modality , that is, whether there is a cross-modal edge. The set threshold. If the KL divergence is less than , it is considered that these two nodes are similar enough to establish a connection in the graph, otherwise they are not connected.

[0270] Then introduce graph convolution operations so that the intra-modal and cross-modal features can be synchronously optimized during information propagation. For each subgraph Update using the standard graph convolution operation:

[0271]

[0272] Among them, is the normalized adjacency matrix, which is composed of the original intra-modal connections and the cross-modal connections constrained by the KL divergence. Among them, the cross-modal edges are only added when the KL divergence is less than the threshold . is the feature matrix of the L-th layer, is the trainable parameter of graph convolution, is the activation function. In this way, each subgraph can not only maintain its own structural information, but also realize the mutual propagation of features through cross-modal edges, thereby enhancing the feature consistency between modalities.

[0273] At the same time, during the optimization process, it is also necessary to combine the calculated loss function and use the gradient descent algorithm again to integrate each subgraph network into a global cross-modal feature graph representation:

[0274]

[0275] Among them represents the final cross-modal feature graph, represents the subgraph network corresponding to modality i after optimization.

[0276] Optionally, S3 includes S31 to S34:

[0277] S31: Combine the text knowledge base in the mainstream value vertical field, use the LDA model to perform topic modeling on the text in the text knowledge base, and extract potential topic words;

[0278] S32: Based on the extracted keywords, the RoBERTa-CRF model is used to extract event triplets, build an event graph, and retrieve nodes related to hot spots from the event graph through the graph retrieval engine;

[0279] S33: Construct a subgraph network from related nodes, and convert the subgraph network into a vector representation to form a hotspot pattern;

[0280] S34: By calculating the Jaccard similarity, the content features similar to the hotspot mode are matched in the cross-modal feature graph representation obtained in S2, and the potential hotspot content in the cross-modal feature graph is identified.

[0281] Optionally, S31 includes:

[0282] Combined with the hot information in the mainstream value vertical domain knowledge base, the LDA model is used to perform topic modeling on the data in the text knowledge base and extract the key words; the LDA topic modeling process is formalized as follows:

[0283]

[0284] in, For terms In Theme Distribution under, documentation The topic distribution Subject to the Dirichlet prior, the term In Theme The distribution under Construct a topic-word matrix;

[0285] For each document, Select a topic distribution , for each word in the document , first select a topic from the topic distribution of the document , and then select words from the word distribution corresponding to the topic ;in, Is the theme The word distribution of is a dimensional vector, is the number of topics; is a dimensional vector, is the size of the vocabulary;

[0286] In mathematical expression, given Topics and documents ,document The generation probability of the word for:

[0287]

[0288] wherein, represents the probability of the word under the topic ; represents the probability of the topic in the document ; is a topic-word distribution, is a document-topic distribution, represents the number of words in the document .

[0289] Optionally, S32 includes:

[0290] encoding the input text through the RoBERTa layer to obtain the vector representation of each word ;

[0291] learning the sequence probability of word labels based on the vector representation through the CRF layer , wherein is the label sequence, and the CRF layer predicts the optimal label sequence for each word by maximizing the conditional probability :

[0292]

[0293] wherein, is the normalization factor, is the weight of the transition feature, is the label of the th word, is the label of the i-th word, represents the index of the label; is the exponential function, is the length of the word sequence, is the number of label types.

[0294] Converting the event triple into an event graph to represent the association relationship of events in a graph structure; the subject and object in the event triple are used as nodes in the graph, and the predicate is used as the edge connecting the nodes:

[0295]

[0296] wherein, represents the event graph, is the node set, is the edge set;

[0297] Each entity pair in the event triple are connected, and the expression methods include:

[0298]

[0299] Among them, is the set of all possible predicate relations.

[0300] Optionally, S33 includes:

[0301] For each node , is the node set, generating a node sequence with a fixed length , as the context relationship of the event graph, where q is the random walk length;

[0302] Through the Skip-Gram model, embedding learning is performed on the node sequence to maximize the co-occurrence probability between the node and its context:

[0303]

[0304] Among them, is the context node set of the node , is the probability of the node generating the context node .

[0305] The transition probability of the random walk method is:

[0306]

[0307] Among them, is the transition weight from the node to , is the normalization constant; the Node2Vec model controls the depth-first and breadth-first tendencies of the random walk by introducing two hyperparameters and , making the transition weight defined as:

[0308]

[0309] Among them, represents the shortest path distance between the nodes and in the graph; the hyperparameter controls the probability of returning to the node, and the hyperparameter controls the probability of exploring new nodes, thereby achieving a balanced modeling of the local and global structures of the graph.

[0310] The node Probability of generating context nodes is , which is represented by the Softmax function as:

[0311]

[0312] where represents the dot product of the embedding vectors of nodes and , which is used to measure similarity; if negative sampling is used for optimization, the objective function is approximated as:

[0313]

[0314] where is the Sigmoid function, and there is ; is the negative sampling distribution, is the number of negative samples; represents the mathematical expectation operation, is an element sampled from the negative sampling distribution ;

[0315] After training, the embedding vector of each node is its representation in the d-dimensional vector space; for the entire subgraph , the overall vector representation of the subgraph can be generated by aggregating the embedding vectors of all its nodes, and the aggregation methods include mean calculation, weighted summation, and graph pooling-based methods; finally, the subgraph vector is represented as:

[0316]

[0317] where is the aggregation function, represents the subgraph vector, that is, the finally formed hot spot pattern.

[0318] Optionally, S34 includes:

[0319] By calculating the Jaccard similarity between the subgraph vector and the cross-modal graph representation obtained from S2, find the content features similar to the hot spot pattern in the obtained cross-modal graph representation, so as to identify potential hot content; the calculation formula of the Jaccard similarity is:

[0320]

[0321] where represents the cross-modal graph representation obtained from S2, B represents the set of hot spot pattern features to be compared, from , from the subgraph vector Extract features from is and the number of elements in the intersection, while is and the number of elements in the union; the value range of the Jaccard similarity is between 0 and 1, and the larger the value, the higher the similarity between the two sets;

[0322] Finally, by comparing similarities, if it exceeds the threshold , the content will be recognized as potential hot content.

[0323] Optionally, S4 includes S41 to S43:

[0324] S41: Construct a cross-modal content graph based on the cross-modal feature map obtained in S2;

[0325] S42: Use graph embedding technology for semantic alignment, and use the labeled abnormal data set to learn potential abnormal patterns, and then optimize the abnormal detection model;

[0326] S43: Use the trained abnormal detection model to perform abnormal content detection.

[0327] Optionally, S41 includes:

[0328] In S2, the global feature map representation has been obtained. Then, map the graph representations of all modalities to a high-dimensional semantic space. The dimension of this semantic space is d, so for the graph representation of each modality, define a mapping function

[0329]

[0330] This function maps the graph representation of each modality to the high-dimensional semantic space, and the generated semantic vector is denoted as , where:

[0331]

[0332] The graph representations of all modalities will be aligned in the high-dimensional semantic space through this process to obtain a set of modality feature vectors after semantic alignment, that is, the cross-modal content graph.

[0333] Optionally, S42 includes:

[0334] Use graph embedding technology to further align the features of different modalities. The nodes of the cross-modal graph are , these nodes represent different entities or events, and each node corresponds to a specific modal feature. The goal of graph embedding is to capture the semantic relationships between modalities by calculating the similarity between modal features. Specifically, by constructing a semantic similarity matrix S:

[0335]

[0336] where represents the similarity between modality i and modality j, and the cosine similarity is used to calculate the similarity of each pair of modalities. are the feature vectors of modality i and modality j mapped to the high-dimensional semantic space, respectively.

[0337] Using the labeled abnormal data, learn the potential abnormal patterns. The labeled abnormal data set is , which contains the feature representations of abnormal patterns. Use this labeled data to train the anomaly detection model. First, the model uses a graph neural network (GNN) to detect abnormal nodes in the cross-modal graph according to the semantic similarity matrix S:

[0338]

[0339] where is the feature representation of node in the graph, the abnormal probability of node , indicating whether node is an abnormal node. Through the calculation of the graph neural network, the output value will be between [0,1], representing the probability of abnormality. represents the sigmoid activation function, represents the weight of the feature in the model, usually represents the parameters learned during the training process, used to adjust the model output. represents the weighted sum of the feature of node and the features of its neighbor nodes .

[0340] Finally, define the anomaly detection loss function in the following way :

[0341]

[0342] Get the loss function After that, the anomaly detection model can be optimized by minimizing .

[0343] Optionally, S43 includes:

[0344] Through the trained anomaly detection model, potential anomaly nodes in the cross-modal content graph can be marked as anomalous content. These anomaly nodes may correspond to anomalous content such as false information, sensitive images, or dense videos. The detection result of the anomaly nodes can be achieved through the following process:

[0345] 1) Calculate the anomaly probability of all nodes in the modal content graph.

[0346] 2) Filter the anomaly nodes according to a preset threshold Let , then the node is determined to be an anomaly node.

[0347] 3) Mark the anomaly nodes and perform further processing or alarm on the corresponding modal content.

[0348] Finally, the anomaly nodes identified by the graph neural network model are anomalous content such as false information, sensitive images, and dense videos.

[0349] Optionally, S5 includes S51 and S52:

[0350] S51: Construct a SKIR model to simulate the propagation process of information among users;

[0351] S52: Adopt a two-factor coupled long-tail information cascade method for high-precision prediction of the propagation links of hot content and anomalous content.

[0352] Optionally, S51 includes:

[0353] Construct a SKIR model. The SKIR model is based on the classical SIR (Susceptible-Informed-Transmitted) model and introduces the informed state (K) to expand the propagation mechanism of the model. The specific construction process is as follows: The model defines four states: unknown information state (S), informed state (K), transmitted state (I), and unaware state (R):

[0354]

[0355]

[0356]

[0357]

[0358] Among them represents the number of individuals in the unknown information state at time t, represents the number of individuals in the informed state at time t, represents the number of users in the transmitted state at time t, The number of individuals indicating that the user is in an unaware state at time t Indicates how the number of users in an unknown information state at time t changes over time Indicates how the number of informed users at time t changes over time Indicates the change in the number of users in the dissemination state at time t Indicates the dynamic change in the number of unaware state users at time t Is the decay rate corresponding to different states Is the dissemination intensity Is the dissemination diffusion coefficient related to the informed state, controlling the dissemination rate of information from the informed state to the dissemination state Is the transition rate from the informed state to the unaware state, indicating that informed users stop disseminating due to losing interest in the information Is the hesitant informed rate, indicating the proportion of informed users who stop disseminating due to reasons such as doubting the information

[0359] Then, by setting the stability condition, it is judged whether the SKIR model reaches an equilibrium state and whether information dissemination will stop. This stability condition is defined by the following formula

[0360]

[0361] In the formula Indicates the stability of the equilibrium point of the SKIR model. When Δ < 0, the model reaches equilibrium and stops information dissemination Is the resistance coefficient of information dissemination. By adjusting these dissemination parameters, the scope and speed of information dissemination can be regulated, thereby precisely intervening in the diffusion direction of information in the system

[0362] Through these precise formula and parameter regulations, the SKIR model can capture the fine-grained dynamic characteristics of information dissemination among users, improve the interpretability of the dissemination path, and at the same time provide strong support for dissemination prediction. The dissemination parameters in the model determine the scope and speed of information dissemination. Further, by adjusting these parameters, it is possible to accurately predict and intervene in the diffusion direction of the unknown state during the information dissemination process

[0363] Optionally, S52 includes

[0364] Combining the long-tail information cascade modeling and prediction method with two-factor coupling to predict the dissemination link. The following is the specific implementation process of the two-factor coupling long-tail information cascade method

[0365] First, optimize the sample selection of the model through long-tail distribution sampling and class-balanced sampling. The calculation formula for the sampling probability is

[0366]

[0367]

[0368] And through weighted combination of long-tail distribution sampling and class-balanced sampling, the mixed sampling probability is obtained:

[0369]

[0370] Among them, is the long-tail distribution sampling probability of node j, is the eigenvalue of node j, is the total number of the i-th node, R is the sampling parameter, C represents the number of classes, is the probability of class-balanced sampling, and the sampling probability of each class is equal and is , is the mixed sampling probability, and e is the balance factor used to adjust the weights of different sampling strategies.

[0371] Secondly, the propagation intensity is closely related to the sampling probability. The model captures the information diffusion relationship in the propagation process through the global dependence module and the local dependence module. The calculation formula for the global propagation intensity is:

[0372]

[0373]

[0374] Among them, represents the propagation intensity at time t, is the global propagation parameter, is the time of the current event propagation, is the time of the previous propagation event, and δ is the global decay factor, is the sampling probability calculated through the sampling strategy, reflecting the influence of the sampling probability on the propagation intensity of different nodes; is the prediction of the propagation intensity within the local time window, and are usually functions calculated from the time difference or other input features, used to represent the influence degree of the node in the propagation process, and are the weighted coefficients of the global and local propagation models, further affecting the calculation of the local propagation intensity and ensuring that the sampling probability is reflected in different modules.

[0375] Finally, combining the stability condition of the above SKIR model , the final integrated global and local propagation intensity is obtained:

[0376]

[0377] Among them, is the final content propagation link, that is, the final propagation intensity, is the propagation intensity from different modules, is the weighting coefficient of each module, is the stability adjustment factor, used to control the influence of the stability condition on the propagation link strength.

[0378] Optionally, S6 includes S61 to S65:

[0379] S61: Based on the SKIR model framework, introduce the dynamic adjustment factor and the node weight , construct the control equation, and control the state transition in the information propagation process through the adjustment factor and the node weight, including the probability changes of the susceptible state, the informed state, and the propagation state.

[0380] S62: Monitor and update the parameters of the propagation link of S5, and use the Kalman filter algorithm to update the model parameters, including state prediction and observation correction. The state prediction is based on the historical parameters to predict the current parameters.

[0381] S63: Perform key node weight adjustment, and calculate the node adjustment weight based on the node betweenness centrality and the real-time influence index.

[0382] S64: Optimize the propagation direction and rate, and establish an optimal control problem with the goal of maximizing the target content coverage rate.

[0383] S65: Generate a dynamic distribution strategy, input the optimized parameters, divide the users into multiple groups through the spectral clustering algorithm, define the group sensitivity, and calculate the best placement time window.

[0384] Optionally, S61 includes:

[0385] This equation is based on the SKIR model framework, and introduces the dynamic adjustment factor and the node weight , construct the dynamic propagation control equation, and the specific implementation process is as follows:

[0386]

[0387] Among them, is the neighbor set of node , is the dynamic diffusion adjustment factor, with a range between [0, 1], used to control the information penetration intensity between nodes. is the propagation weight of node , with a range between [0, 2], and by adjusting the propagation influence of the node can be suppressed or amplified. Represents the rate of change from the unknown information state to the propagation state, is the propagation diffusion coefficient related to the informed state, which controls the propagation rate of information from the informed state to the propagation state, is the transition rate from the informed state to the unaware state, indicating that informed users stop spreading due to losing interest in the information.

[0388] Optionally, S62 includes:

[0389] Monitor and update the parameters of the propagation link in S5 in real time, and use the Kalman filter algorithm to update the model parameters. State prediction is based on historical parameters to predict the current parameters :

[0390]

[0391] Among them, is the state transition matrix, is the control input matrix, is the external intervention signal, such as an artificial regulation instruction, is the process noise, is the final propagation link output by S5.

[0392] Observation correction is used to correct the parameters through real-time observation data :

[0393]

[0394] Among them, is the observation matrix, is the Kalman gain matrix, represents the updated state prediction parameters.

[0395] Optionally, S63 includes:

[0396] Based on the node betweenness centrality and the real-time influence index , calculate the node adjustment weight:

[0397]

[0398] Among them, is the learning rate, which is used to control the amplitude of weight update. Finally, the adjusted set of node weights is output.

[0399] Optionally, S64 includes:

[0400] Optimize the propagation direction and rate to maximize the target content coverage as the goal, and establish an optimal control problem:

[0401]

[0402] Among them, is the delivery period, is the weight regularization coefficient. The Hamiltonian function is solved using the Pontryagin maximum principle , and the optimal control law is derived:

[0403]

[0404] Among them, is the co-state variable, represents the optimal output weight.

[0405] Optionally, S65 includes:

[0406] Generate a dynamic distribution strategy, that is, determine the best time, location, and target audience for content delivery. By inputting the optimized parameters , use the spectral clustering algorithm to divide users into groups , define the group sensitivity , and calculate the best delivery time window:

[0407]

[0408] Among them, represents the best delivery time window, that is, the best content delivery time for the group, is the decay coefficient, represents the average value of the number of users in the propagation state at time s. The triple represents delivering content to group at time with weight .

[0409] The embodiment of the present application also provides a network content propagation system based on the integration of cognitive understanding and intelligent governance, which implements the network content propagation method based on the integration of cognitive understanding and intelligent governance described in the above embodiment. It includes:

[0410] A decoupling module for decoupling the data features of different modalities and outputting the decoupled data features; different modalities include text, video, audio, and images;

[0411] A generation module for constructing a subgraph network for each modality according to the decoupled data features of audio, video, images, and text, and performing convolutional operations on cross-modal features through multiple subgraph networks to generate cross-modal feature chart representations;

[0412] A hot content recognition module, which is used to perform topic modeling on the texts in the text knowledge base, extract potential topic words, extract event triples from the potential topic words, and construct an event graph; retrieve nodes related to the hot topic from the event graph, convert the subgraph composed of the relevant nodes into a vector representation through graph embedding technology to form a hot topic pattern, and match the content features similar to the hot topic pattern in the cross-modal feature chart representation to identify potential hot content;

[0413] An abnormal content recognition module, which is used to construct a cross-modal content graph in combination with the cross-modal feature chart representation, learn abnormal patterns using the labeled abnormal data, train an abnormal detection model according to the abnormal pattern, and identify the abnormal content in the cross-modal content graph through the trained abnormal detection model; the abnormal content includes false information, sensitive images and dense videos;

[0414] A prediction module, which is used to simulate the information dissemination process among users and predict the content dissemination link of the hot content and the content dissemination link of the abnormal content;

[0415] A determination module, which is used to obtain the dissemination direction and speed by adjusting the weights of the key nodes in the content dissemination link, and determine the best time, location and target audience for content delivery.

[0416] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the specific implementation manners of the present application can still be modified or equivalently replaced, and any modification or equivalent replacement without departing from the spirit and scope of the present application shall be covered by the protection scope of the claims of the present application.

Claims

1. A network content dissemination method based on integrated cognitive understanding and intelligent governance, characterized in that: include: Step 1: Decouple data features of different modalities and output the decoupled data features; different modalities include text, video, audio, and image; Step 2: Based on the decoupled data features of audio, video, image, and text, a subgraph network for each modality is constructed, and convolution operations on cross-modal features are performed through multiple subgraph networks to generate cross-modal feature graph representations; Step 3: Perform topic modeling on the text in the text knowledge base, extract potential topic words, and extract event triplets from the potential topic words to construct an event graph; retrieve nodes related to hot spots from the event graph to form hot spot patterns, match content features related to hot spot patterns in the cross-modal feature graph representation, and identify potential hot content; Step 4: Combine the cross-modal feature graph representation to build a cross-modal content map, use the labeled abnormal data to learn abnormal patterns, train an anomaly detection model based on the abnormal patterns, and identify abnormal content in the cross-modal content map through the trained anomaly detection model; abnormal content includes false information, sensitive images, and dense videos; Step 5: Simulate the process of information propagation among users and predict the content propagation links of hot content and abnormal content; Step 6: Adjust the weights of key nodes in the content dissemination chain to determine the time, location, and target audience for content delivery; The step 5 comprises: Step 51: Construct a SKIR model to simulate the process of information dissemination among users; Step 52: using a long-tail information cascade method with dual-factor coupling to predict the propagation links of hot content and the propagation links of abnormal content; The step 51 comprises: The SKIR model is constructed, and the informed state is introduced to expand the propagation mechanism of the model: SKIR model definition: unknown information state, informed state, propagation state, unaware state: in, represents the number of individuals whose information is unknown at time t, represents the number of individuals who are in the informed state at time t, It represents the number of users in the propagation state at time t. represents the number of individuals who are in an unaware state at time t, represents how the number of users with unknown information status at time t changes over time, represents how the number of informed users at time t changes over time, represents the change in the number of users in the propagation state at time t, Indicates the dynamic change of the number of users in the unaware state at time t, is the decay rate corresponding to different states, is the transmission intensity; is the propagation diffusion coefficient associated with the informed state, controlling the rate at which information propagates from the informed state to the propagation state; is the rate of transition from the informed state to the unaware state, indicating that informed users stop spreading information because they lose interest in it; is the hesitant informed rate; The stability condition is used to determine whether the SKIR model has reached a state of equilibrium and whether information propagation will stop. The stability condition is defined by the following formula: in, Indicates the stability of the equilibrium point of the SKIR model. When Δ<0, the model will reach equilibrium and stop information propagation. is the resistance coefficient of information dissemination; The step 52 comprises: The sample selection of the model is optimized by long-tail distribution sampling and class-balanced sampling. The calculation formula of the sampling probability is: And by weighted combination of long-tail distribution sampling and class-balanced sampling, we get the mixed sampling probability: in, is the long-tail distribution sampling probability of node j, is the eigenvalue of node j, is the total number of the i-th node, R is the sampling parameter, C represents the number of categories, is the probability of class-balanced sampling, where the sampling probability of each class is equal and is , is the mixed sampling probability, e is the balance factor, which is used to adjust the weights of different sampling strategies; The calculation formula of global propagation intensity is: in, represents the propagation intensity at time t, is the global propagation parameter, is the time at which the current event was propagated, is the time of the last propagation event, δ is the global decay factor, It is the sampling probability calculated by the sampling strategy, which reflects the influence of the propagation intensity of different nodes on the sampling probability; is the propagation intensity prediction within a local time window, and It is a function calculated from the time difference or other input features, which is used to indicate the influence of a node in the propagation process. and are the weighting coefficients of the global and local propagation models, Affects the calculation of local propagation intensity; Combining the stability conditions of the SKIR model , and the final fusion of global and local propagation strength is obtained: in, is the final content dissemination link, that is, the final dissemination intensity, is the propagation strength from different modules, is the weight coefficient of each module, It is a stability adjustment factor used to control the impact of stability conditions on the strength of the propagation link.

2. The network content dissemination method based on integrated cognitive understanding and intelligent governance according to claim 1 is characterized in that: The step 1 comprises: Step 11: Extract data features of text, video, audio, and image and input them into the encoder, extract features of the corresponding modality, and align video features using dynamic time warping; Step 12: Model the extracted cross-modal data features; Step 13: Extract the cross-modal decoupled data features from the modeled cross-modal features through an attention mechanism-based method.

3. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 2 is characterized in that: The step 11 comprises: Extract text data features, video features, audio data features and image data features from text, video, audio and image respectively; The video features are calculated by the following formula And the distance between different time steps of audio data feature A, we get the cumulative cost matrix: Based on this distance, the cumulative cost matrix cost is obtained by the following formula: in, Represents the distance between the i-th time step of the video and the j-th time step of the audio; yes The i-th element in represents the feature of the i-th time step of the video; is the jth element in A, representing the features of the jth time step of the audio; represents the Euclidean distance; represents the cumulative cost matrix, From the starting point arrive The minimum cumulative cost of From the end point of the cumulative cost matrix , that is, starting from the last time step of the video and audio, by backtracking to its starting point, accumulating the initial position D(0,0) of the cost matrix, and calculating the optimal matching path P by the following formula: Among them, P represents an optimal path for aligning the video and audio time steps, indicates that the last time step of video and audio is aligned, represents the total number of time steps of video features, represents the total number of time steps of audio features, is a point on the path, Represents the video time step and the audio time step The alignment relationship, Indicates choosing the path with the smallest cumulative cost. represents the path number, K represents the maximum path number, Represents the previous video time step and the previous audio time step The cumulative cost to the current position, taking into account the total time steps of video and audio and As an adjustment factor; Through the optimal matching path P, the aligned video features are obtained.

4. The network content dissemination method based on integrated cognitive understanding and intelligent governance according to claim 2 is characterized in that: The step 12 comprises: The data features extracted from all modes are integrated by linear weighted summation to generate a weighted global data feature combination. , the formula is: in, represents the global data features after post-weighted combination, represents the feature vector after decoding of the i-th modality, i represents the modality type, and m represents the number of modalities of the cross-modal data features. Representation characteristics The weight vector of , and satisfy the constraints ; By splicing , retaining global and fine-grained features and forming the final cross-modal data feature vector representation: Where X represents the vector of the final cross-modal data features, [·] is the concatenation operation; The step 13 comprises: The data feature vector X is decoupled by a method based on the attention mechanism to obtain the decoupled data features.

5. The network content dissemination method based on integrated cognitive understanding and intelligent governance according to claim 1 is characterized in that: The step 2 comprises: Step 21: Use the cross entropy loss function to optimize the node graph representation within each subgraph network; Step 22: KL divergence is used as the loss constraint condition between different subgraph networks to quantify the similarity of node distribution between modalities, establish the similarity matrix of nodes between modalities, and calculate the total loss function between modalities; Step 23: Fuse each subgraph network according to the similarity matrix, and combine the total loss function and use graph convolution operations to integrate each subgraph network into a global cross-modal feature map representation.

6. The network content dissemination method based on integrated cognitive understanding and intelligent governance according to claim 5 is characterized in that: The step 21 comprises: Initialize the subgraph network constructed for each modality i , using loss function and graph convolution operation to optimize ; Point set represents the nodes in mode i, Indicates the first Nodes, edge sets Represents the connection relationship between the internal nodes of modality i, which is initially defined as node similarity: in, is the cosine similarity, represents the eigenvector of the i-th mode of the j-th node after cross-modal decoupling, represents the eigenvector of the i-th mode of the k-th node after cross-modal decoupling, represents the cosine similarity between node j and node k in the i-th mode; Then the node graph representation inside each subgraph network is optimized using the cross entropy loss function through the following formula: Use gradient descent to optimize the node graph representation in each subgraph network: in, represents the normalized probability distribution of the jth node in the i-th modal subgraph network, represents the total number of nodes in the i-th modal subgraph network, Representation Node The characteristic vector of , k represents the node number in the i-th modal subgraph network, represents the exponential map, It means calculating the sum of the eigenvalues ​​of all nodes in the i-th modal subgraph network after exponential mapping. represents the total distribution loss within the subgraph network, m represents the total number of modality types, and η is the learning rate.

7. The network content dissemination method based on integrated cognitive understanding and intelligent governance according to claim 5 is characterized in that: In step 22, the KL divergence of nodes between different modes is obtained by the following formula: in, , Indicates the mode number, Representing modality Midpoint j and mode The KL divergence between nodes k, Representing modality The normalized probability distribution of node j in is, Representing modality Normalized probability distribution of node k; Will Mapped to the corresponding matrix elements , build a similarity matrix : in, Representing modality Midpoint j and mode The KL divergence between nodes k, Representing modality Midpoint n and mode KL divergence between nodes n; The similarity loss between modalities is calculated by the following formula : Combining similarity loss between modalities And the total distribution loss within the subgraph network , and get the total loss function : in, Represents the similarity matrix of nodes between modalities, storing the KL divergence between node pairs, represents the KL divergence of the node pair (j, k) between modalities, represents the KL divergence loss between modalities, Represents the balance coefficient.

8. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 5 is characterized in that: In step 23: In the process of inter-modal correlation fusion, based on the similarity matrix Determine the similarity of nodes between modalities and establish additional cross-modal connections based on the original subgraph network: in, Representing modality Node j and mode in The adjacency relationship between nodes k in Set thresholds; For each subgraph Update, perform standard graph convolution operation through the following formula: in, It is a normalized adjacency matrix, which is composed of the original connections within the modality and the cross-modal connections constrained by KL divergence; is the feature matrix of the Lth layer, are the trainable parameters of graph convolution, is the activation function; During the optimization process, the loss function is combined with the gradient descent algorithm to integrate each subgraph network into a global cross-modal feature graph representation: in, represents the final cross-modal feature map, Represents the subgraph network corresponding to modality i after optimization.

9. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 1 is characterized in that: The step 3 comprises: Step 31: Combine the mainstream value vertical field text knowledge base, use the LDA model to perform topic modeling on the text in the text knowledge base, and extract potential topic words; Step 32: Based on the extracted keywords, the RoBERTa-CRF model is used to extract event triplets, an event graph is constructed, and nodes related to hot spots are retrieved from the event graph through a graph retrieval engine; Step 33: Construct a subgraph network from related nodes, and convert the subgraph network into a vector representation to form a hotspot pattern; Step 34: By calculating the Jaccard similarity, the content features similar to the hotspot mode are matched in the cross-modal feature graph representation obtained in step 2 to identify potential hotspot content in the cross-modal feature graph.

10. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 9 is characterized in that: In step 31: The following formula is used to implement topic modeling of texts in the text knowledge base using the LDA model: in, For terms On topic Distribution under, documentation The topic distribution Subject to the Dirichlet prior, the term On topic The distribution under Construct a topic-word matrix.

11. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 9 is characterized in that: The step 32 comprises: Through the RoBERTa layer to the input text Encode and get the vector representation of each word ; Learning Sequence Probabilities of Word Labels Based on Vector Representations ,in is a label sequence, according to the maximum conditional probability To predict the optimal label sequence for each word: in, is the normalization factor, is the weight of the transferred feature, It is The word label, is the label of the i-th word, represents the index of the tag, is an exponential function, is the length of the word sequence, is the number of label types; The event triples are converted into event graphs, and the association relationship of events is represented by a graph structure; the subject and object in the event triples are used as nodes in the graph, and the predicate is used as the edge connecting the nodes: in, Represents the event graph, is a collection of nodes, is the edge set; Each entity pair in the event triple By predicate connected.

12. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 9 is characterized in that: The step 33 comprises: For each node , Is a node set, generating a fixed-length node sequence , as the context of the event graph, where q is the walk length; Node sequence Embedding learning is performed to maximize the co-occurrence probability between a node and its context by the following formula: in, Is a node The context node set of Is a node Generate context node probability; Each node The embedding vector That is its representation in d-dimensional vector space; for the entire subgraph , the overall vector representation of the subgraph is generated by aggregating the embedding vectors of all its nodes. The subgraph vector It is expressed as: in, is an aggregate function, Represents the subgraph vector, that is, the hotspot pattern finally formed.

13. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 9 is characterized in that: The step 34 comprises: By calculating the Jaccard similarity between the subgraph vector and the cross-modal graph representation obtained in step 2, content features similar to the hot spot pattern are searched in the obtained cross-modal graph representation to identify potential hot content; by comparing the Jaccard similarity, potential hot content in the cross-modal feature graph is identified.

14. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 1 is characterized in that: The method of combining the cross-modal feature graph representation to construct a cross-modal content graph includes: Combined with the global feature map representation obtained in step 2 , mapping all modal graph representations to a high-dimensional semantic space; the dimension of the semantic space is d, and the graph representation of each modality is Define a mapping function : Mapping Function The graph representation of each modality is mapped to a high-dimensional semantic space, and the generated semantic vector is recorded as ,in: Get the set of modal feature vectors after semantic alignment , that is, the cross-modal content graph.

15. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 14 is characterized in that: The method of learning anomaly patterns using labeled anomaly data and training anomaly detection models according to the anomaly patterns includes: The features of different modalities are semantically aligned using graph embedding technology. The nodes of the cross-modal graph are , each node corresponds to the characteristics of a mode; By constructing a semantic similarity matrix : in, Represents the similarity between modality i and modality j, using cosine similarity Calculate the similarity of each pair of modes, and They are the feature vectors of modality i and modality j after being mapped to the high-dimensional semantic space; The annotated anomaly dataset is And use it to train anomaly detection model: According to the semantic similarity matrix , using graph neural networks to detect abnormal nodes in cross-modal graphs: in, Nodes in the graph The characteristic representation of node The abnormal probability of node Is it an abnormal node? represents the sigmoid activation function, represents the weight of the feature in the model, represents the parameters learned during the training process, Representation Node The features and their neighbor nodes The weighted sum of the features of When the anomaly detection model is trained, the loss function of the anomaly detection model is defined by the following formula: : Get the loss function After that, by minimizing To optimize the anomaly detection model.

16. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 1 is characterized in that: The identifying abnormal content in the cross-modal content graph by using the trained anomaly detection model includes: Calculate the abnormal probability of all nodes in the modal content graph, filter out abnormal nodes according to preset thresholds, and process or alarm the corresponding modal content; identify abnormal nodes through the graph neural network model.

17. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 1 is characterized in that: The step 6 comprises: Step 61: Introduce dynamic adjustment factors based on the SKIR model framework and node weight , construct the control equation, and control the state transition in the information propagation process through the adjustment factors and node weights. The state transition includes the probability changes of susceptible state, informed state and propagation state; Step 62: Monitor and update the parameters of the propagation link in step 5, and use the Kalman filter algorithm to update the model parameters, where the updated model parameters include state prediction and observation correction; Step 63: perform key node weight adjustment, and calculate the node adjustment weight based on the node betweenness centrality and real-time influence index; Step 64: Optimize the propagation direction and rate to maximize the target content coverage and establish an optimal control problem; Step 65: Generate a dynamic distribution strategy, input the optimized parameters, divide users into multiple groups through the spectral clustering algorithm, define group sensitivity, and calculate the delivery time window.

18. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 17 is characterized in that: In step 61, based on the SKIR model framework, a dynamic adjustment factor is introduced and node weight , construct the control equations, including: in, For Node The neighbor set of is a dynamic diffusion adjustment factor, ranging from [0,1], which is used to control the information penetration intensity across nodes. For Node The propagation weight ranges from [0,2], and is adjusted by It can suppress or amplify the propagation influence of nodes. represents the rate at which the unknown information state changes to the propagation state, is the propagation diffusion coefficient associated with the informed state, controlling the rate at which information propagates from the informed state to the propagated state, It is the transition rate from the informed state to the unaware state, indicating that informed users stop spreading information because they lose interest in it.

19. The network content dissemination method based on integrated cognitive understanding and intelligent governance according to claim 18 is characterized in that: The step 62 comprises: State prediction based on historical parameters To predict the current parameters : in, is the state transfer matrix, is the control input matrix, For external intervention signals, is the process noise, is the final propagation link outputted in step 5; Observation correction through real-time observation data To modify the parameters: in, is the observation matrix, is the Kalman gain matrix, Represents the updated state prediction parameters.

20. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 17 is characterized in that: The step 63 comprises: Based on node betweenness centrality and real-time impact indicators , calculate the node adjustment weight: in, is the learning rate, which is used to control the weight update amplitude, and finally outputs the adjusted node weight set .

21. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 17 is characterized in that: The step 64 comprises: Optimize the direction and rate of dissemination to maximize the coverage of target content As the goal, establish the optimal control problem: in, For the delivery cycle, is the weight regularization coefficient, and the Pontryagin maximum principle is used to solve the Hamiltonian function , derive the optimal control law: in, is a covariate variable, represents the optimal output weight.

22. The network content dissemination method based on integrated cognitive understanding and intelligent management according to claim 18 is characterized in that: The step 65 comprises: By entering the optimized parameters , using spectral clustering algorithm to divide users into Groups , defining group sensitivity , calculate the delivery time window: in, It represents the best delivery time window, that is, the best time for content delivery to a group. is the attenuation coefficient, It represents the average number of users in the propagation state at time s, the triple , indicating that In time By weight Deliver content.

23. A network content dissemination system based on integrated cognitive understanding and intelligent management, which implements the network content dissemination method based on integrated cognitive understanding and intelligent management as described in any one of claims 1 to 22, characterized in that: include: A decoupling module is used to decouple data features of different modes and output the decoupled data features; Different modalities include text, video, audio, and images; The generation module is used to construct a subgraph network for each modality based on the decoupled data features of audio, video, image, and text, and to perform convolution operations on cross-modal features through multiple subgraph networks to generate cross-modal feature graph representations; The hot content identification module is used to perform topic modeling on the text in the text knowledge base, extract potential topic words, and extract event triplets from the potential topic words to construct an event graph; retrieve nodes related to hot spots from the event graph to form hot spot patterns, match content features related to hot spot patterns in the cross-modal feature graph representation, and identify potential hot content; The abnormal content identification module is used to combine the cross-modal feature graph representation, construct a cross-modal content map, use the annotated abnormal data to learn abnormal patterns, and train an anomaly detection model based on the abnormal patterns to identify abnormal content in the cross-modal content map; abnormal content includes false information, sensitive images, and dense videos; The prediction module is used to simulate the process of information dissemination among users and predict the content dissemination links of hot content and abnormal content; The determination module is used to adjust the weights of key nodes in the content dissemination chain and determine the time, location and target audience of content delivery.

Images