Reinforcement learning-based multi-agent driven network sentiment regulation method and system

Abstract

The application discloses a kind of multi-agent driven network sentiment regulation method and system based on reinforcement learning, belong to artificial intelligence and network information processing technical field.The method obtains text, speech, image, video, behavior, relationship and propagation path data, constructs space-time heterogeneous interaction graph and extracts graph structure feature;Combined with multi-modal basic feature, establish causal attention sentiment state representation;Multi-agent collaborative decision-making environment, sentiment regulation action set, constraint reward function and counterfactual propagation environment are constructed;Collaborative communication training is carried out using multi-agent constrained reinforcement learning, and the output joint regulation strategy;According to sentiment feedback, behavior feedback and propagation feedback, closed-loop update is carried out, and according to negative sentiment proportion, sentiment entropy, propagation risk index and constraint satisfaction rate, iterative control is carried out.The application can improve the accuracy, collaboration, robustness and constraint controllability of network sentiment regulation.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, network information processing, affective computing, swarm intelligence decision-making and intelligent control, and in particular to a reinforcement learning-based multi-agent driven network affective regulation method and system that integrates multimodal perception, spatiotemporal heterogeneous graph neural networks, causal attention mechanisms, counterfactual evaluation and multi-agent constraints. Background Technology

[0002] With the rapid development of social media platforms, short video platforms, forums, and instant interactive networks, the phenomenon of emotional dissemination in cyberspace, driven by text, voice, images, videos, and interactive behaviors, is becoming increasingly complex. Online emotions not only manifest as individual-level emotional expressions but also spread, amplify, and dynamically evolve through forwarding, commenting, aggregated discussions, and group interactions, thereby influencing online public opinion trends, community atmosphere stability, and information ecosystem security. Therefore, how to accurately perceive, dynamically assess, and effectively regulate emotional states in cyberspace has become a key technical issue in intelligent information processing and platform governance.

[0003] In existing technologies, solutions for network sentiment analysis and modulation mainly fall into the following categories: one is to identify the sentiment polarity of text based on dictionary rules, traditional machine learning or deep learning models; the other is to improve the accuracy of sentiment recognition by fusing multi-source data such as text, speech and images through multimodal learning methods.

[0004] However, existing technologies still have at least the following shortcomings:

[0005] First, existing sentiment modeling methods mostly focus on relevance learning, usually using attention mechanisms to weight and fuse information from different modalities or different neighborhoods. However, most of these weights only reflect the strength of statistical correlation and are difficult to distinguish between the causal factors that truly drive sentiment changes and accidental co-occurrence, short-term noise, or pseudo-related information. Therefore, under conditions of hotspot migration, propagation mutation, and complex confounding, the stability and interpretability of sentiment state representation are insufficient.

[0006] Secondly, most existing communication regulation schemes treat cyberspace as a single decision-making object, lacking the ability to collaboratively model different communities, different communication levels, different node roles, and different regulation tasks. This makes it difficult to adapt to the complex characteristics of multi-regional concurrency, multi-path coupling, and multi-subject interaction in online emotional communication, thus affecting the precision, real-time nature, and global coordination of regulation strategies.

[0007] Third, existing reinforcement learning regulation methods typically construct reward functions based solely on direct feedback after an action, making it difficult to effectively distinguish between the marginal regulatory contribution of the action itself and the surface changes brought about by natural environmental evolution. This leads to biases in the reward signal, which in turn affects the stability of policy training and the accuracy of policy optimization direction.

[0008] Fourth, existing solutions typically lack unified modeling and joint control of real-world constraints such as user experience, fairness, propagation security, platform resources, and intervention frequency. This can easily lead to problems where local indicators improve but the overall governance boundary becomes unbalanced, making it difficult to deploy stably in real network environments.

[0009] Fifth, existing technologies often treat emotion recognition, propagation evaluation, policy generation, and feedback updates in a fragmented manner, lacking a closed-loop adaptive mechanism from emotion state modeling to policy execution and then to feedback correction. This results in insufficient continuous adjustment capability of the system under conditions of environmental changes, topic migration, and behavioral pattern drift.

[0010] Therefore, it is necessary to propose a multi-agent-driven network emotion regulation method and system based on reinforcement learning to at least solve one or more of the above-mentioned technical problems. Summary of the Invention

[0011] The purpose of this invention is to provide a multi-agent-driven network sentiment regulation method and system based on reinforcement learning, in order to solve the problems of insufficient causality in the representation of network sentiment state, insufficient multi-agent collaborative regulation ability, difficulty in characterizing the real marginal regulation contribution of reward signals, difficulty in uniformly incorporating real constraints, and lack of closed-loop adaptive update mechanism in the prior art, thereby improving the accuracy, robustness, interpretability, collaboration and constraint controllability of network sentiment regulation.

[0012] To achieve the above objectives, this invention provides a multi-agent-driven network sentiment regulation method based on reinforcement learning. This method, executed by a processor, includes: constructing a spatiotemporal heterogeneous interaction graph based on multi-source heterogeneous data in the target network space, and extracting graph structure features corresponding to propagation topological dependencies using a graph neural network; inputting multimodal basic features, the graph structure features, intervention context vectors, and decontamination representation vectors into a causal attention encoding network, calculating the causal contribution weights of each modality feature, neighborhood feature, and propagation path feature to the target sentiment state, and generating a causal-enhanced sentiment state representation; constructing a multi-agent collaborative decision-making environment based on the causal-enhanced sentiment state representation, global graph state, historical joint actions, and constraint states; constructing a reward function including task benefit items, counterfactual reward items, and constraint penalty items, and constructing a counterfactual propagation environment on the spatiotemporal heterogeneous interaction graph, performing propagation deduction on actual joint actions and counterfactual reference actions respectively, and using the improvement amount of the target sentiment trajectory and the suppression of propagation risk... The difference between the amount of control or the improvement in group stability is used as the counterfactual reward. A multi-agent constrained reinforcement learning algorithm is adopted. Under the framework of centralized training and decentralized execution, the policy network and value network of each agent are trained by combining the message passing attention mechanism between agents. The degree of satisfaction of user experience constraints, fairness constraints, propagation security constraints, platform resource constraints and intervention frequency constraints is controlled by Lagrange multiplier update, constraint cost estimation or feasible region projection to obtain a joint regulation strategy. Emotion regulation is performed based on the joint regulation strategy. According to the regulated emotional feedback, behavioral feedback and propagation feedback, the spatiotemporal heterogeneous interaction graph, the causal contribution weight, the constraint state and the policy network parameters are updated. The updated group emotion distribution, emotion entropy, propagation risk index and constraint satisfaction rate are used to determine whether the preset regulation target has been achieved. If not, the graph structure update, causal representation update, policy learning and regulation execution process are continued to form a closed-loop adaptive regulation for the dynamic evolution of network emotion.

[0013] To achieve the above objectives, another aspect of the present invention provides a multi-agent driven network emotion regulation system based on reinforcement learning, comprising:

[0014] The spatiotemporal heterogeneous graph construction module is used to construct a spatiotemporal heterogeneous interaction graph based on multi-source heterogeneous data in the target network space, and to extract graph structure features corresponding to propagation topological dependencies using graph neural networks; the causal sentiment representation module is used to input multimodal basic features, the graph structure features, intervention context vectors and decontamination representation vectors into a causal attention encoding network, calculate the causal contribution weights of each modality feature, neighborhood feature and propagation path feature to the target sentiment state, and generate a causal enhanced sentiment state representation;

[0015] The multi-agent environment modeling module is used to construct a multi-agent collaborative decision-making environment based on the causal enhanced emotional state representation, global graph state, historical joint actions and constraint states.

[0016] The reward and constraint modeling module is used to construct a reward function that includes task benefit items, counterfactual reward items, and constraint penalty items;

[0017] The counterfactual assessment module is used to construct a counterfactual propagation environment on the spatiotemporal heterogeneous interaction graph, perform propagation deduction on actual joint actions and counterfactual reference actions respectively, and determine counterfactual rewards;

[0018] The policy training module is used to train the policy network and value network of each agent under a multi-agent constrained reinforcement learning algorithm, in a framework of centralized training and distributed execution, combined with the message passing attention mechanism between agents. It also controls the degree to which user experience constraints, fairness constraints, propagation security constraints, platform resource constraints and intervention frequency constraints are satisfied, so as to obtain a joint regulation policy.

[0019] The adjustment execution and closed-loop update module is used to perform emotion adjustment based on the joint adjustment strategy, and update the spatiotemporal heterogeneous interaction graph, the causal contribution weight, the constraint state and the strategy network parameters according to the adjusted emotion feedback, behavioral feedback and propagation feedback, and drive continued iterative operation when the preset adjustment target is not reached.

[0020] Compared with the prior art, the present invention has at least the following beneficial effects:

[0021] First, by constructing a spatiotemporally heterogeneous interaction graph and extracting the graph structure features corresponding to the propagation topology dependencies, this invention can incorporate node relationships, path relationships, topic relationships, and temporal evolution relationships in network sentiment propagation into a unified modeling process, thereby improving the ability to represent complex propagation structures.

[0022] Second, this invention introduces a causal attention mechanism, which no longer relies solely on statistical correlation for feature weighting. Instead, it combines intervention context vectors and deconfounding representation vectors to determine the causal contribution weights of multimodal features, neighborhood features, and propagation path features to the target emotional state. This helps to suppress the interference of pseudo-related information, short-term noise information, and accidental co-occurrence information on emotional modeling, thereby improving the stability and interpretability of emotional state representation.

[0023] Third, this invention constructs a multi-agent collaborative decision-making environment, enabling multiple agents to make collaborative decisions for different communities, different propagation levels, different intervention targets, or different regulation tasks. It also achieves collaborative communication through a message passing attention mechanism between agents, thereby improving the parallelism, precision, and global coordination capabilities of network emotion regulation.

[0024] Fourth, this invention constructs a counterfactual propagation environment on a spatiotemporally heterogeneous interaction graph, and performs propagation deduction on actual joint actions and counterfactual reference actions respectively. The difference between the propagation effects of the two constitutes the counterfactual reward, which can more accurately distinguish between the marginal adjustment benefits brought by the action itself and the surface changes caused by natural environmental changes, thereby improving the effectiveness of reward construction and the accuracy of strategy training.

[0025] Fifth, this invention integrates user experience constraints, fairness constraints, propagation security constraints, platform resource constraints, and intervention frequency constraints into the training and execution process through a multi-agent constrained reinforcement learning algorithm. It also utilizes Lagrange multiplier updates, constraint cost estimation, or feasible region projection to control the degree of constraint satisfaction. This allows the joint regulation strategy to improve regulation effectiveness while taking into account the real governance boundary, thereby enhancing the system's deployability and constraint controllability.

[0026] Sixth, after the adjustment is performed, the present invention uses emotional feedback, behavioral feedback and propagation feedback to update the spatiotemporal heterogeneous interaction graph, causal contribution weight, constraint state and policy network parameters, and judges whether the preset adjustment target has been achieved based on the group emotional distribution, emotional entropy, propagation risk index and constraint satisfaction rate, so as to form a closed-loop adaptive adjustment mechanism, thereby improving the system's continuous adaptability and robustness to changes in network environment, topic migration and behavioral pattern drift.

[0027] Seventh, this invention can improve the accuracy, robustness, interpretability, and controllability of online sentiment regulation, and is applicable to various online spaces such as social platforms, forums, short video platforms, and interactive content distribution platforms for sentiment regulation and dissemination governance. Attached Figure Description

[0028] Figure 1 This is a flowchart of the multi-agent-driven network sentiment regulation method based on reinforcement learning provided by the present invention. Detailed Implementation

[0029] The following describes specific embodiments of the present invention in conjunction with the method steps of the present invention. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Equivalent substitutions, modifications, or improvements made by those skilled in the art without departing from the concept of the present invention should all fall within the scope of protection of the present invention.

[0030] This embodiment uses "a method and system for emotion regulation in multi-agent driven networks based on reinforcement learning" as an example for illustration. The method can be executed by a server, cloud computing platform, edge computing node, dedicated processor, or electronic device containing a processor and memory.

[0031] Figure 1This is a flowchart of the multi-agent-driven network sentiment regulation method based on reinforcement learning provided by the present invention, such as... Figure 1 As shown, the multi-agent-driven network emotion regulation method based on reinforcement learning provided by this invention includes the following steps.

[0032] S1. Acquisition of multi-source heterogeneous data and construction of multimodal observation sample sets

[0033] In this embodiment, raw data is obtained from the data interface, log system, content database, relational database, and streaming messaging system of the target network space. The raw data includes at least text data, voice data, image data, video data, user behavior data, social relationship data, and propagation path data. The text data may come from post text, comment text, reply text, title text, and tag text; the voice data may come from voice comments, live audio tracks, and voice message clips; the image data may come from illustrations, emoticons, cover images, and content screenshots; the video data may come from short video files, live clips, or keyframe sequences; the user behavior data may include click events, like events, comment events, forwarding events, dwell time events, and exit events; the social relationship data may include follow relationships, fan relationships, friend relationships, and group relationships; and the propagation path data may include the initial content publishing node, each forwarding node, comment reply chains, and cross-topic propagation paths.

[0034] In this embodiment, the input data is sampled using a preset time window. Various modal data falling within the same time window are aggregated into a multimodal observation unit, forming a time frame. Multimodal observation set:

[0035]

[0036] in, Indicates time The multimodal observation set, Indicates time Text data, Indicates time Voice data, Indicates time Image data, Indicates time The video data, Indicates time User behavior data, Indicates time Social relationship data, Indicates time The propagation path data.

[0037] In this implementation, text data undergoes word segmentation, lexical normalization, stop word filtering, facial expression mapping, and length truncation; speech data undergoes sampling rate unification, background noise removal, endpoint detection, and speech segmentation; image and video data undergo size normalization, color normalization, frame sampling, and low-quality sample removal; user behavior data undergoes outlier removal, duplicate event merging, and event frequency standardization; and social relationship data and propagation path data undergo isolated node correction, duplicate edge merging, and timestamp alignment. For objects with modal missingness, cross-modal prediction models or historical window completion methods are used for missing data completion. After unified encoding, the result is:

[0038]

[0039] in, Indicates time The observation samples after preprocessing and uniform coding, This represents the multimodal preprocessing and unified coding function. Indicates time A multimodal observation set.

[0040] All uniformly coded samples from all time windows are arranged in chronological order to form a multimodal observation sample set:

[0041]

[0042] in, This represents a multimodal observation sample set. Indicates the first The unified coded observation samples corresponding to each time window This indicates the total number of time windows.

[0043] In this embodiment, the unified coding function Self-supervised pre-training methods can be used, such as training with cross-modal alignment loss, reconstruction loss, or contrastive learning loss; supervised fine-tuning methods can also be used to make the unified encoding results more suitable for subsequent sentiment state representation tasks. Cross-modal completion models can be trained offline using datasets with complete modality samples.

[0044] The output of this step is a multimodal observation sample set. Its function is to uniformly collect, clean, complete, and encode multi-source heterogeneous data, enabling subsequent steps to process content, relational, and behavioral information at a unified spatiotemporal granularity. Its beneficial effect is to avoid information fragmentation caused by inconsistent granularity of multi-source data, improving the consistency and completeness of input for subsequent graph modeling and sentiment representation.

[0045] S2. Construction of Spatiotemporal Heterogeneous Interaction Graphs and Extraction of Graph Structure Features

[0046] In this embodiment, the multimodal observation sample set output in step S1 is used. As input, read the user entity, content entity, topic entity, and event entity within each time window to construct the time frame. Spatiotemporal heterogeneous interaction graph:

[0047]

[0048] in, Indicates time Spatiotemporal heterogeneous interaction graph, Indicates time The set of nodes, Indicates time The set of edges, This represents a node type mapping function, used to map nodes as user nodes, content nodes, topic nodes, or event nodes. This function represents the edge type mapping function, used to map edges to post edges, comment edges, retweet edges, follow edges, co-occurrence edges, or topic propagation edges. Indicates time The node-edge attribute mapping function.

[0049] In this implementation, the node construction rules are as follows: user identifiers are mapped to user nodes; posts, comments, audio clips, image content, and video content are mapped to content nodes; clustered keyword themes and topic tag clusters are mapped to theme nodes; and hot events or public opinion triggering events are mapped to event nodes. The edge construction rules are as follows: if a user publishes content, a publishing edge is established between the user node and the content node; if a user comments on, forwards, or likes the content, a comment edge, forwarding edge, or interaction edge is established; if two pieces of content belong to the same theme cluster, a co-occurrence edge is established; if the content participates in the same propagation chain, a propagation path edge is established; if a theme and an event co-occur frequently within the same time window, a theme-event association edge is established.

[0050] The node attribute vector can be represented as:

[0051]

[0052] in, Represents a node At any moment The initial attribute vector, Represents a node At any moment The basic characteristics of unified coding Represents a node At any moment Historical behavioral statistical characteristics Represents a node At any moment Modal quality or reliability characteristics.

[0053] The edge attribute vector can be represented as:

[0054]

[0055] in, Indicates time node To the node The edge attribute vector, Represents a node To the node Interaction frequency, Represents a node To the node The propagation delay Represents a node To the node The strength of the relationship Represents a node To the node The path attenuation coefficient.

[0056] In this embodiment, a graph neural network is used to encode the relationship propagation in the spatiotemporal heterogeneous interaction graph. The graph neural network can be selected as a relational graph convolutional network, a graph attention network, a heterogeneous graph transformation network, or a spatiotemporal graph neural network. In this embodiment, taking a relational graph convolutional network as an example, the node update is as follows:

[0057]

[0058] in, Represents a node At any moment After the first The updated representation after layered graph neural network, Represents a node At any moment After the first The representation obtained from the layered graph neural network, Represents a non-linear activation function. Indicates the first Layer self-connection weight matrix, Represents a set of relation types. Indicates time Next node In relation types The set of neighboring nodes on, Represents a node In relation types The normalized coefficients under the following conditions Indicates the first Hierarchical Relationship Type The corresponding transformation weight matrix.

[0059] The output of the last layer of the graph neural network is used as the node at time t. Graph structural features:

[0060]

[0061] in, Represents a node At any moment The graph structure features, Represents a node At any moment After the first The representation of the output of a layered graphical neural network. This represents the total number of layers in the graph neural network.

[0062] In this embodiment, the graph neural network can be trained using supervised training, semi-supervised training, or self-supervised training. If sentiment labels, propagation risk labels, or key node labels exist, supervised loss is used for training; if labels are insufficient, link prediction loss, attribute reconstruction loss, and contrastive learning loss are used to train the graph encoder. The graph encoder can be pre-trained offline and then fine-tuned online, or it can be jointly trained with a causal attention network.

[0063] The output of this step is a spatiotemporal heterogeneous interaction graph. and node graph structural features Its function is to organize users, content, topics, and events in a unified graph space, and to extract propagation topological dependencies and higher-order relationships. Its beneficial effects include enhancing the ability to characterize complex propagation structures, key propagation paths, and propagation hubs, providing a structured propagation context for subsequent causal sentiment representation and strategic decision-making.

[0064] S3. Multimodal basic feature extraction and causal attention-based emotion state modeling

[0065] In this embodiment, the unified encoded samples output in step S1 and the graph structure features output in step S2 are used as inputs to extract modality-level basic features for each node. The text modality outputs text embeddings through a pre-trained language model, the speech modality outputs prosodic embeddings through an acoustic encoder, the image modality outputs visual embeddings through a visual encoder, the video modality outputs video embeddings through a temporal visual network, and the behavior modality outputs behavior embeddings through a behavior sequence encoder. The nodes... At any moment The multimodal basic feature organization is as follows:

[0066]

[0067] in, Represents a node At any moment The multimodal basic feature set, Represents a node At any moment The Modal feature vectors, Indicates the number of modes.

[0068] Construct nodes At any moment Intervention context vector and de-hybridization representation vector The intervention context vector can be obtained by concatenating the platform's current adjustment strategy, topic popularity status, user segmentation information, and historical intervention records; the decontamination representation vector can be obtained by encoding variables such as time, community type, user activity, content length, and event intensity by a contamination variable encoder.

[0069] Calculate modal causal attention scores using a causal attention scoring function:

[0070]

[0071] in, Represents a node At any moment For the first Causal attention scores for each modal feature, This represents the causal attention scoring function. Represents a node At any moment The Each modal feature vector Represents a node At any moment Intervention context vector, Represents a node At any moment The decontamination representation vector.

[0072] Modal causal attention weights are obtained based on causal attention scores:

[0073]

[0074] in, Represents a node At any moment For the first Causal attention weights for each modality feature, Represents an exponential function. Represents a node At any moment For the first Causal attention scores for each modal feature, This represents the modal index used for normalized summation. This represents the total number of modes.

[0075] Simultaneously, causal contribution modeling is performed on the node's neighborhood, and neighborhood causal attention weights are calculated:

[0076]

[0077] in, Represents a node At any moment For nodes Neighborhood causal attention weights Represents a node At any moment The query vector, Represents a node At any moment The key vector, This represents the transpose of a vector. Represents a node The set of neighboring nodes.

[0078] Based on this, a causal-enhanced sentiment state representation is generated:

[0079]

[0080] in, Represents a node At any moment The causal-enhanced affective state indicates that Represents a node At any moment For the first Causal attention weights for each modality feature, Represents a node At any moment The Each modal feature vector Represents a node At any moment For nodes Neighborhood causal attention weights Represents a node At any moment The graph structure features.

[0081] In this embodiment, the causal attention network is trained using a multi-task joint training method. The loss function includes at least sentiment classification loss, sentiment intensity regression loss, and causal regularization loss. When it is necessary to improve causal robustness, sentiment rating weighted loss, counterfactual consistency loss, or environmental invariance constraint loss can also be introduced. The causal attention network can be trained independently or jointly trained end-to-end with a graph neural network. The modal encoder parameters can be initialized using publicly available pre-trained models, and the pseudo-intervention labels required for causal training can be extracted from the platform's historical intervention logs.

[0082] In this invention, to improve the interpretability and robustness of the causal contribution weights, the causal contribution weights can be further determined based on the average processing effect, conditional average processing effect, or causal contribution score of the same-modal features and different neighborhood features on the target emotional outcome. The average processing effect characterizes the overall average influence of a feature on the target emotional outcome; the conditional average processing effect characterizes the conditional influence of the feature on the target emotional outcome under given intervention context and confounding conditions; and the causal contribution score comprehensively reflects the actual driving effect of the feature on the change in target emotional state in the current state. Based on the average processing effect, conditional average processing effect, or causal contribution score, the conventional attention weights are modified, thereby reducing the interference of pseudo-related information, accidental co-occurrence information, and short-term noise propagation information on emotional state modeling.

[0083] The output of this step is a node-level causal-enhanced sentiment state representation. And a global sentiment state set. Its role is to evaluate the causal contribution of multimodal features, neighborhood features, and propagation path features by using intervention context and deconfounding representation. Its beneficial effects are to reduce the interference of pseudo-related information, accidental co-occurrence information, and short-term noise propagation information on sentiment modeling, and to improve the authenticity, stability, and interpretability of sentiment state representation.

[0084] S4, Construction of Multi-Agent Collaborative Decision-Making Environment

[0085] In this embodiment, a multi-agent collaborative decision-making environment is constructed using the causal enhanced sentiment state representation output in step S3, the graph structure output in step S2, and the historical actions and constraint states as inputs. The entire network space is partitioned according to the propagation community, topic cluster, user group, or platform business logic, with each partition corresponding to an agent; when needed, a global coordinating agent can also be set up to generate high-level constraints or resource allocation instructions.

[0086] No. An intelligent agent at time Local observation is defined as:

[0087]

[0088] in, Indicates the first An intelligent agent at time Local observations Indicates the first Each agent is responsible for a subset of nodes corresponding to a set of causal enhanced sentiment states. Indicates the first Local subgraphs corresponding to each agent Indicates the first The action performed by the agent in the previous moment. Indicates the first An intelligent agent at time The local constraint state vector.

[0089] The global reinforcement learning state is defined as:

[0090]

[0091] in, Indicates time The global reinforcement learning state, Indicates time A global set of causal emotional states. Indicates time Spatiotemporal heterogeneous interaction graph, Indicates the combined actions of the previous moment. Indicates time The global constraint state vector.

[0092] Each agent outputs its action, and the actions of all agents together form a joint action:

[0093]

[0094] in, Indicates time joint actions, Indicates the first An intelligent agent at time The action, This represents the total number of intelligent agents.

[0095] The output of this step is a local observation. Global reinforcement learning state and joint actions The modeling framework decomposes the overall network regulation task into multiple related local decision-making tasks and manages them in a coupled manner through a unified global state. Its beneficial effects include balancing fine-grained local regulation with global collaborative control, and improving parallel decision-making capabilities under conditions of multiple communities, multiple propagation levels, and multiple policy clusters.

[0096] S5. Construction of Emotion Regulation Action Set and Constrained Reward Function

[0097] In this embodiment, the global state and joint actions from step S4 are used as input to define an emotion regulation action space for each agent. The actions include at least content injection, ranking adjustment, topic guidance, node flow limiting, path damping, positive information enhancement, and regulation intensity allocation. These actions can be discrete, continuous, or a hybrid discrete-continuous approach. For example, ranking adjustment can be represented as increasing, decreasing, or maintaining; node flow limiting ratio, path damping coefficient, and content injection intensity can be represented as continuous variables. The joint action space is represented as follows:

[0098]

[0099] in, Represents the joint action space, Indicates the first The action space of an intelligent agent This represents the Cartesian product operation. Indicates the number of agents.

[0100] Construct the total reward function:

[0101]

[0102] in, Indicates time Total reward Indicates time Task rewards items, Indicates time Counterfactual rewards This represents the counterfactual reward weighting coefficient. Indicates the total number of constraint terms. Indicates the first The penalty weight coefficient for each constraint term. Indicates the first A constraint violation degree function.

[0103] The task benefit item is defined as:

[0104]

[0105] in, Indicates time Task rewards items, This represents the weighting coefficient of each component of the task's reward. Indicates time The target emotional distribution improvement amount, Indicates time The improvement in group emotional entropy Indicates time The risk of transmission has decreased.

[0106] User experience constraints can be expressed as:

[0107]

[0108] in, Indicates the degree of violation of user experience constraints. Indicates time The negative feedback rate caused by intervention This indicates the upper limit of the allowed negative feedback rate.

[0109] Fairness constraints can be expressed as:

[0110]

[0111] in, Indicates the degree of violation of fairness constraints. Represents a set of user groups. Indexes representing different user groups Indicates user group At any moment The intensity of the intervention received Indicates user group At any moment The intensity of the intervention received This represents the absolute value operation. This indicates the threshold for allowing fairness differences.

[0112] Resource constraints can be represented as:

[0113]

[0114] in, Indicates the degree of resource constraint violation. Indicates time Resource consumption, This indicates the upper limit of resource consumption.

[0115] The intervention frequency constraint can be expressed as:

[0116]

[0117] in, Indicates the degree of violation of the intervention frequency constraint. This indicates the number of interventions per unit of time. This indicates the maximum allowed intervention frequency.

[0118] The output of this step is the total reward. Task Rewards And the set of constraint violation degrees. Its role is to unify the effect of emotional regulation with the boundary of real-world governance and incorporate it into the optimization objective. Its beneficial effect is that it upgrades the learning objective of multi-agent systems from maximizing a single benefit to optimizing the comprehensive benefits under constraints, thereby improving the safety, controllability, fairness, and engineering deployability of the strategy.

[0119] S6. Construction of Counterfactual Communication Environment and Calculation of Counterfactual Rewards

[0120] In this embodiment, the spatiotemporal heterogeneous interaction graph of step S2 is used. Global state in step S4 and the actual combined action of step S5 As input, construct a counterfactual propagation environment. For each actual joint action, construct a corresponding counterfactual reference action. The counterfactual reference action may include a zero-intervention action, a historical baseline policy action, a random perturbation action, a joint action after removing the contribution of a certain agent, or an alternative action obtained by reallocating intervention resources while keeping the total strength unchanged.

[0121] Under the same initial conditions, the propagation effects of the actual joint action and the counterfactual reference action are calculated using the propagation simulation module or the propagation effect evaluation model, respectively:

[0122]

[0123] in, Indicates time Estimates of the propagation effect after the actual joint action is carried out. Indicates time Estimates of the propagation effect after performing a counterfactual reference action. This represents the function for evaluating the propagation effect. Indicates time The global state, Indicates time Actual joint actions Indicates time Counterfactual reference actions.

[0124] The propagation effect evaluation function is defined as follows:

[0125]

[0126] in, Indicates action In state The combined dissemination effect under the following circumstances This represents the weighting coefficient of each effect component. Indicates action At any moment The corresponding target sentiment improvement estimate is as follows: Indicates action At any moment The corresponding estimate of transmission risk suppression, Indicates action At any moment The corresponding estimate of the increase in population stability.

[0127] Counterfactual reward is calculated as follows:

[0128]

[0129] in, Indicates time Counterfactual rewards This indicates the overall propagation effect of the actual joint actions. This indicates the combined propagation effect of counterfactual reference actions. This refers to a counterfactual reference action that corresponds to the actual joint action.

[0130] In this implementation, the propagation effect assessment model can be trained under supervision using historical propagation data from the platform, and its labels can be obtained statistically from actual propagation results. If a graph propagation prediction network is used, the global state, action embedding, and current sentiment state can be input, and the predicted values ​​of sentiment indicators and propagation indicators for the next few steps can be output. This model can be trained offline first, and then fine-tuned based on feedback data during online operation.

[0131] The output of this step is a counterfactual reward. This is then written back to the total reward function in step S5. Its purpose is to compare the propagation effects of the actual combined action and the reference action under the same environmental conditions. Its beneficial effect is to more accurately distinguish between the marginal regulatory gain brought about by the action itself and the superficial improvement caused by natural environmental changes, thereby improving the ability of reward evaluation to identify the true regulatory contribution.

[0132] S7, Multi-agent Constraint Reinforcement Learning Training and Collaborative Communication

[0133] In this embodiment, the local observation in step S4 is used. Using the total reward function of step S5 and the counterfactual reward of step S6 as inputs, a policy network and a value network for multiple agents are trained. The policy network of each agent is represented as: Value network is represented as The joint strategy is expressed as:

[0134]

[0135] in, Indicates a joint strategy, This represents the set of policy network parameters for all agents. Represents the set of local observations of all agents. This indicates a series of multiplication operations. Indicates the first An agent observes locally. Down Output Action The strategy function.

[0136] To enhance collaboration between intelligent agents, a message-passing attention mechanism is introduced. An intelligent agent at time The received message vector is:

[0137]

[0138] in, Indicates time No. The cooperative message vector received by each agent. Indicates time No. The first agent on the first Communication attention weights of individual agents Represents the message mapping matrix. Indicates the first An intelligent agent at time The hidden state representation.

[0139] The communication attention weight can be represented as:

[0140]

[0141] in, Indicates the communication attention weight, Indicates the first The query vector of each agent Indicates the first The key vector of each agent This represents the transpose of a vector.

[0142] The agent's final policy input can be formed by concatenating local observations and cooperative messages:

[0143]

[0144] in, Indicates the first An intelligent agent at time Enhanced observation input, This represents a vector concatenation operation. Indicates the first Local observations of an agent Indicates the first The cooperative message vector received by each agent.

[0145] The constrained optimization objective can be expressed as:

[0146]

[0147] in, Represents the set of policy parameters The optimization goal, Represents the mathematical expectation. Indicates the discount factor. Indicates time Total reward Indicates the total number of constraint terms. Indicates the first Lagrange multipliers of each constraint term, Indicates the first A constraint violation degree function, Indicates the first Each constraint allows an upper limit. This represents the total number of time steps in a training sequence or training round.

[0148] Lagrange multipliers are updated to:

[0149]

[0150] in, Indicates the first The current Lagrange multipliers of each constraint term, This indicates projection onto the non-negative interval. This indicates the step size for updating the Lagrange multipliers. Indicates the first The cumulative expected violation of the discount for each constraint term. This indicates the upper limit allowed by the corresponding constraint.

[0151] In this embodiment, the policy network and value network of each agent are trained using a multi-agent constrained reinforcement learning algorithm. The training methods include: storing and sampling the state-action-reward-next-state trajectory through an experience replay mechanism; performing offline pre-training of the policy network and value network using historical log data; and performing online fine-tuning and updates in a real system or high-fidelity simulation environment. During training, target network, priority experience replay, gradient pruning, and entropy regularization mechanisms are also employed to improve training stability and convergence performance.

[0152] In this invention, to improve the consistency of constraint boundary learning during multi-agent collaborative training, each agent can share a global constraint cost estimator, which is used to estimate the cumulative violation degree of each constraint term based on the global state, joint actions, and feedback results. Alternatively, each agent can share a feasible region discrimination network, which is used to determine whether the current state-action combination satisfies user experience constraints, fairness constraints, propagation security constraints, platform resource constraints, and intervention frequency constraints. The global constraint cost estimator or the feasible region discrimination network can be trained synchronously with the policy networks of each agent, and provide a unified constraint reference for policy updates under a centralized training and distributed execution framework, thereby improving the consistency and stability of constraint boundary learning during multi-agent collaborative training.

[0153] The output of this step is the trained joint policy parameters, the updated value network parameters, and the set of Lagrange multipliers. Its purpose is to learn a joint regulation policy that satisfies the constraints within a centralized training and distributed execution framework. Its beneficial effects include balancing multi-region parallel regulation, agent cooperative communication, and global governance boundary control, thereby improving the stability, coordination, and constraint controllability of policy training.

[0154] S8. Emotional Regulation Execution and Closed-Loop Feedback Update

[0155] In this embodiment, the joint strategy parameters output in step S7 and the current global state are used as input to execute joint actions and apply sentiment adjustment to the target network space. The adjustment execution module maps the actions to platform-executable instructions. For example, the ranking adjustment action is mapped to the recommendation ranking weight update, the path damping action is mapped to the propagation edge weight decay, the node rate limiting action is mapped to the exposure limit of a specific node, and the content injection action is mapped to the positive content recommendation insertion ratio adjustment.

[0156] After adjustments are implemented, feedback data is collected in real time. This feedback data includes changes in sentiment distribution, click-through rate, comment rate, dwell time, complaint rate, dissemination depth, and activity levels at key nodes. The graph attributes are updated based on the feedback.

[0157]

[0158] in, Indicates time The result of the graph attribute mapping function. This represents the graph attribute update function. Indicates time The result of the graph attribute mapping function. Indicates time The amount of feedback change.

[0159] Update the global causal sentiment state set based on feedback:

[0160]

[0161] in, Indicates time A global set of causal emotional states. This represents a causal sentiment state update function. Indicates time A global set of causal emotional states. Indicates the amount of feedback change. This represents the updated spatiotemporal heterogeneous interaction graph.

[0162] Update the constraint state and policy parameters based on feedback:

[0163]

[0164] in, Indicates time The constraint state vector, This represents the constraint state update function. Indicates time The constraint state vector, This represents the updated set of joint policy parameters. This indicates the function for updating policy parameters. Indicates time The set of joint strategy parameters.

[0165] The output of this step is an updated spatiotemporal heterogeneous interaction graph, a set of causal sentiment states, constraint state vectors, and policy parameters. Its purpose is to feed the adjusted real feedback back into the propagation structure, sentiment states, and policy model. Its beneficial effect is that it enables the system to continuously correct its propagation structure cognition, sentiment state cognition, and policy control direction as the environment changes, thereby forming a closed-loop adaptive adjustment mechanism.

[0166] S9. Adjustment Target Determination and Iterative Control

[0167] In this implementation, the update result of step S8 is used as input to calculate the current negative sentiment percentage, group sentiment entropy, propagation risk index, and constraint satisfaction rate. The negative sentiment percentage is calculated as follows:

[0168]

[0169] in, Indicates time The percentage of negative emotions, Indicates time The number of negative sentiment nodes or samples, Indicates time The total number of nodes or samples.

[0170] The group emotional entropy is calculated as follows:

[0171]

[0172] in, Indicates time The group emotional entropy This represents the total number of emotion categories. Indicates time Belongs to the The percentage of samples with similar emotions Represents a logarithmic function.

[0173] The transmission risk index is calculated as follows:

[0174]

[0175] in, Indicates time The transmission risk index This represents the weighting coefficient of each risk component. Indicates the length of the critical propagation path. Indicators representing the speed of propagation This indicates the activity level of high-risk nodes.

[0176] The constraint satisfaction rate is calculated as follows:

[0177]

[0178] in, Indicates time The constraint satisfaction rate Indicates the total number of constraint terms. This represents an indicator function that takes the value 1 if the condition within the parentheses is true, and 0 otherwise. Indicates the first A constraint violation degree function.

[0179] The termination criteria are as follows:

[0180]

[0181] in, This indicates the threshold for the proportion of negative emotions. Represents the threshold of group emotional entropy. This indicates the threshold for the transmission risk index. This represents the constraint satisfaction rate threshold.

[0182] If the above conditions are met, the final adjustment result is output. The final adjustment result includes at least the final joint strategy, the adjustment result of key nodes, the risk mitigation result, and the constraint satisfaction status. If the above conditions are not met, the iteration returns to steps S2 to S8.

[0183] In this invention, updating the causal contribution weights and policy network parameters may further include adaptive compensation for environmental nonstationarity. When a propagation topology abrupt change, hot topic migration, increased modality missing rate, or user behavior pattern drift is detected, the system identifies high-confidence modalities, high-confidence neighborhoods, or high-stability subgraphs based on historical stability statistics, current feedback consistency results, and prediction error results. It then increases the corresponding causal contribution weights, neighborhood attention weights, or policy input weights, while simultaneously reducing the impact of unstable signals, abnormal jump signals, or low-confidence observation signals on action decisions. This enhances the system's robustness and continuous adaptability in nonstationary environments. The output of this step can be either the final adjustment result or a control signal for continued iteration. Its function is to jointly determine the effect of emotion regulation, propagation security, and constraint satisfaction. Its beneficial effect is to ensure that the system does not simply pursue the suppression of negative emotions, but achieves overall optimization while satisfying governance boundaries and propagation security.

[0184] Corresponding to the above-described method implementation, this invention also provides a multi-agent driven network sentiment regulation system based on reinforcement learning, comprising: a data acquisition and preprocessing module, a spatiotemporal heterogeneous graph construction module, a causal sentiment representation module, a multi-agent environment modeling module, a reward and constraint modeling module, a counterfactual evaluation module, a policy training module, and a regulation execution and closed-loop update module. The data acquisition and preprocessing module performs step S1; the spatiotemporal heterogeneous graph construction module performs step S2; the causal sentiment representation module performs step S3; the multi-agent environment modeling module performs step S4; the reward and constraint modeling module performs step S5; the counterfactual evaluation module performs step S6; the policy training module performs step S7; and the regulation execution and closed-loop update module performs steps S8 and S9. Each module can be deployed on the same computing platform or in a distributed manner across multiple computing nodes, and interacts with data through message queues or interface services.

[0185] It should be understood that the above embodiments are merely preferred embodiments of the present invention, used to illustrate the technical solutions of the present invention, and not to limit the scope of protection of the present invention. For those skilled in the art, adjustments, substitutions, or combinations can be made to multimodal data types, spatiotemporal heterogeneous graph construction methods, causal attention implementation forms, counterfactual reward calculation methods, multi-agent constrained reinforcement learning algorithm types, cooperative communication mechanisms, and feedback update strategies, etc., without departing from the concept and essence of the present invention. Any technical solution formed by adopting the technical means disclosed in the present invention specification and claims and their equivalent technical features should fall within the protection scope of the present invention.

Claims

1. A method and system for emotion regulation based on reinforcement learning and multi-agent driven networks, characterized in that, Executed by the processor, including: A spatiotemporal heterogeneous interaction graph is constructed based on multi-source heterogeneous data in the target network space, and graph neural networks are used to extract graph structure features corresponding to propagation topological dependencies. The multimodal basic features, the graph structure features, the intervention context vector, and the decontamination representation vector are input into the causal attention encoding network. The causal contribution weights of each modality feature, neighborhood feature, and propagation path feature to the target emotional state are calculated to generate a causal enhanced emotional state representation. A multi-agent collaborative decision-making environment is constructed based on the causal enhanced emotional state representation, global graph state, historical joint actions and constraint states, wherein each agent corresponds to a different community, a different propagation level, a different intervention object or a different cluster of regulation strategies. A reward function is constructed that includes task benefit items, counterfactual reward items, and constraint penalty items. A counterfactual propagation environment is constructed on the spatiotemporal heterogeneous interaction graph. Propagation deductions are performed on actual joint actions and counterfactual reference actions respectively. The difference between the improvement amount of the target emotional trajectory, the amount of propagation risk suppression, or the amount of group stability improvement is used as the counterfactual reward. A multi-agent constrained reinforcement learning algorithm is adopted. Under the framework of centralized training and distributed execution, the policy network and value network of each agent are trained by combining the message passing attention mechanism between agents. The degree of satisfaction of user experience constraints, fairness constraints, propagation security constraints, platform resource constraints and intervention frequency constraints is controlled by Lagrange multiplier update, constraint cost estimation or feasible region projection, so as to obtain a joint regulation policy. Emotional regulation is performed based on the joint regulation strategy, and the spatiotemporal heterogeneous interaction graph, the causal contribution weight, the constraint state, and the strategy network parameters are updated according to the regulated emotional feedback, behavioral feedback, and propagation feedback. The updated group sentiment distribution, sentiment entropy, propagation risk index, and constraint satisfaction rate are used to determine whether the preset adjustment target has been achieved. If not, the graph structure update, causal representation update, strategy learning, and adjustment execution process continue to form a causal-driven closed-loop adaptive adjustment for the dynamic evolution of network sentiment.

2. The method according to claim 1, characterized in that: The graph neural network is a relational graph convolutional network, a graph attention network, a spatiotemporal graph neural network, a heterogeneous graph transformation network, or a combination thereof; transformation parameters are set for different types of nodes and different types of edges to distinguish user influence relationships, content propagation relationships, and topic coupling relationships.

3. The method according to claim 1, characterized in that: The causal contribution weight is determined based on the average processing effect, conditional average processing effect, or causal contribution score of the target sentiment result by the same modality features and different neighborhood features. The conventional attention weight is then adjusted according to the causal contribution score to reduce the interference of pseudo-related information, accidental co-occurrence information, or short-term noise propagation information on the sentiment state modeling.

4. The method according to claim 1, characterized in that: The causal attention weights satisfy: in, Represents a node For the Causal attention weights for each modality feature, This represents the causal attention scoring function. Represents a node The Modal feature vectors, Represents a node Intervention context vector, Represents a node The decontamination representation vector. Indicates the number of modalities; the causal enhanced emotional state indicates that it satisfies: in, Represents a node At any moment The causal-enhanced affective state is represented by... Represents a node For nodes Neighborhood causal attention weights Represents a node The set of neighboring nodes, Represents a node At any moment The graph structure features.

5. The method according to claim 1, characterized in that... The constraint function includes at least one of the following: user experience constraints based on user negative feedback rate, fairness constraints based on adjustment differences among different user groups, propagation security constraints based on amplification of abnormal propagation, platform resource constraints based on computing resources, display positions or intervention quotas, and intervention frequency constraints based on the number of interventions per unit time.

6. The method according to claim 1, characterized in that: The multi-agent constrained reinforcement learning algorithm is a constrained policy optimization algorithm, a Lagrange relaxation reinforcement learning algorithm, a multi-agent actor-critic algorithm, a value function decomposition constrained reinforcement learning algorithm, or a combination thereof; each agent shares a global constraint cost estimator or a feasible region discrimination network to improve the learning consistency of constraint boundaries during multi-agent collaborative training.

7. The method according to claim 1, characterized in that: The counterfactual reference action is at least one of the following: a zero-intervention action, an action generated by a historical baseline policy, a random perturbation action, a joint action after removing the contribution of a certain agent, or an alternative action obtained by reallocating intervention resources while keeping the total strength of the joint action unchanged; the counterfactual reward satisfies: in, Indicates time Counterfactual rewards This represents the function for evaluating the propagation effect. Indicates time The global state, Indicates time Actual joint actions This refers to a counterfactual reference action constructed while keeping the environmental state unchanged.

8. The method according to claim 1, characterized in that: The inter-agent message passing attention mechanism satisfies: in, Indicates time intelligent agent Received collaborative message vector, Indicates time intelligent agent For intelligent agents Communication attention weights, Represents the message mapping matrix. Indicates time intelligent agent The hidden state representation; Each agent updates its policy output in conjunction with the collaborative message vector and local observations to enhance the collaborative adjustment capabilities across communities, levels, and policy clusters.

9. The method according to claim 1, characterized in that: The updates to the causal contribution weights and the policy network parameters also include adaptive compensation for environmental nonstationarity; when a propagation topology mutation, hot topic migration, increased modality missing rate, or user behavior pattern drift is detected, the weights corresponding to high-confidence modalities, high-confidence neighborhoods, or high-stability subgraphs are increased, and the impact of unstable signals on action decisions is reduced.

10. A multi-agent-driven network emotion regulation system based on reinforcement learning, characterized in that, include: The spatiotemporal heterogeneous graph construction module is used to construct a spatiotemporal heterogeneous interaction graph based on multi-source heterogeneous data in the target network space, and to extract graph structure features corresponding to propagation topological dependencies using graph neural networks; The causal sentiment representation module is used to input multimodal basic features, graph structure features, intervention context vectors and decontamination representation vectors into the causal attention encoding network, calculate the causal contribution weights of each modality feature, neighborhood feature and propagation path feature to the target sentiment state, and generate a causal enhanced sentiment state representation. The multi-agent environment modeling module is used to construct a multi-agent collaborative decision-making environment based on the causal enhanced emotional state representation, global graph state, historical joint actions and constraint states. The reward and constraint modeling module is used to construct a reward function that includes task benefit items, counterfactual reward items, and constraint penalty items; the counterfactual evaluation module is used to construct a counterfactual propagation environment on the spatiotemporal heterogeneous interaction graph, perform propagation deduction on actual joint actions and counterfactual reference actions respectively, and determine counterfactual rewards. The policy training module employs a multi-agent constrained reinforcement learning algorithm, combining a centralized training and distributed execution framework with an inter-agent message passing attention mechanism to train the policy networks and value networks of each agent. It controls the degree to which user experience constraints, fairness constraints, propagation security constraints, platform resource constraints, and intervention frequency constraints are satisfied, resulting in a joint regulation policy. The regulation execution and closed-loop update module performs emotion regulation based on the joint regulation policy. It updates the spatiotemporal heterogeneous interaction graph, the causal contribution weights, the constraint states, and the policy network parameters based on the regulated emotion feedback, behavioral feedback, and propagation feedback. It also drives continued iterative operation if the preset regulation target is not reached.

Images