Social network opinion consistency method and device based on multi-agent reinforcement learning

Abstract

The application designs a social network opinion consistency method and device based on multi-agent reinforcement learning, which comprises a key neighbor selection strategy based on a limited confidence interval and an external media learning strategy based on reinforcement learning. The confidence interval between public media is adaptively adjusted according to harmony, and the learning optimization strategy is carried out through local public agent information. The confidence interval between public agents is set to a fixed value, the neighbor set of the agent i at a certain moment is obtained, the agent i selects a neighbor with the largest state difference value from itself, and then selects a neighbor with the smallest state difference value from itself. Subsequently, communication and state evolution are carried out according to the screened neighbor information and the external media information in the dynamic confidence interval. The media-public consistency protocol can adapt to large-scale high-density social network scenes and enhance the effect of opinion consistency.

Description

Technical Field

[0001] This disclosure relates to the field of consensus in multi-agent systems, and more particularly to a method and apparatus for consensus in social networks based on multi-agent reinforcement learning. Background Technology

[0002] Opinions, as a form of public opinion, are widely present in public life. With the rapid development of wireless communication networks and internet technology, using internet platforms to voice opinions and comment in online communities has gradually become a daily activity for the public. In online communities, the public uses internet platforms such as WeChat, Weibo, and QQ to express their own opinions and express individual views. The emergence of multiple viewpoints and the continuous interaction between individual viewpoints have led to different phenomena in group opinions, such as consistency, dispersion, and polarization. Currently, one of the core issues in social network public opinion management is "how to achieve consensus among the public."

[0003] Multi-agent systems are a new type of distributed control system. Compared with traditional centralized control systems with central controllers, the advantages of distributed control multi-agent systems are mainly manifested in: powerful system functions, easy system expansion, high cost performance, high reliability, high flexibility, and strong robustness.

[0004] Most existing methods for enhancing consensus among multi-agent systems are based on a combination of optimization of interaction rules, leader-following methods, trust mechanisms, and external pressure. However, in scenarios with uneven density distribution and large-scale high-density social networks, most algorithms converge too slowly in such scenarios and large-scale agent system structures, making them unsuitable. Furthermore, there is a lack of work on how to utilize external media information to enhance consensus. Summary of the Invention

[0005] The purpose of this disclosure is to overcome the problem of slow convergence of existing consensus algorithms in large-scale agent system architectures. It provides a media-public social network opinion consensus based on multi-agent reinforcement learning, which utilizes external media information and optimizes the key neighbor selection strategy based on finite confidence intervals to select more valuable neighbors for communication, thereby reducing communication costs and improving convergence speed.

[0006] This disclosure proposes a social network opinion consensus method based on multi-agent reinforcement learning. The media uses reinforcement learning to predict public opinion preferences, while the public agents comprehensively consider external media information and key neighbor information to update and evolve their states. The specific steps are as follows:

[0007] A social network opinion consensus method based on multi-agent reinforcement learning includes:

[0008] In the viewpoint dynamics model of distributed discrete-time continuous state space, at any given moment, an agent scans all public neighbor agents and stores neighbor information within a finite confidence interval;

[0009] The agent finds the agent with the largest difference from its own state in the neighbor information, and finds the agent with the smallest difference from its own state.

[0010] The agent calculates the harmony degree based on the information of any neighbor within all finite confidence intervals;

[0011] Using reinforcement learning, we can determine whether a public intelligent agent is capable of receiving information from external media.

[0012] The public intelligent agent exchanges information with the filtered intelligent agent information, obtains control input through a consensus protocol, and updates its own state information by controlling the input amount. It can effectively combine the information of all intelligent agent neighbors and external media information to achieve consensus on social network views.

[0013] The intelligent agent information includes information about public intelligent agents and external media intelligent agents.

[0014] The method of using reinforcement learning to determine whether a public intelligent agent can accept external media information includes:

[0015] The number of media outlets is fixed at 1. In each iteration of the public agent, a portion of public information is randomly selected by the media for reinforcement learning. After training, a value is randomly selected according to its normal probability distribution as new external media information. ,like This indicates that the public intelligent agent is capable of receiving information from external media. ; The viewpoint information of public intelligent agent i at time k.

[0016] A multi-agent system is a state-dependent topology. The agent's viewpoint information represents its state information. Considering only the agent's one-dimensional viewpoint information, the agent's viewpoint at the next moment is jointly determined by the views of its neighbors within its confidence interval and the views of external media information. Its state information at the next moment can be represented as:

[0017] ; This refers to the state information of agent i at time k; Information for external media.

[0018] Neighbor Information The definition is: for public intelligent agents i and j, if - If r is the length of the confidence interval of the agent, then agents i and j are neighbors.

[0019] Regarding whether the public intelligent agent i can accept information from external media. depending on - Whether it is valid, This represents the harmony level of agent i. If true, it means that agent i can accept external media information.

[0020] The consensus protocol is as follows:

[0021] ;

[0022] in, , The adjustment factor is a number greater than 0 and less than 1. This indicates whether the public intelligent agent i has a connection with external media.

[0023] The communication topology between the public intelligent agent and external media is also time-varying. It means that if If the value is 1, then the value is 1; otherwise, the value is 0.

[0024] The conditions under which public intelligent agents reach a consensus are:

[0025] For public intelligent agents i and j, if the condition is met... - =0, then consensus is reached, that is, the task of reaching a consensus has been completed.

[0026] Harmony is a metric that measures the magnitude of the differences in viewpoints between public agent i and all its neighbors, and is defined as follows: ;

[0027] Where n is the number of people in the public. and These represent the viewpoints of public intelligent agents i and j, respectively.

[0028] An electronic device for achieving consensus on social network opinions, comprising:

[0029] Storage medium used to store computer programs;

[0030] The processing unit exchanges data with the storage medium and, when performing opinion consensus processing, executes the computer program through the processing unit to perform the steps of the social network opinion consensus method based on multi-agent reinforcement learning as described above.

[0031] A readable storage medium, characterized in that:

[0032] The readable storage medium stores a computer program;

[0033] When the computer program is run, it executes the steps of the social network opinion consensus method based on multi-agent reinforcement learning as described above.

[0034] Beneficial effects:

[0035] The beneficial effects of this disclosure include at least the following:

[0036] The social network opinion consensus method based on multi-agent reinforcement learning disclosed herein utilizes a social network opinion consensus algorithm based on external media information, while simultaneously optimizing a key neighbor selection strategy based on finite confidence intervals to reduce redundant information, thereby rapidly achieving consensus in a distributed multi-agent system. Attached Figure Description

[0037] Figure 1 This is a graph of a neighborhood selection model based on finite confidence intervals. White represents agent i, and black represents agent i's neighbors within its confidence interval. Red represents the neighbors outside the confidence interval of agent i.

[0038] Figure 2 This is a diagram of a media selection model based on dynamic confidence intervals. Green represents external media agents, black represents public agents within the dynamic confidence interval, and red represents public agents outside the dynamic confidence interval.

[0039] Figure 3 This is a diagram showing the selected neighbors. Detailed Implementation

[0040] Various embodiments of this disclosure will be described more fully below. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0041] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of features, numbers, steps, operations, elements, components, or combinations of the foregoing.

[0042] In various embodiments of this disclosure, the expression "or" or "at least one of A and / or B" includes any combination or all combinations of the words listed simultaneously. For example, the expression "A or B" or "at least one of A and / or B" may include A, may include B, or may include both A and B.

[0043] The terms used in the various embodiments of this disclosure (such as "first," "second," etc.) may modify various components in the various embodiments, but do not limit the corresponding components. For example, the above terms do not limit the order and / or importance of the components. The above terms are only used for the purpose of distinguishing one component from others. For example, a first user device and a second user device refer to different user devices, although both are user devices. For example, a first component may be referred to as a second component without departing from the scope of the various embodiments of this disclosure, and similarly, a second component may also be referred to as a first component.

[0044] It should be noted that if a description is made of "connecting" one component to another, then the first component can be directly connected to the second component, and a third component can be "connected" between the first and second components. Conversely, when a component is "directly connected" to another component, it can be understood that there is no third component between the first and second components.

[0045] The term "user" as used in various embodiments of this disclosure may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0046] The terminology used in the various embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the various embodiments of this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this disclosure pertain. Terms (such as those defined in a generally used dictionary) are to be interpreted as having the same meaning as in the context of the relevant technical field and are not to be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this disclosure. Specific Implementation Example 1:

[0048] A social network opinion consensus method based on multi-agent reinforcement learning can effectively combine information from all agent neighbors and external media information. The specific steps are as follows:

[0049] Step 1: As Figure 1In the viewpoint dynamics model of distributed discrete-time continuous state space, at time k, agent i scans all public neighbor agents and stores the neighbor information located within a finite confidence interval. ;

[0050] Step 2: Agent i in the neighborhood Find the agent with the largest difference from its own state, and denote it as... Find the agent whose state is least different from its own, and denote it as... intelligent agent i to The state difference is denoted as intelligent agent i to The state difference is denoted as ;

[0051] Step 3: Agent i bases its actions on the neighbor information within all finite confidence intervals. Calculate the harmony degree HD. Harmony degree is a metric that measures the difference in opinions between public agent i and all its neighbors, and is defined as follows:

[0052] ;

[0053] Where n is the number of people in the public. and These represent the viewpoints of public intelligent agents i and j, respectively.

[0054] Step Four: As Figure 2-3 The number of media outlets is fixed at 1. Each iteration of the public agent i involves randomly selecting ℎ% of the public's information for reinforcement learning. After y rounds of training, the information is distributed according to its normal probability distribution. Randomly select a value as the new media information ,like This indicates that public intelligent agents can receive information from external media. ;

[0055] Step 5: Public agent i exchanges information with the filtered agent information (public agent and external media agent), obtains a control input through a consensus protocol, and updates its own state information through the control input.

[0056] The network communication topology is time-varying, represented by an adjacency matrix D. Under topology switching, D is represented by D(k). The communication topology connection between any public agent i and the external media is also time-varying, using... It means that if If the value is 1, then the value is 1; otherwise, it is 0. A multi-agent system is a state-dependent topology. The agent's viewpoint information represents its state information. Considering only the agent's one-dimensional viewpoint information, the agent's viewpoint at the next moment is jointly determined by the views of its neighbors within its confidence interval and the external media information. Its state information at the next moment can be represented as follows, and its consensus protocol can be represented as follows: ;Neighbor The definition of comparability is expressed as follows: for any public intelligent agents i, j, if - Let r be the length of the confidence interval for the agents, then agents i and j are neighbors. However, the question remains whether any public agent i can receive external media information. depending on - Whether it is valid, This represents the harmony level of agent i. If true, it means that agent i can accept external media information.

[0057] According to the consensus protocol, the state information of agent i is updated based on the state information of its neighbors and external media information. All public agents achieve consensus based on the state information of their neighbors and external media information. The consensus protocol is as follows:

[0058] ;

[0059] in, , The adjustment factor is a number greater than 0 and less than 1. This indicates whether the public intelligent agent i has a connection with external media. If the condition is met, the value is 1; otherwise, it is 0. The condition for agents to reach consensus is that for any public agents i and j, if the condition is met... - If the value is 0, then we say that the system has reached consensus, that is, the task of achieving consensus has been completed.

[0060] By taking external conditions (media) into account and optimizing the key neighbor selection strategy, the communication cost of each agent is reduced, which can accelerate the convergence speed of the system and thus enhance consistency.

[0061] In a multi-agent system, the confidence interval of each agent is finite. Let the length of the finite confidence interval between public agents be defined as... The dynamic confidence interval between public media is set as follows: The neighbor set of public agent i at time k is represented as: Agent i selects a neighbor with the largest difference from its own state, and records the difference as follows: Then select the neighbor with the smallest difference from its own state, and denote it as... Therefore, the control input protocol from neighbors for public intelligent agent i is: , where p is the identifier of the neighboring public intelligent agent. And the information from the external media at time k is represented as... The corresponding control input protocol from external media is: - , where m is the identifier for external media information.

[0062] The specific inputs for the method described in this embodiment are as follows:

[0063] Algorithm 1: Media-Public Consensus Algorithm (MPOC): Input: Number of public agents N, finite confidence interval The media training process consists of M training rounds, T maximum time steps, and ζ learning rate. The steps are: a) for episode = 1:M; b) randomly selecting ℎ percent of the public's information for learning and calculating immediate rewards. c) Through instant rewards Optimization strategy for dynamic learning rate (ζd) e) end forf) Media based on optimized strategy Randomly select one as media information g) Each public intelligent agent i according to Calculated h) Public intelligent agents based on calculate and And update the neighbor set of the selected public agents as follows i) The public intelligent agent calculates its own harmony degree HD. j) Calculates based on HD. Decide whether to accept information output from external media: , .

[0064] The neighbor set obtained by Algorithm 1 and external media information Incorporating consensus protocols

[0065]

[0066] Perform state evolution.

[0067] Compared with existing research algorithms, the advantages of this disclosure are that it considers external media information, optimizes the key neighbor selection strategy based on finite confidence intervals, reduces redundant information, reduces the communication cost of agents, accelerates the system convergence speed, and enhances consistency to a certain extent.

[0068] The experimental method disclosed herein is simple in procedure, considers only a single external media information and does not require communication with all neighboring agents, but only with the given valuable neighbors, thereby reducing the communication cost of the system and improving the convergence speed. Specific Implementation Example 2:

[0070] This disclosure also provides an embodiment:

[0071] An electronic device for social network opinion consensus includes: a storage medium and a processing unit; wherein the storage medium is used to store a computer program; the processing unit exchanges data with the storage medium and, when performing opinion consensus processing, executes the computer program through the processing unit to perform the steps of the social network opinion consensus method based on multi-agent reinforcement learning as described in Specific Embodiment 1.

[0072] The CPU described above can perform various appropriate actions and processes according to the program stored in the storage medium. The electronic device also includes the following peripherals, including input sections such as a keyboard and mouse, and output sections such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker.

[0073] This disclosure also provides an embodiment:

[0074] A readable storage medium: the readable storage medium stores a computer program; when the computer program is run, it executes the steps of the social network opinion consensus method based on multi-agent reinforcement learning as described in Specific Embodiment 1.

[0075] In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0076] The above disclosures only cover a few specific implementation scenarios. However, this disclosure is not limited to these, and any variations that can be conceived by those skilled in the art should fall within the protection scope of this disclosure. The serial numbers in this disclosure are for descriptive purposes only and do not represent the superiority or inferiority of the implementation scenarios.

Claims

1. A social network opinion consensus method based on multi-agent reinforcement learning, characterized in that, include: In the viewpoint dynamics model of distributed discrete-time continuous state space, at any given moment, an agent scans all public neighbor agents and stores neighbor information within a finite confidence interval; The agent finds the agent with the largest difference from its own state in the neighbor information, and finds the agent with the smallest difference from its own state. The agent calculates the harmony degree based on the information of any neighbor within all finite confidence intervals; Using reinforcement learning, we can determine whether a public intelligent agent is capable of receiving information from external media. The public intelligent agent exchanges information with the filtered intelligent agent information, obtains control input through a consensus protocol, and updates its own state information by controlling the input amount. It can effectively combine the information of all intelligent agent neighbors and external media information to achieve consensus on social network views. The intelligent agent information includes information about public intelligent agents and external media intelligent agents; The method of using reinforcement learning to determine whether a public intelligent agent can accept external media information includes: The number of media outlets is fixed at 1. In each iteration of the public agent, a portion of public information is randomly selected by the media for reinforcement learning. After training, a value is randomly selected according to its normal probability distribution as new external media information. ,like This indicates that the public intelligent agent is capable of receiving information from external media. ; The viewpoint information of public agent i at time k; HD represents the harmony degree; The harmony degree HD is an indicator that measures the magnitude of the difference in viewpoints between public agent i and all its neighbors, and is defined as follows: ; Where n is the number of people in the public. and These represent the viewpoints of public intelligent agents i and j, respectively.

2. The social network opinion consensus method based on multi-agent reinforcement learning according to claim 1, characterized in that: A multi-agent system is a state-dependent topology. An agent's viewpoint information represents its state information. Considering only the agent's one-dimensional viewpoint information, the agent's viewpoint at the next moment is jointly determined by the views of its neighbors within its confidence interval and the views of external media information. Its state information at the next moment is represented as: ； in, This refers to the state information of agent i at time k; Information for external media.

3. The social network opinion consensus method based on multi-agent reinforcement learning according to claim 1, characterized in that: Neighbor Information The definition is: for public intelligent agents i and j, if - If r is the length of the confidence interval of the agent, then agents i and j are neighbors. Regarding whether the public intelligent agent i can accept information from external media. depending on - Whether it is valid, This represents the harmony level of agent i. If true, it means that agent i can accept external media information.

4. The social network opinion consensus method based on multi-agent reinforcement learning according to claim 1, characterized in that: The consensus protocol is as follows: ； in, , The adjustment factor is a number greater than 0 and less than 1. This indicates whether the public intelligent agent i has a connection with external media.

5. The social network opinion consensus method based on multi-agent reinforcement learning according to claim 4, characterized in that: The communication topology between the public intelligent agent and external media is also time-varying. It means that if If it is 1, then it is 1; otherwise, it is 0. HD represents harmony.

6. The social network opinion consensus method based on multi-agent reinforcement learning according to claim 1, characterized in that: The conditions under which public intelligent agents reach a consensus are: For public intelligent agents i and j, if the condition is met... - =0, then consensus is reached, that is, the task of reaching a consensus has been completed.

7. An electronic device for achieving consensus on social network opinions, characterized in that, include: Storage medium used to store computer programs; The processing unit exchanges data with the storage medium and is used to execute the computer program during opinion consensus processing to perform the steps of the social network opinion consensus method based on multi-agent reinforcement learning as described in any one of claims 1-6.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program; When the computer program is run, it executes the steps of the social network opinion consensus method based on multi-agent reinforcement learning as described in any one of claims 1-6.

Images