Social network viewpoint consistency method and device based on multi-agent reinforcement learning

Abstract

The invention designs a social network viewpoint consistency method and device based on multi-agent reinforcement learning. The device comprises a key neighbor selection strategy based on a limited confidence interval and an external media learning strategy based on reinforcement learning. The confidence interval between the public media is adaptively adjusted according to the harmony degree, and an optimization strategy is learned through local public agent information. The confidence interval between the public agents is set to be a fixed value, a neighbor set of the agent i at a certain moment is obtained, and the agent i selects a neighbor with the maximum state difference value with the agent i and then selects a neighbor with the minimum state difference value with the agent i. And performing communication and state evolution according to the screened neighbor information and the external media information in the dynamic confidence interval. The media-public consistency protocol can adapt to a large-scale high-density social network scene, and the effect of enhancing viewpoint consistency is achieved.

Description

Technical Field

[0001] The present disclosure relates to the field of multi-agent system consistency, and in particular to a social network opinion consistency method and device based on multi-agent reinforcement learning. Background Art

[0002] Opinions, as a form of public opinion, are widely present in public life. With the rapid development of wireless communication networks and Internet technology, using Internet platforms to express opinions and make comments in online communities has gradually become a daily activity for the public. In online communities, the public uses Internet platforms such as WeChat, Weibo, and QQ to express their own opinions and express their individual views. The emergence of multiple opinions and the continuous interaction of opinions between individuals have gradually led to different phenomena such as consistency, dispersion, and polarization in group opinions. At present, one of the core issues of social network public opinion management is "how to make the public reach a consensus."

[0003] The multi-agent system is a new type of distributed control system. Compared with the traditional centralized control system with a central controller, the advantages of the distributed control multi-agent system are mainly as follows: powerful system functions, easy system expansion, high cost performance, high reliability, high flexibility and strong robustness.

[0004] Most of the existing methods for enhancing the consistency of multi-agent opinions are based on a combination of optimized interaction rules, leader-following methods, trust mechanisms, external pressure, etc. However, in scenarios of uneven density distribution and large-scale high-density social networks, most algorithms converge too slowly under the large-scale intelligent agent system structure in the above scenarios and are not applicable. There is also little work on how to use external media information to achieve consistency enhancement. Summary of the invention

[0005] The purpose of the present invention is to overcome the problem that the existing consensus algorithm converges too slowly under the large-scale intelligent system structure, and provide a media-public social network opinion consistency based on multi-agent reinforcement learning, which utilizes external media information and optimizes the key neighbor selection strategy based on a finite confidence interval to select more valuable neighbors for communication, thereby reducing the communication cost and improving the convergence speed.

[0006] The social network opinion consistency method based on multi-agent reinforcement learning proposed in this disclosure predicts the public group opinion preferences through reinforcement learning, while the public agent comprehensively considers external media information and key neighbor information to update and evolve the state. The specific steps are as follows:

[0007] A social network opinion consensus method based on multi-agent reinforcement learning, including:

[0008] Under the distributed discrete-time continuous state space opinion dynamics model, the agent scans all public neighbor agents at any time and stores the neighbor information within a limited confidence interval;

[0009] The agent finds an agent with the largest state difference from its own state in the neighbor information, and finds an agent with the smallest state difference from its own state;

[0010] The intelligent agent calculates harmony according to any neighbor information within all limited confidence intervals;

[0011] Using reinforcement learning, we can determine whether public agents can accept external media information;

[0012] The public agent exchanges information with the screened agent information, obtains control input through a consistency protocol, updates its own state information by controlling the input amount, and can effectively combine the information of all agent neighbors and external media information to reach a consensus on social network views;

[0013] The agent information includes information of public agents and external media agents.

[0014] The method of using reinforcement learning to determine whether the public agent can accept external media information includes:

[0015] The preset number of media is fixed to 1. Each time the public agent iterates, the media randomly selects some public information for reinforcement learning. After training, a value is randomly selected according to its normal probability distribution as the new external media information x m (k), if |x m (k)-x i (k)|≤HD means that the public agent can accept external media information x m (k); x i (k) is the opinion information of public agent i at time k.

[0016] The multi-agent system is a state-dependent topological structure. The agent's view information represents its state information. Only the one-dimensional view information of the agent is considered. The agent's view at the next moment is determined by the views of its neighbors within the confidence interval and the external media information. Its state information at the next moment can be expressed as: i (k+1)=x i (k)+u i (k); x i (k) is the state information of agent i at time k; u i (k) is external media information.

[0017] Neighbor Information N i is defined as: for public agents i and j, if ||xi -x j ||≤r, r is the length of the agent's confidence interval, then agents i and j are neighbors;

[0018] As for whether the public agent i can accept external media information x m (k) depends on ||x i -x m ||≤HD i Is it established? HD i It represents the harmony degree of agent i. If it holds, it means that agent i can accept external media information.

[0019] The consistency protocol is:

[0020]

[0021] Among them, α, β are adjustment factors, which are numbers greater than 0 and less than 1, and b im Indicates whether public agent i is connected to external media.

[0022] The communication topology connection between the public intelligent agent and the external media is also time-varying, using b im It means that if |x m (k)-x i (k)|≤HD, then it is 1, otherwise it is 0;

[0023] The conditions for the public agents to reach consensus are:

[0024] For public agents i and j, if the condition lim is satisfied x→∞ ||x i -x j ||=0, then consensus is reached, that is, the task of reaching consensus is completed.

[0025] Harmony is an indicator that measures the difference in opinions between public agent i and all its neighbors, and is defined as follows:

[0026]

[0027] Where n is the number of the public, x i and x j Represent the opinion information of public agent i and public agent j respectively.

[0028] An electronic device for social network opinion consistency, comprising:

[0029] Storage medium for storing computer programs;

[0030] A processing unit exchanges data with the storage medium, and is used to execute the computer program through the processing unit when performing three-dimensional ultrasound imaging to perform the steps of the social network opinion consistency method based on multi-agent reinforcement learning as described above.

[0031] A readable storage medium, characterized in that:

[0032] The readable storage medium stores a computer program;

[0033] When the computer program is running, it executes the steps of the social network opinion consistency method based on multi-agent reinforcement learning as described above.

[0034] Beneficial effects:

[0035] The beneficial effects of the present disclosure include at least:

[0036] The social network opinion consistency method based on multi-agent reinforcement learning described in the present invention utilizes the social network opinion consistency algorithm of external media information, and simultaneously optimizes the key neighbor selection strategy based on finite confidence intervals to reduce redundant information, thereby quickly achieving the consistency of the distributed multi-agent system. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a diagram of the neighborhood selection model based on a finite confidence interval. The white one is agent i, and the black one is the neighbors N within the confidence interval of agent i. i (k), the red ones are neighbors outside the confidence interval of agent i.

[0038] Figure 2 This is a diagram of the media selection model based on dynamic confidence intervals. Green represents external media agents, black represents public agents within the dynamic confidence interval, and red represents public agents outside the dynamic confidence interval.

[0039] Figure 3 Schematic diagram of the selected neighbors. DETAILED DESCRIPTION

[0040] In the following, various embodiments of the present disclosure will be described more fully. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of the present disclosure.

[0041] Hereinafter, the terms "include" or "may include" used in various embodiments of the present disclosure indicate the presence of disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. In addition, as used in various embodiments of the present disclosure, the terms "include", "have", and their cognates are intended only to indicate specific features, numbers, steps, operations, elements, components, or a combination of the foregoing, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, elements, components, or a combination of the foregoing or the possibility of adding one or more features, numbers, steps, operations, elements, components, or a combination of the foregoing.

[0042] In various embodiments of the present disclosure, the expression "or" or "at least one of A or / and B" includes any combination or all combinations of the words listed at the same time. For example, the expression "A or B" or "at least one of A or / and B" may include A, may include B, or may include both A and B.

[0043] The expressions (such as "first", "second", etc.) used in the various embodiments of the present disclosure may modify the various constituent elements in the various embodiments, but may not limit the corresponding constituent elements. For example, the above expressions do not limit the order and / or importance of the elements. The above expressions are only used for the purpose of distinguishing one element from other elements. For example, the first user device and the second user device indicate different user devices, although both are user devices. For example, without departing from the scope of the various embodiments of the present disclosure, the first element may be referred to as the second element, and similarly, the second element may also be referred to as the first element.

[0044] It should be noted that if it is described that one component element is “connected” to another component element, the first component element may be directly connected to the second component element, and a third component element may be “connected” between the first component element and the second component element. Conversely, when one component element is “directly connected” to another component element, it can be understood that there is no third component element between the first component element and the second component element.

[0045] The term “user” used in various embodiments of the present disclosure may indicate a person using an electronic device or a device (eg, an artificial intelligence electronic device) using the electronic device.

[0046] The terms used in the various embodiments of the present disclosure are only used for the purpose of describing specific embodiments and are not intended to limit the various embodiments of the present disclosure. As used herein, the singular form is intended to also include the plural form, unless the context clearly indicates otherwise. Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the meanings commonly understood by ordinary technicians in the field to which the various embodiments of the present disclosure belong. The terms (such as the terms defined in the generally used dictionary) will be interpreted as having the same meaning as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning, unless clearly defined in the various embodiments of the present disclosure. Specific embodiment 1:

[0048] A social network opinion consensus method based on multi-agent reinforcement learning can effectively combine the information of all agent neighbors and external media information. The specific steps are as follows:

[0049] Step 1: If Figure 1 In the distributed discrete-time continuous state space viewpoint dynamics model, at time k, agent i scans all public neighbor agents and stores the neighbor information N within a limited confidence interval. i (k);

[0050] Step 2: Agent i in Neighborhood N i (k) Find the agent with the largest difference from its own state, denoted as x max (k), find the agent with the smallest state difference from its own state, denoted as x min (k), agent i to x min The state difference of (k) is denoted as r min , agent i to x max The state difference of (k) is denoted as r max ;

[0051] Step 3: Agent i is based on all neighbor information N within the limited confidence interval i (k) Calculate the harmony degree HD, which is an indicator to measure the difference in opinions between public agent i and all its neighbors, and is defined as follows:

[0052]

[0053] Where n is the number of the public, x i and x j Represent the opinion information of public agent i and public agent j respectively.

[0054] Step 4: If Figure 2-3, the number of media is fixed to 1. For each iteration of public agent i, the media randomly selects h percent of the public information for reinforcement learning. After y rounds of training, according to its normal probability distribution N(u k ,σ k ) Randomly select a value as the new media information x m (k), if |x m (k)-x i (k)|≤HD means that the public agent can receive external media information x m (k);

[0055] Step 5: Public agent i exchanges information with the screened agent information (public agents and external media agents), obtains a control input through the consistency protocol, and updates its own state information through the control input amount.

[0056] The communication topology of the network is time-varying. The adjacency matrix D is used to represent the topology. Under the switching topology, D is represented by D(k). The communication topology connection between any public intelligent agent i and the external media is also time-varying. im It means that if |x m (k)-x i (k)|≤HD, then it is 1, otherwise it is 0; the multi-agent system is a state-dependent topological structure. The agent's view information represents its state information. Only the one-dimensional view information of the agent is considered. The agent's view at the next moment is determined by the views of its neighbors in the confidence interval and the external media information. Its state information at the next moment can be expressed as, and its consistency protocol can be expressed as: i (k+1)=x i (k)+u i (k); Neighbor N i The definition of can be expressed as for any public agent i, j, if ||x i -x j ||≤r, r is the length of the agent's confidence interval, then agents i and j are neighbors. For any public agent i, can it accept external media information x m (k) depends on ||x i -x m ||≤HD i Is it established? HD i It represents the harmony degree of agent i. If it holds, it means that agent i can accept external media information.

[0057] According to the consistency protocol, the state information of agent i is updated according to the state information of its neighbors and external media information. All public agents reach a consensus based on the state information of their neighbors and external media information. The consistency protocol is:

[0058]

[0059] Among them, α, β are adjustment factors, which are numbers greater than 0 and less than 1, and b im Indicates whether public agent i is connected to external media. If |x m (k)-x i (k)|≤HD, then it is 1, otherwise it is 0; when the agent reaches a consensus, for any public agent i, j, if the condition lim is satisfied x→∞ ||x i -x j ||=0, then we say that the system has reached consensus, that is, it has completed the task of reaching consensus.

[0060] By taking external conditions (media) into consideration and optimizing the key neighbor selection strategy, the communication cost of each agent is reduced, which can speed up the convergence of the system and enhance consistency.

[0061] The confidence interval of each agent in the multi-agent system is finite. The length of the finite confidence interval between public agents is set to r p , the dynamic confidence interval between public media is set to r m The neighbor set of public agent i at time k is represented by N i (k), agent i selects a neighbor with the largest state difference from its own state, and the difference is recorded as max(N i (k)), and then select a neighbor with the smallest state difference from its own, denoted as min(N i (k)), therefore, the control input protocol of public agent i from its neighbors is: Where p is the identifier of the neighboring public agent. The information of the external media at time k is represented by x m (k), the corresponding control input protocol from external media is: Wherein m is the identifier of the external media information.

[0062] The specific input and output of the method described in this embodiment are as follows:

[0063]

[0064]

[0065] The neighbor set Φ obtained by Algorithm 1 i (k) and external media information x m (k) Introducing the consistency protocol

[0066]

[0067] Evolving state.

[0068] Compared with existing research algorithms, the advantage of the present invention is that it takes external media information into consideration and optimizes the key neighbor selection strategy based on a finite confidence interval, thereby reducing redundant information, reducing the communication cost of the intelligent agent, accelerating the system convergence speed, and enhancing consistency to a certain extent.

[0069] The steps of the disclosed experimental method are concise, only a single external media information is considered and there is no need to communicate with all neighboring intelligent agents. Instead, only the given valuable neighbors need to be communicated with, which reduces the communication cost of the system and improves the convergence speed. Specific embodiment 2:

[0071] The present disclosure also provides an embodiment:

[0072] An electronic device for social network opinion consistency, comprising: a storage medium and a processing unit; wherein the storage medium is used to store a computer program; the processing unit exchanges data with the storage medium, and is used to execute the computer program through the processing unit when performing three-dimensional ultrasound imaging, and perform the steps of the social network opinion consistency method based on multi-agent reinforcement learning as described in specific embodiment 1.

[0073] The CPU can perform various appropriate actions and processes according to the program stored in the storage medium. The electronic device also includes the following peripherals, including input parts such as keyboards and mice, and output parts such as cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers.

[0074] The present disclosure also provides an embodiment:

[0075] A readable storage medium: the readable storage medium stores a computer program; when the computer program is running, the steps of the social network opinion consistency method based on multi-agent reinforcement learning as described in specific embodiment 1 are executed.

[0076] In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, device or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0077] The above disclosures are only several specific implementation scenarios of the present disclosure, but the present disclosure is not limited thereto, and any changes that can be thought of by a person skilled in the art should fall within the protection scope of the present disclosure. The above disclosure serial numbers are only for description and do not represent the advantages and disadvantages of the implementation scenarios.

Claims

1. A social network opinion consistency method based on multi-agent reinforcement learning, characterized in that: include: Under the distributed discrete-time continuous state space opinion dynamics model, the agent scans all public neighbor agents at any time and stores the neighbor information within a limited confidence interval; The agent finds an agent with the largest state difference from its own state in the neighbor information, and finds an agent with the smallest state difference from its own state; The intelligent agent calculates harmony according to any neighbor information within all limited confidence intervals; Using reinforcement learning, we can determine whether public agents can accept external media information; The public agent exchanges information with the screened agent information, obtains control input through a consistency protocol, updates its own state information by controlling the input amount, and can effectively combine the information of all agent neighbors and external media information to reach a consensus on social network views; The agent information includes information of public agents and external media agents.

2. The social network opinion consistency method based on multi-agent reinforcement learning according to claim 1 is characterized in that: The method of using reinforcement learning to determine whether the public agent can accept external media information includes: The preset number of media is fixed to 1. Each time the public agent iterates, the media randomly selects some public information for reinforcement learning. After training, a value is randomly selected according to its normal probability distribution as the new external media information x m (k), if |x m (k)-x i (k)|≤HD means that the public agent can accept external media information x m (k); x i (k) is the opinion information of public agent i at time k.

3. The social network opinion consistency method based on multi-agent reinforcement learning according to claim 1, characterized in that: The multi-agent system is a state-dependent topological structure. The agent's view information represents its state information. Only the one-dimensional view information of the agent is considered. The agent's view at the next moment is determined by the views of its neighbors within the confidence interval and the external media information. Its state information at the next moment can be expressed as: i (k+1)=x i (k)+u i (k); x i (k) is the state information of agent i at time k; u i (k) is external media information.

4. The social network opinion consistency method based on multi-agent reinforcement learning according to claim 1, characterized in that: Neighbor Information N i is defined as: for public agents i and j, if ||x i -x j ||≤r, r is the length of the agent's confidence interval, then agents i and j are neighbors; As for whether the public agent i can accept external media information x m (k) depends on ||x i -x m ||≤HD i Is it true, HD i It represents the harmony degree of agent i. If it holds, it means that agent i can accept external media information.

5. The social network opinion consistency method based on multi-agent reinforcement learning according to claim 1, characterized in that: The consistency protocol is: Among them, α, β are adjustment factors, which are numbers greater than 0 and less than 1, and b im Indicates whether public agent i is connected to external media.

6. The social network opinion consistency method based on multi-agent reinforcement learning according to claim 5 is characterized by: The communication topology connection between the public intelligent agent and the external media is also time-varying, using b im It means that if |x m (k)-x i (k)|≤HD, then it is 1, otherwise it is 0.

7. The method for social network opinion consistency based on multi-agent reinforcement learning according to claim 1, characterized in that: The conditions for the public ng agent to reach consensus are: For public agents i and j, if the condition lim is satisfied x→∞ ||x i -x j ||=0, then consensus is reached, that is, the task of reaching consensus is completed.

8. The social network opinion consistency method based on multi-agent reinforcement learning according to claim 1, characterized in that: Harmony is an indicator that measures the difference in opinions between public agent i and all its neighbors, and is defined as follows: Where n is the number of the public, x i and x j Represent the opinion information of public agent i and public agent j respectively.

9. An electronic device for social network opinion consistency, characterized in that: include: Storage medium for storing computer programs; A processing unit that exchanges data with the storage medium, and is used to execute the computer program through the processing unit when performing three-dimensional ultrasound imaging to perform the steps of the social network opinion consistency method based on multi-agent reinforcement learning as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program; When the computer program is running, it executes the steps of the social network opinion consistency method based on multi-agent reinforcement learning as described in any one of claims 1 to 8.

Images