Communication management method and device, equipment, storage medium and computer program product

Abstract

The invention discloses a communication management method. The method comprises the following steps: determining a first state parameter and a first environment parameter sensed by a first intelligent agent; wherein the first state parameter is an operation parameter of the first intelligent agent; determining a return value and a dynamic communication threshold value corresponding to the first intelligent agent based on the first state parameter and the first environment parameter; determining a trigger operation based on the return value and the dynamic communication threshold; wherein the triggering operation is used for triggering whether the first intelligent agent communicates with at least one second intelligent agent or not; and executing the trigger operation. The invention further discloses a communication management device and equipment, a storage medium and a computer program product.

Description

Technical Field

[0001] The present application relates to the field of wireless communication technology, and in particular to a communication management method, apparatus, device, storage medium, and computer program product. Background Art

[0002] With the rapid development of wireless communication technology, its application is becoming increasingly widespread. Most intelligent agents can now exchange information through wireless communication. In other words, most multi-agent collaboration relies on wireless transmission to share observations. Currently, multi-agent collaboration using wireless communication to share observations is primarily achieved through time-triggered communication. Specifically, this process uses a constant period as the communication control frequency, allowing for fully connected communication interactions to be performed on schedule. The sampling period is designed based on a worst-case scenario, and all sampled signals are transmitted over the network.

[0003] However, based on the aforementioned multi-agent communication method, as the number of agents and the amount of data increases, the communication frequency of each agent increases, which can easily lead to problems such as packet loss and increased communication delays, shortened battery life, and reduced performance of the multi-agent system. Minimizing unnecessary interactions between agents and ensuring efficient communication between them has become a pressing technical challenge.

[0004] Application Contents

[0005] In order to solve the above technical problems, the present application hopes to provide a communication management method, device, equipment, storage medium and computer program product, which solves the current problem of low communication efficiency between multiple intelligent agents, proposes a method for realizing communication between multiple intelligent agents, reduces the probability of packet loss and communication delay when communicating between multiple intelligent agents, and ensures the communication efficiency between multiple intelligent agents.

[0006] The technical solution of this application is achieved as follows:

[0007] This embodiment of the present application provides a communication management method, the method comprising:

[0008] Determining a first state parameter and a first environment parameter perceived by a first agent; wherein the first state parameter is a parameter of the operation of the first agent;

[0009] Determining a reward value and a dynamic communication threshold corresponding to the first agent based on the first state parameter and the first environment parameter;

[0010] Determining a trigger operation based on the reward value and the dynamic communication threshold; wherein the trigger operation is used to trigger whether the first agent communicates with at least one second agent;

[0011] The trigger operation is executed.

[0012] In the above solution, determining the first state parameter and the first environment parameter perceived by the first agent includes:

[0013] Perceiving a second state parameter and a second environment parameter of the first agent;

[0014] The second state parameter and the second environment parameter are preprocessed to obtain the first state parameter and the first environment parameter.

[0015] In the above solution, determining the reward value and dynamic communication threshold corresponding to the first agent based on the first state parameter and the first environment parameter includes:

[0016] Determining the reward value based on the analysis performance parameter included in the first state parameter and the device performance parameter of the first agent, and the wireless link characteristic parameter included in the first environment parameter;

[0017] determining a target probability of the first agent sending or receiving a signal;

[0018] The dynamic communication threshold is determined based on the occupied bandwidth parameter, the target probability, the calculation delay parameter, the analysis performance parameter and the device performance parameter included in the first state parameter, and the position distribution parameters of at least one perceived third agent and the first agent, the number of perceived agents and the connection status parameters with at least one said third agent included in the first environmental parameter; wherein at least one of the second agents belongs to at least one said third agent.

[0019] In the above solution, determining the triggering operation based on the reward value and the dynamic communication threshold includes:

[0020] If the reward value is less than the dynamic communication threshold, determining that the triggering operation is an information receiving operation;

[0021] If the reward value is greater than or equal to the dynamic communication threshold, the triggering operation is determined to be a message sending operation.

[0022] In the above solution, executing the trigger operation includes:

[0023] Determining at least one of said second agents from at least one third agent;

[0024] establishing a communication link with each of the second agents;

[0025] Based on each of the communication links, a communication operation with each of the second intelligent agents is performed; wherein the communication operation includes an information sending operation and / or an information receiving operation.

[0026] In the above solution, the step of establishing a communication link with each second agent includes:

[0027] After establishing a physical connection with each second agent, performing LCP link authentication with the corresponding second agent through a Logical Link Control Protocol (LCP) configuration request in a preset frame format to obtain an authentication result;

[0028] If the authentication result is authentication passed, network layer protocol configuration docking is performed through the network layer control protocol NCP configuration request in the preset format to establish a communication link.

[0029] In the above scheme, the preset frame format includes at least: a header, an information payload and a tail; wherein the header includes: a flag field indicating the start of the frame with a first preset identifier, an address field with a second preset identifier, a control field with a third preset identifier and a protocol field for identifying the protocol type, the information payload is an information field of a preset length, and the tail includes: a frame check sequence and a flag field indicating the end of the frame with a third preset identifier.

[0030] In the above solution, the method further includes:

[0031] If at least one third state parameter and third environment parameter sent by the second agent is not received, the historical state parameters and historical environment parameters received at the most recent moment are stored.

[0032] In the above solution, the method further includes:

[0033] If link information for closing a communication link with a fourth agent is detected, closing the communication link with the fourth agent; wherein the fourth agent belongs to at least one of the second agents;

[0034] Release the link resources between the fourth agent and the intelligent agent.

[0035] In the above solution, the link information is at least one of the following: LCP link shutdown, NCP link shutdown, and external link shutdown triggering event.

[0036] In the above solution, the releasing of the link resources between the agent and the fourth agent includes:

[0037] Release the NCP with the fourth agent;

[0038] releasing the Internet Protocol IP address bound to the fourth agent;

[0039] Release the LCP with the fourth agent.

[0040] The present application provides a communication management device, comprising: a sensing unit, a first determining unit, a second determining unit, and an executing unit; wherein:

[0041] The perception unit is used to determine a first state parameter and a first environment parameter perceived by the first agent; wherein the first state parameter is a parameter for the operation of the first agent;

[0042] The first determining unit is configured to determine a reward value and a dynamic communication threshold corresponding to the first agent based on the first state parameter and the first environment parameter;

[0043] The second determining unit is configured to determine a triggering operation based on the reward value and the dynamic communication threshold; wherein the triggering operation is configured to trigger whether the first agent communicates with at least one second agent;

[0044] The execution unit is used to execute the trigger operation.

[0045] The present application provides a first intelligent agent, which comprises at least: a communication interface, a memory, a processor, and a communication bus; wherein:

[0046] The memory is used to store executable instructions;

[0047] The communication bus is used to realize the communication connection between the communication interface, the processor and the memory;

[0048] The processor is configured to execute the communication management program stored in the memory to implement the steps of any one of the communication management methods described above.

[0049] The present application provides a storage medium having a communication management program stored thereon. When the communication management program is executed, it is used to implement the steps of any of the communication management methods described above.

[0050] The present application provides a computer program product, comprising a computer program, which implements the steps of the communication management method according to any one of the above items when executed by a processor.

[0051] The embodiments of the present application provide a communication management method, apparatus, device, storage medium, and computer program product. After determining the first state parameter and the first environmental parameter perceived by the first intelligent agent, the reward value and dynamic communication threshold corresponding to the first intelligent agent are determined based on the first state parameter and the first environmental parameter. Then, based on the reward value and the dynamic communication threshold, a trigger operation is determined and executed. In this way, the first intelligent agent determines the corresponding reward value and dynamic communication threshold based on the perceived first state parameter and the first environmental parameter, triggers the corresponding trigger operation, and solves the current problem of low communication efficiency between multiple intelligent agents through event-triggered operations. A method for realizing communication between multiple intelligent agents is proposed, which reduces the probability of packet loss and communication delay when communicating between multiple intelligent agents, thereby ensuring the communication efficiency between multiple intelligent agents. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flow chart of a communication management method provided in an embodiment of the present application;

[0053] Figure 2 A flowchart of another communication management method provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a communication architecture provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of an application scenario of a dynamic communication threshold provided in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of an implementation flow of establishing a communication link provided in an embodiment of the present application;

[0057] Figure 6 A schematic diagram of the structure of a protocol frame provided in an embodiment of the present application;

[0058] Figure 7 A schematic diagram of the structure of a communication management device provided in an embodiment of the present application;

[0059] Figure 8 A schematic structural diagram of a first intelligent agent provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0061] The embodiment of the present application provides a communication management method, referring to Figure 1 As shown, the method is applied to a first agent, and the method comprises the following steps:

[0062] Step 101: Determine a first state parameter and a first environment parameter perceived by a first agent.

[0063] Among them, the first state parameter is the parameter of the operation of the first agent.

[0064] In an embodiment of the present application, the first agent may be a device capable of communicating with other devices, such as other smart devices or the Internet, such as a smart mobile terminal, including smart cars, smart phones, and other devices, or even a communication base station. The first agent collects its own current operating parameters to obtain first state parameters, and obtains environmental parameters of the first agent's current environment, such as the network environment in which the first agent is located, to obtain first environmental parameters.

[0065] Step 102: Based on the first state parameter and the first environment parameter, determine the reward value and dynamic communication threshold corresponding to the first agent.

[0066] In an embodiment of the present application, the first state parameter and the first environment parameter are analyzed to determine the reward value and dynamic communication threshold corresponding to the first agent, and the reward value is used to evaluate the communication state of the first agent.

[0067] Step 103: Determine a triggering operation based on the reward value and the dynamic communication threshold.

[0068] The trigger operation is used to trigger whether the first agent communicates with at least one second agent.

[0069] In the embodiment of the present application, the reward value of the first agent is compared and analyzed with the dynamic communication threshold to determine the corresponding triggering operation for triggering communication between the first agent and other agents.

[0070] Step 104: Execute a trigger operation.

[0071] In an embodiment of the present application, after determining that a trigger operation is obtained, the first intelligent agent executes the trigger operation, realizing a method for triggering communication between the first intelligent agent and other intelligent devices based on an event, thereby improving the communication efficiency of the first intelligent agent and reducing unnecessary waste of resources.

[0072] The communication management method provided in the embodiment of the present application determines the first state parameter and the first environmental parameter perceived by the first agent, and then determines the reward value and dynamic communication threshold corresponding to the first agent based on the first state parameter and the first environmental parameter. Then, based on the reward value and the dynamic communication threshold, a trigger operation is determined and executed. In this way, the first agent determines the corresponding reward value and dynamic communication threshold based on the perceived first state parameter and the first environmental parameter, triggers the corresponding trigger operation, and solves the current problem of low communication efficiency between multiple agents through event-triggered operations. A method for realizing communication between multiple agents is proposed, which reduces the probability of packet loss and communication delay when communicating between multiple agents, and ensures the communication efficiency between multiple agents.

[0073] Based on the above embodiments, the embodiments of the present application provide a communication management method, referring to Figure 2 As shown, the method is applied to a first agent, and the method comprises the following steps:

[0074] Step 201: Perceive the current second state parameter and second environment parameter of the first agent.

[0075] In an embodiment of the present application, the first agent collects its own state parameters and network environment parameters in real time to obtain corresponding second state parameters and second environment parameters respectively.

[0076] Step 202: Preprocess the second state parameter and the second environment parameter to obtain the first state parameter and the first environment parameter.

[0077] Among them, the first state parameter is the parameter of the operation of the first agent.

[0078] In the embodiment of the present application, the preprocessing method may be data normalization, data cleaning, or encoding of the second state parameter and the second environmental parameter to convert the data from high-dimensional data into low-dimensional data to reduce the amount of data transmission. The first agent preprocesses the collected second state parameter and the second environmental parameter using a preset processing method to obtain the first state parameter and the first environmental parameter.

[0079] Step 203: Based on the first state parameter and the first environment parameter, determine the reward value and dynamic communication threshold corresponding to the first agent.

[0080] In an embodiment of the present application, the first state parameter and the first environmental parameter are calculated and analyzed to determine the reward value corresponding to the first intelligent agent. The calculation and analysis method may be, for example, a corresponding intelligent artificial algorithm, such as a trained reward prediction model stored locally by the first intelligent agent; the first state parameter and the first environmental parameter are calculated and analyzed to determine the dynamic communication threshold corresponding to the current first environmental parameter of the first intelligent agent. In this way, the dynamic communication threshold is determined based on the actual situation of the first intelligent agent, which effectively ensures the communication quality of the first intelligent agent.

[0081] Step 204: Determine a triggering operation based on the reward value and the dynamic communication threshold.

[0082] The trigger operation is used to trigger whether the first agent communicates with at least one second agent.

[0083] In an embodiment of the present application, a comparative analysis is performed on the reward value of the first intelligent agent and the dynamic communication threshold, such as a size comparison, and the threshold range corresponding to the reward value is determined based on the dynamic communication threshold, and then the trigger operation corresponding to the threshold range can be determined.

[0084] Step 205: Execute the trigger operation.

[0085] In the embodiment of the present application, corresponding triggering operations are performed, such as triggering the first agent to establish a communication connection with other agent devices, or the first agent not to establish a communication connection with other agent devices.

[0086] Based on the above embodiment, in other embodiments of the present application, step 203 can be implemented by steps 203a to 203c:

[0087] Step 203a: Determine a reward value based on the analysis performance parameters included in the first state parameters and the device performance parameters of the first agent, and the wireless link characteristic parameters included in the first environment parameters.

[0088] In an embodiment of the present application, the analysis performance parameters may be, for example, the computing requirements corresponding to the completion of the current data task of the first intelligent agent device, such as the requirements for the artificial intelligence model used when estimating the current data task; the device performance parameters of the first intelligent agent may be, for example, parameters such as the computing resources and storage resources of the first intelligent agent device, and the wireless link characteristic parameters included in the first environmental parameters may be parameters such as the channel quality parameters and signal transmission distance of the wireless channel that the first intelligent agent can currently provide.

[0089] The analysis performance parameters, device performance parameters and wireless link characteristic parameters of the first agent are scored respectively, and then the scores are calculated. For example, the reward value of the first agent can be calculated by averaging or weighted averaging.

[0090] Step 203b: Determine the target probability of the first agent sending or receiving a signal.

[0091] In the embodiment of the present application, the target probability may be an upper limit of the probability of the first agent sending or receiving a signal in each time slot, which may be determined by using entropy calculation of Gaussian distribution.

[0092] Step 203c, determine the dynamic communication threshold based on the occupied bandwidth parameter, target probability, calculation delay parameter, analysis performance parameter and device performance parameter included in the first state parameter, and the position distribution parameter of at least one third intelligent agent and the first intelligent agent, the number of perceived intelligent agents and the connection status parameter with at least one third intelligent agent included in the first environment parameter.

[0093] Wherein, at least one second agent belongs to at least one third agent.

[0094] In an embodiment of the present application, the occupied bandwidth parameter represents the bandwidth data that has been used in the bandwidth of the first agent, the calculation delay parameter is the delay corresponding to the first agent performing calculation and analysis on the current data task, the number of agents is the number of multiple agents that the first agent can perceive, and the connection status parameter with at least one third agent can be that a communication connection has been established with the third agent, or that a communication connection has not been established with the third agent.

[0095] When determining the dynamic communication threshold, the occupied bandwidth parameter, target probability, calculation delay parameter, analysis performance parameter and equipment performance parameter included in the first state parameter, as well as the position distribution parameter of at least one third intelligent agent and the first intelligent agent, the number of perceived intelligent agents and the connection state parameter with at least one third intelligent agent included in the first environment parameter are input into a trained threshold estimation neural network model for calculation, or it can be calculated through a determined threshold calculation function.

[0096] Based on the above embodiment, in other embodiments of the present application, step 204 can be implemented by step 204a or step 204b:

[0097] Step 204a: If the reward value is less than the dynamic communication threshold, determine that the triggering operation is a receiving information operation.

[0098] In an embodiment of the present application, when the reward value is less than the dynamic communication threshold, it indicates that the first intelligent agent cannot perform subsequent operations such as decision-making processing based on the first state parameters and first environmental parameters collected by itself. Therefore, the first intelligent agent can determine that the trigger operation is an information receiving operation, that is, receiving the perceived information sent by other intelligent agents to assist the first intelligent agent in performing subsequent operations.

[0099] Step 204b: If the return value is greater than or equal to the dynamic communication threshold, determine that the triggering operation is a message sending operation.

[0100] In an embodiment of the present application, when the reward value is greater than or equal to the dynamic communication threshold, the first state parameters and first environmental parameters collected by the first intelligent agent are highly reliable. At this time, the first intelligent agent can directly perform subsequent operation analysis based on the first state parameters and first environmental parameters perceived by itself, and the first intelligent agent can also send the perceived first state parameters and first environmental parameters to other intelligent devices that need to perceive information. Therefore, it can be determined that the triggering operation at this time is an information sending operation.

[0101] Based on the above embodiment, in other embodiments of the present application, when the triggering operation is to trigger the first agent to communicate with at least one third agent, step 205 can be implemented by steps 205a to 205c:

[0102] Step 205a: Determine at least one second agent from at least one third agent.

[0103] In an embodiment of the present application, a first agent selects at least one second agent from at least one third agent it senses to establish a communication connection. The reward value of the perception information of the at least one second agent is greater than or equal to its corresponding dynamic communication threshold. Furthermore, the at least one second agent also meets one or more of the following conditions: being an agent within a preset range of the first agent, having a communication channel quality that exceeds a certain threshold with the first agent, or being within a pre-specified agent range.

[0104] Step 205b: Establish a communication link with each second agent.

[0105] In the embodiment of the present application, based on the manner of establishing a communication link between the first agent and each second agent, a communication link is established with each second agent so as to transmit data through the communication link.

[0106] Step 205c: Based on each communication link, perform a communication operation with each second agent.

[0107] The communication operation includes sending information and / or receiving information.

[0108] In an embodiment of the present application, after a communication link is established between the first intelligent agent and each second intelligent agent, data services are transmitted through the communication link according to actual business needs. For example, data information can be sent to the corresponding second intelligent agent through the established communication link to implement the information sending operation, or data information sent by the corresponding second intelligent agent can be received through the communication link to implement the information receiving operation.

[0109] Based on the above embodiment, in other embodiments of the present application, step 205b can be implemented by steps a11 to a12:

[0110] Step a11: After establishing a physical connection with each second agent, perform LCP link authentication with the corresponding second agent through a Logical Link Control Protocol LCP configuration request in a preset frame format to obtain an authentication result.

[0111] In an embodiment of the present application, when a communication link is established between a first intelligent agent and a corresponding second intelligent agent, a physical connection is first established between the first intelligent agent and the corresponding second intelligent agent, and then a physical link is established between the first intelligent agent and the corresponding second intelligent agent. Specifically, the data link can be configured and detected through a frame with a Link Control Protocol (LCP) protocol field, and the LCP link authentication is performed using a three-way handshake mechanism of the Challenge Handshake Authentication Protocol (CHAP) authentication to obtain an authentication result.

[0112] Step a12: If the authentication result is authentication passed, the network layer protocol configuration docking is performed through a network layer control protocol NCP configuration request in a preset format to establish a communication link.

[0113] Among them, the preset frame format includes at least: a header, an information payload and a tail; among them, the header includes: a flag field indicating the start of the frame with a first preset identifier, an address field with a second preset identifier, a control field with a third preset identifier and a protocol field for identifying the protocol type, the information payload is an information field of a preset length, and the tail includes: a frame check sequence and a flag field indicating the end of the frame with a third preset identifier.

[0114] In the embodiment of the present application, if the authentication result is authentication failure, it indicates that the link establishment failed and the link is terminated. When the authentication result is authentication success, it indicates that the LCP link authentication is successful, and the network layer authentication configuration of the link between the first intelligent agent and the corresponding second intelligent agent continues. The first intelligent agent uses a pre-agreed Network Control Protocol (NCP) configuration request in a preset format to perform network layer protocol configuration docking with the corresponding second intelligent agent. After the docking is successful, the NCP link authentication is successful, and the communication link is successfully established.

[0115] Based on the above embodiment, in other embodiments of the present application, after the first agent executes step 205c, it is further configured to execute step 205d:

[0116] Step 205d: If the third state parameter and the third environment parameter sent by at least one second agent are not received, the historical state parameter and the historical environment parameter received at the most recent moment are stored.

[0117] In an embodiment of the present application, after a communication link is successfully established between a first intelligent agent and at least one second intelligent agent, when data is interacted with at least one second intelligent agent, if it is detected that the third state parameters and third environmental parameters of the second intelligent agent sent by at least one of the second intelligent agents are not received at a certain data reception moment, the first intelligent agent will store the historical state parameters and historical environmental parameters sent by the second intelligent agent that did not send the third state parameters and third environmental parameters at the most recent moment to the current moment, so as to subsequently update the dynamic communication threshold based on the stored historical state parameters and historical environmental parameters.

[0118] Based on the above embodiment, in other embodiments of the present application, after the first intelligent device executes step 205c, it is further configured to execute steps 205e to 205f:

[0119] Step 205e: If link information for closing the communication link with the fourth agent is detected, the communication link with the fourth agent is closed.

[0120] The fourth agent belongs to at least one second agent. The link information is at least one of the following: an LCP closed link, an NCP closed link, and an external link closed trigger event.

[0121] Step 205f: Release the link resources with the fourth agent.

[0122] Based on the above embodiment, in other embodiments of the present application, step 205f can be implemented by steps b11 to b13:

[0123] Step b11: Release the NCP with the fourth agent.

[0124] Step b12: Release the Internet Protocol IP address bound to the fourth agent.

[0125] Step b13: Release the LCP with the fourth agent.

[0126] Based on the above embodiments, the present application provides a multi-agent communication architecture based on event triggering, such as Figure 3 As shown in the figure, it includes: Env represents the environment and agent perception module, which is used to perceive the state parameters and environmental parameters of the agent; ENC represents the encoding module, which is used to encode the state parameters and environmental parameters perceived by Env, and can be implemented by the encoding neural network; ACT is used to represent the agent's agent action module, which is used to implement the agent's action execution, and can be implemented by the action neural network; ET represents the event-triggered gating module, which is used to implement gate control, and can be implemented by the gating neural network; ETSNet represents the event-triggered sending network, which is used to decide whether the current information is sent to other agents; ETRNet represents the event-triggered receiving network, which is used to decide whether to receive information from other agents; ZOH represents the zero-order holder module, which is used to implement interpolation between sampling points. Figure 3 In the above, Agent represents the agent, Actions represents the execution of the agent, O represents the agent's own perception information, rm represents the perception information received from other agents, and m represents the historical parameter information stored for other agents. Figure 3 The specific structure of ActorNet Inputs shown in Figure 3 The structure in the dotted box on the right is shown.

[0127] based on Figure 3 The multi-agent communication architecture shown in the figure, the execution process of the event-triggered communication network can be divided into three stages:

[0128] Phase 1: The intelligent agent perceives its own state and network environment.

[0129] Among them, each intelligent agent perceives its network environment and its own intelligent agent operation status through the environment and intelligent agent perception module Env of the multi-agent communication architecture, and obtains the perception signal including its own status and network environment. The encoding module ENC encodes the perception signal perceived by the intelligent agent perception module Env, encodes the high-dimensional data into low-dimensional data, obtains the observation signal, and thus reduces the data transmission volume.

[0130] The second stage: The event-triggered gating control module ETC performs gating control on the observation signal and the sending / receiving of the signal to be transmitted.

[0131] Among them, the observation signal is used as the input of the event-triggered gating module, and the gating neural network includes an event-triggered sending network and an event-triggered receiving network.

[0132] The third stage: the decision-making of the agent action module ATC, which determines which agents the agent selects to send / receive information, as well as the transmission mode.

[0133] In the third phase of decision-making, if an agent opens its transmission gate, its current information will be sent to other agents via the event-triggered sending network. Alternatively, through the event-triggered receiving network, it will send requests to other agents and receive their current information to determine the next cooperative behavior of the agents.

[0134] The principle of an event-triggered sending network is that an agent estimates the value of its own perception information to other agents, known as the reward value. When the reward value exceeds a dynamic communication threshold, the agent's event-triggered sending network transmits relevant information to other agents. For example, it can send observation signals detected by itself to other agents to assist them in decision-making. For example, in collaborative navigation tasks, intelligent vehicles can assist other vehicles in making navigation decisions by sharing their observation signals.

[0135] The principle of the event-triggered receiving network is to estimate the value of the agent's perceived information to itself. When the reward value falls below the dynamic communication threshold, the event-triggered receiving network receives current information from other agents to assist in its own decision-making. For example, in a routing task, it can assist its own routing decisions by collecting status information from other intelligent routers.

[0136] In the two sending and receiving paradigms mentioned above, if the agent does not receive the latest information from other agents, it uses the zero-order hold module to memorize the most recently received information instead of replacing it with a zero vector to ensure continuous and effective collaboration of the agents.

[0137] In the second stage of gate control and signal transmission / reception, due to the uncertainty of the wireless network environment, the dynamic number and location of agents in the network, and the diversity of agent needs, the agent transmission gating threshold changes and updates dynamically. Correspondingly, a dynamic communication threshold determination process can refer to Figure 4 As shown:

[0138] Step c11: The first agent determines the number N of agents that can communicate with each other and the sampling frequency F.

[0139] The sampling frequency F is determined by the first agent according to the length of the transmission symbol L. It should be noted that in a system with M agents, each agent can communicate with at most M-1 other agents.

[0140] Step c12: The first agent calculates the communication reward value and the dynamic communication threshold.

[0141] Among them, when calculating the communication reward value, the factors to be considered can be as follows: (1) The agent estimates the effect that the AI ​​model of the local agent device can achieve based on the computing requirements of the current data task, such as the computing requirements of using an artificial intelligence (AI) model to calculate the data task. The better the estimated effect, the greater the communication reward value, and vice versa; (2) The transmission distance between the two agents, the better the wireless channel conditions, whether the two agents have established a point-to-point protocol (PPP) communication connection, etc.; (3) The computing resources and storage resources of the agent. After normalizing and quantifying the factors in (1), (2) and (3), the communication reward value is calculated by, for example, using a weighted average method.

[0142] Regarding calculating the dynamic communication threshold: The dynamic communication threshold is determined based on the upper bound of the probability of an agent sending or receiving a signal and the bandwidth resources occupied. Furthermore, the network environment and device status must be comprehensively considered, including the latency and computational requirements of the current computing task, the topology of the agent system (including the location, number, and connectivity of multiple agents), and the agent's computing and storage resources. When calculating the dynamic communication threshold, these factors can be calculated using an artificial intelligence model or a predetermined computational algorithm. The upper bound of the probability of an agent sending or receiving a signal can be calculated based on the entropy of a Gaussian distribution.

[0143] Step c13: The first agent determines the relationship between the communication reward value and the dynamic communication threshold. If the communication reward value is less than the dynamic communication threshold, execute step c14; if the communication reward value is greater than or equal to the dynamic communication threshold, execute step c15.

[0144] Step c14: The first agent triggers the receiving network to participate in communication.

[0145] Step c15: The first agent triggers sending a network to participate in communication.

[0146] Step c16: The first agent determines whether it has received the latest message from other agents via the receiving network. If it has received the latest message from other agents, it proceeds to step c17; if it has not received the latest message from other agents, it proceeds to step c18.

[0147] Step c17: Store the latest messages received from other intelligent agents.

[0148] Step c18: Use the zero-order holder to store the most recent historical message.

[0149] Step c19: Update the dynamic communication threshold according to the latest message or the most recent historical message.

[0150] In the third stage, the first agent establishes a communication connection with other agents, and sends the perception information currently perceived by the first agent, including the network environment and device status, to other designated agents through an event-triggered sending network, or sends a perception information request to other designated agents through an event-triggered receiving network, and receives the current perception information of other agents. In the implementation process, the specific implementation of directional point-to-point transmission between multiple agents can adopt the multi-agent PPP connection protocol, which includes data link layer content, physical layer content, and network layer content. Correspondingly, a process of establishing a multi-agent PPP session based on event triggering is explained by taking the establishment of a communication link and data transmission between a first agent and a second agent as an example. The corresponding implementation process can be as follows: Figure 5 As shown, the following steps are included:

[0151] Step d11: The link is inactive.

[0152] At this time, there is no communication link between the first agent and other agents.

[0153] Step d12: Physical layer link is established. If the establishment is successful, execute step d13; otherwise, execute step d20.

[0154] Among them, a physical layer connection is established between the first intelligent agent and the second intelligent agent. The specific implementation process is that the first intelligent agent sends a frame with an LCP protocol field to configure and detect the data link between the first intelligent agent and the second intelligent agent. At this time, the information payload field of the frame with the LCP protocol field contains a specific LCP configuration request, for example, including the PPP parameters to be used for negotiation and selection, including identity authentication, negotiated maximum transmission unit, information compression, etc.

[0155] Step d13: Link authentication. If the authentication passes, go to step d14; otherwise, go to step d18.

[0156] When link authentication is implemented, the LCP link between the first agent and the second agent is established. Specifically, the three-way handshake mechanism of CHAP authentication can be used to implement link authentication.

[0157] Step d14: Network layer protocol.

[0158] Among them, the network layer protocol is carried out between the first intelligent agent and the second intelligent agent. Specifically, the first intelligent agent can send a frame with an NCP protocol field to select and configure a network layer protocol and negotiate network layer parameters; or the first intelligent agent uses a frame with an IP protocol field to select and configure the IP layer protocol to implement the network layer protocol.

[0159] Step d15: The link is opened.

[0160] Among them, after completing step a14, the communication link between the first intelligent agent and the second intelligent agent completes the authentication process between the data link layer, physical layer and network layer. At this point, the communication link between the first intelligent agent and the second intelligent agent is established, and the subsequent data transmission communication process can be carried out.

[0161] Step d16: Data transmission.

[0162] After the point-to-point connection configuration is completed, the first agent and the second agent can send their own protocol layer data packets, which include their own status information and network environment information.

[0163] Step d17: The link is closed.

[0164] The communication link between the first agent and the second agent will be maintained until the link is closed by the LCP or NCP, or some external event occurs (such as excessive idle time, user intervention, or link failure). When the link is closed, the NCP is first released to release the IP address, and then the LCP is released to close the data link connection and release all resources.

[0165] Step d18: The link is terminated.

[0166] Step d19: The LCP link is terminated.

[0167] Step d20: LCP configuration negotiation fails.

[0168] Based on the above embodiment, the embodiment of the present application also provides a PPP connection protocol frame format based on event triggering. The PPP connection protocol frame format consists of 7 parts, which can be referred to as Figure 6 As shown in the figure, the PPP connection protocol frame format includes: header (FAC-protocol)-information payload-tail (FCS-F). Among them:

[0169] F is used to represent Flag, which is a flag field in the header and is used to indicate the start of a frame. It can be set to occupy one byte, for example, a fixed value of 0x7E can be specified.

[0170] A is used to represent Address, which is the address field in the header. The link layer does not need IP, and it can be set to occupy one byte. For example, a fixed value of 0xFF can be specified.

[0171] C is used to represent Control, which is the control field in the header. It can be set to occupy one byte. For example, a fixed value of 0x03 can be specified.

[0172] Protocol, used to indicate the protocol type used in the subsequent information payload, can be set to occupy two bytes. Specifically:

[0173] When the protocol field is the first field, exemplarily 0xC021, the information field of the PPP frame can be used to indicate that the current data is the Logical Link Control Protocol (LCP). In this case, the event-triggered PPP connection protocol frame is used by the agent to enter the link establishment state, corresponding to step d12 above. The information field contains a specific LCP configuration request and performs an authentication check.

[0174] When the protocol field is the second field, exemplarily 0x8021, the information field of the PPP frame can be used to indicate that the current data is the Network Layer Control Protocol (NCP). In this case, the event-triggered PPP connection protocol frame is used to complete the configuration docking of the network protocol. The Network Control Protocol (NCP) of the intelligent agent exchanges network layer-specific network control packets with each other. For example, the Internet Protocol Control Protocol (IPCP) is used to negotiate IP parameters, and the Internetwork Packet Exchange Control Protocol (IPXCP) is used to negotiate IPx parameters. When running on a low-speed link, the intelligent agent can compress the Transmission Control Protocol (TCP) and IP headers, thereby reducing the number of bits sent on the link.

[0175] Information Payload: The length of the information payload field is variable, but it is generally required to be no longer than a certain number of bytes. For example, the information payload field can be limited to 1500 bytes. During the communication link establishment phase, the information payload carries information about the connection configuration between layers. After the communication link is established, during the data transmission phase, the information payload carries the agent's state and perceived network environment information.

[0176] FCS is a frame check sequence using CRC, which can be set to occupy 2 bytes.

[0177] F stands for Flag, which is a flag field at the end and is used to indicate the end of a frame. For example, a fixed value of 0x7E may be specified.

[0178] It should be noted that the fixed values ​​specified in the PPP connection protocol frame format can be modified according to actual conditions and can be agreed upon in advance. The embodiments of this application are only given as examples.

[0179] In this way, the event-triggered communication mechanism proposed in the embodiment of the present application only participates in the collaboration between multiple intelligent agents when necessary, which can effectively reduce the frequency and data of the intelligent agents' communication and improve the utilization of bandwidth.

[0180] It should be noted that, for the description of the same steps and contents in this embodiment as those in other embodiments, reference can be made to the description in other embodiments and will not be repeated here.

[0181] The communication management method provided in the embodiment of the present application determines the first state parameter and the first environmental parameter perceived by the first agent, and then determines the reward value and dynamic communication threshold corresponding to the first agent based on the first state parameter and the first environmental parameter. Then, based on the reward value and the dynamic communication threshold, a trigger operation is determined and executed. In this way, the first agent determines the corresponding reward value and dynamic communication threshold based on the perceived first state parameter and the first environmental parameter, triggers the corresponding trigger operation, and solves the current problem of low communication efficiency between multiple agents through event-triggered operations. A method for realizing communication between multiple agents is proposed, which reduces the probability of packet loss and communication delay when communicating between multiple agents, and ensures the communication efficiency between multiple agents.

[0182] Based on the above embodiments, the embodiments of the present application provide a communication management device, which can be applied to Figures 1-2 In the communication management method provided in the corresponding embodiment, refer to Figure 7 As shown, the communication management device 3 may include: a sensing unit 31, a first determining unit 32, a second determining unit 33 and an executing unit 34; wherein:

[0183] A perception unit 31 is configured to determine a first state parameter and a first environment parameter perceived by the first agent; wherein the first state parameter is a parameter of the first agent's operation;

[0184] A first determining unit 32 is configured to determine a reward value and a dynamic communication threshold corresponding to the first agent based on the first state parameter and the first environment parameter;

[0185] A second determining unit 33 is configured to determine a triggering operation based on the reward value and the dynamic communication threshold; wherein the triggering operation is configured to trigger whether the first agent communicates with at least one second agent;

[0186] The execution unit 34 is configured to execute a trigger operation.

[0187] In other embodiments of the present application, the perception unit 31 includes: a perception module and a preprocessing module; wherein:

[0188] A perception module, configured to perceive a current second state parameter and a second environment parameter of the first agent;

[0189] The preprocessing module is used to preprocess the second state parameter and the second environmental parameter to obtain the first state parameter and the first environmental parameter.

[0190] In other embodiments of the present application, the first determining unit includes: a first determining module, a second determining module, and a third determining module; wherein:

[0191] A first determining module is configured to determine a reward value based on the analysis performance parameter included in the first state parameter and the device performance parameter of the first agent, and the wireless link characteristic parameter included in the first environment parameter;

[0192] A second determination module is used to determine the target probability of the first agent sending or receiving a signal;

[0193] The third determination module is used to determine the dynamic communication threshold based on the occupied bandwidth parameter, target probability, calculation delay parameter, analysis performance parameter and equipment performance parameter included in the first state parameter, and the position distribution parameter of at least one third intelligent agent and the first intelligent agent, the number of perceived intelligent agents and the connection status parameter with at least one third intelligent agent included in the first environment parameter; wherein, at least one second intelligent agent belongs to at least one third intelligent agent.

[0194] In other embodiments of the present application, the second determining unit includes: a fourth determining module; wherein:

[0195] a fourth determining module, configured to determine that the triggering operation is an information receiving operation if the reward value is less than the dynamic communication threshold;

[0196] The fourth determining module is further configured to determine that the triggering operation is a sending operation if the return value is greater than or equal to the dynamic communication threshold.

[0197] In other embodiments of the present application, when the triggering operation is to trigger the first agent to communicate with at least one third agent, the execution unit includes: a fifth determination module, a creation module, and an execution module; wherein:

[0198] a fifth determining module, configured to determine at least one second agent from at least one third agent;

[0199] an establishing module for establishing a communication link for communicating with each second agent;

[0200] The execution module is used to execute a communication operation with each second intelligent agent based on each communication link; wherein the communication operation includes an information sending operation and / or an information receiving operation.

[0201] In other embodiments of the present application, the establishment module is specifically used to implement the following steps:

[0202] After establishing a physical connection with each second agent, performing LCP link authentication with the corresponding second agent through a Logical Link Control Protocol (LCP) configuration request in a preset frame format to obtain an authentication result;

[0203] If the authentication result is authentication passed, the network layer protocol configuration docking is performed through the preset format of the Network Control Protocol NCP configuration request to establish the communication link.

[0204] In other embodiments of the present application, the preset frame format includes at least: a header, an information payload and a tail; wherein the header includes: a flag field indicating the start of the frame with a first preset identifier, an address field with a second preset identifier, a control field with a third preset identifier and a protocol field for identifying the protocol type, the information payload is an information field of a preset length, and the tail includes: a frame check sequence and a flag field indicating the end of the frame with a third preset identifier.

[0205] In other embodiments of the present application, the execution unit further includes: a storage module; wherein:

[0206] The storage module is used to store the historical state parameters and historical environment parameters received at the most recent moment if the third state parameters and third environment parameters sent by at least one second intelligent agent are not received.

[0207] In other embodiments of the present application, the execution unit further includes: a closing module and a releasing module; wherein:

[0208] a closing module, configured to close the communication link with the fourth agent if link information for closing the communication link with the fourth agent is detected; wherein the fourth agent belongs to at least one second agent;

[0209] The release module is used to release the link resources between the fourth agent and the fourth agent.

[0210] In other embodiments of the present application, the link information is at least one of the following: LCP closed link, NCP closed link, and external link closed trigger event.

[0211] In other embodiments of the present application, the release module is specifically configured to implement the following steps:

[0212] Release the NCP with the fourth agent;

[0213] Release the Internet Protocol (IP) address bound to the fourth agent;

[0214] Release the LCP with the fourth agent.

[0215] It should be noted that the interaction between the units and modules in this embodiment can be referred to Figures 1-2 The implementation process of the communication management method provided in the corresponding embodiment will not be repeated here.

[0216] The communication management device provided in the embodiment of the present application determines the first state parameter and the first environmental parameter perceived by the first agent, and then determines the reward value and dynamic communication threshold corresponding to the first agent based on the first state parameter and the first environmental parameter. Then, based on the reward value and the dynamic communication threshold, it determines a trigger operation and executes the trigger operation. In this way, the first agent determines the corresponding reward value and dynamic communication threshold based on the perceived first state parameter and the first environmental parameter, triggers the corresponding trigger operation, and solves the current problem of low communication efficiency between multiple agents through event-triggered operations. A method for realizing communication between multiple agents is proposed, which reduces the probability of packet loss and communication delay when communicating between multiple agents, and ensures the communication efficiency between multiple agents.

[0217] Based on the above embodiments, the embodiment of the present application provides a first intelligent agent, which can be applied to Figures 1-2 In the communication management method provided in the corresponding embodiment, refer to Figure 8 As shown, the first agent 4 may include: a communication interface 41, a memory 42, a processor 43 and a communication bus 44; wherein:

[0218] Memory 42, for storing executable instructions;

[0219] A communication bus 44 is used to implement communication connections between the communication interface 41, the processor 43, and the memory 42;

[0220] Processor 43 is used to execute the communication management program stored in the memory, so as to implement the following Figures 1-2 The implementation process of the communication management method provided in the corresponding embodiment will not be repeated here.

[0221] Based on the above embodiments, the embodiments of the present application provide a computer-readable storage medium, referred to as a storage medium, which stores one or more programs, which can be executed by one or more processors to implement the reference Figures 1-2The implementation process of the communication management method provided in the corresponding embodiment will not be repeated here.

[0222] Based on the aforementioned embodiments, an embodiment of the present application further provides a computer program product, including a computer program, which can be executed by the processor 43 of the first agent 4 to complete any of the aforementioned method steps.

[0223] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0224] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0225] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0226] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0227] The above description is merely a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application.

Claims

1. A communication management method, characterized in that: The method comprises: Determining a first state parameter and a first environment parameter perceived by a first agent; wherein the first state parameter is a parameter of the operation of the first agent; Determining a reward value and a dynamic communication threshold corresponding to the first agent based on the first state parameter and the first environment parameter; Determining a trigger operation based on the reward value and the dynamic communication threshold; wherein the trigger operation is used to trigger whether the first agent communicates with at least one second agent; The trigger operation is executed.

2. The method according to claim 1, characterized in that The determining of the first state parameter and the first environment parameter perceived by the first agent includes: Perceiving a second state parameter and a second environment parameter of the first agent; The second state parameter and the second environment parameter are preprocessed to obtain the first state parameter and the first environment parameter.

3. The method according to claim 2, characterized in that The determining, based on the first state parameter and the first environment parameter, a reward value and a dynamic communication threshold corresponding to the first agent includes: Determining the reward value based on the analysis performance parameter included in the first state parameter and the device performance parameter of the first agent, and the wireless link characteristic parameter included in the first environment parameter; determining a target probability of the first agent sending or receiving a signal; The dynamic communication threshold is determined based on the occupied bandwidth parameter, the target probability, the calculation delay parameter, the analysis performance parameter and the device performance parameter included in the first state parameter, and the position distribution parameters of at least one perceived third agent and the first agent, the number of perceived agents and the connection status parameters with at least one said third agent included in the first environmental parameter; wherein at least one of the second agents belongs to at least one said third agent.

4. The method according to claim 1, wherein The determining of a triggering operation based on the reward value and the dynamic communication threshold includes: If the reward value is less than the dynamic communication threshold, determining that the triggering operation is an information receiving operation; If the reward value is greater than or equal to the dynamic communication threshold, the triggering operation is determined to be a message sending operation.

5. The method according to claim 4, characterized in that The executing the triggering operation includes: Determining at least one of said second agents from at least one third agent; establishing a communication link with each of the second agents; Based on each of the communication links, a communication operation with each of the second intelligent agents is performed; wherein the communication operation includes an information sending operation and / or an information receiving operation.

6. The method according to claim 5, characterized in that The establishing of a communication link for communicating with each of the second intelligent agents comprises: After establishing a physical connection with each second agent, performing LCP link authentication with the corresponding second agent through a Logical Link Control Protocol (LCP) configuration request in a preset frame format to obtain an authentication result; If the authentication result is authentication passed, network layer protocol configuration docking is performed through the network layer control protocol NCP configuration request in the preset format to establish a communication link.

7. The method according to claim 6, characterized in that The preset frame format includes at least: a header, an information payload and a tail; wherein the header includes: a flag field indicating the start of the frame with a first preset identifier, an address field with a second preset identifier, a control field with a third preset identifier and a protocol field for identifying the protocol type, the information payload is an information field of a preset length, and the tail includes: a frame check sequence and a flag field indicating the end of the frame with a third preset identifier.

8. The method according to claim 5, characterized in that The method further comprises: If at least one third state parameter and third environment parameter sent by the second agent is not received, the historical state parameters and historical environment parameters received at the most recent moment are stored.

9. The method according to claim 5, characterized in that The method further comprises: If link information for closing a communication link with a fourth agent is detected, closing the communication link with the fourth agent; wherein the fourth agent belongs to at least one of the second agents; Release the link resources between the fourth agent and the intelligent agent.

10. The method according to claim 9, characterized in that The link information is at least one of the following: an LCP closed link, an NCP closed link, and an external link closed triggering event.

11. The method according to claim 9, characterized in that The releasing of the link resources between the agent and the fourth agent includes: Release the NCP with the fourth agent; releasing the Internet Protocol IP address bound to the fourth agent; Release the LCP with the fourth agent.

12. A communication management device, characterized in that: The device includes: a sensing unit, a first determining unit, a second determining unit, and an executing unit; wherein: The perception unit is used to determine a first state parameter and a first environment parameter perceived by the first agent; wherein the first state parameter is a parameter for the operation of the first agent; The first determining unit is configured to determine a reward value and a dynamic communication threshold corresponding to the first agent based on the first state parameter and the first environment parameter; The second determining unit is configured to determine a triggering operation based on the reward value and the dynamic communication threshold; wherein the triggering operation is configured to trigger whether the first agent communicates with at least one second agent; The execution unit is used to execute the trigger operation.

13. A first intelligent agent, characterized in that: The first intelligent agent comprises at least: a communication interface, a memory, a processor and a communication bus; wherein: The memory is used to store executable instructions; The communication bus is used to realize the communication connection between the communication interface, the processor and the memory; The processor is configured to execute the communication management program stored in the memory to implement the steps of the communication management method according to any one of claims 1 to 11.

14. A storage medium, characterized in that The storage medium stores a communication management program, which is used to implement the steps of the communication management method according to any one of claims 1 to 11 when executed.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the steps of the communication management method according to any one of claims 1 to 11.

Images