Agent information interaction method, interaction system, device and storage medium

Abstract

This invention provides an intelligent agent information interaction method, interaction system, device, and storage medium, belonging to the field of artificial intelligence technology. The method includes: sending a communication-sensing integrated signal to a second intelligent agent, the integrated communication-sensing integrated signal carrying modulation information of communication signals and sensing signals, and the integrated communication-sensing integrated signal carrying first instruction information for the second intelligent agent; detecting sensing echo signals to obtain action time-frequency data, the action time-frequency data being time-frequency data after the second intelligent agent begins to execute an action in response to the first instruction information, the action time-frequency data being used to describe the time-frequency characteristics of the sensing echo signal; identifying the action classification of the action time-frequency data to determine action semantic instructions, the first intelligent agent storing action semantic instructions corresponding to any action classification; and determining business operations based on the action semantic instructions. This invention can be used for information interaction.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to an intelligent agent information interaction method, an intelligent agent information interaction system, an intelligent agent device, and a machine-readable storage medium. Background Technology

[0002] Artificial intelligence (AI) technology has begun to permeate mobile networks, mobile terminals, and mobile service design. Unmanned services are becoming typical business scenarios, especially in the future 6G era. Unmanned services are completed collaboratively by intelligent agents, with no human intervention in the business process, such as autonomous driving, unmanned manufacturing, and unmanned logistics. Here, an intelligent agent refers to a device capable of interacting with its environment. Intelligent agents can be equipped with various sensing devices such as radar, cameras, ultrasonic sensors, and other sensors, as well as communication devices that implement modulation and demodulation functions.

[0003] Currently, business platforms typically determine business processes based on the business status reported by agents, enabling information exchange between agents. However, this results in high communication latency. Furthermore, direct communication between agents can be explored. During this interaction, communication and perception are separated, leading to significant response delays and low interaction efficiency in unmanned operations, making it difficult to achieve widespread unmanned agent-based business. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent agent information interaction method, interaction system, device and storage medium to avoid the low interaction efficiency caused by the separation of communication and perception of intelligent agents, thereby breaking through the limitations of the ability of intelligent agents to communicate and perceive, and providing a basis for intelligent agent interaction efficiency that meets the general requirements of unmanned intelligent agent business.

[0005] To achieve the above objectives, this specification adopts the following approach:

[0006] In a first aspect, embodiments of the present invention provide an intelligent agent information interaction method, applied to a first intelligent agent, the intelligent agent information interaction method comprising:

[0007] Send a communication and sensing integrated signal to the second intelligent agent. The communication and sensing integrated signal carries modulation information of communication signals and sensing signals, and carries first instruction information for the second intelligent agent.

[0008] The sensing echo signal is detected to obtain action time-frequency data. The action time-frequency data is the time-frequency data after the second intelligent agent starts to execute the action in response to the first instruction information. The action time-frequency data is used to describe the time-frequency characteristics of the sensing echo signal. The sensing echo signal includes the echo signal generated by the integrated communication and sensing signal under the action.

[0009] The action classification of the action time-frequency data is identified to determine the action semantic instructions. The first intelligent agent stores action semantic instructions corresponding to any action classification.

[0010] Based on the action semantic instructions, the business operation is determined.

[0011] Specifically, before sending the integrated communication and sensing signal to the second agent, the agent information interaction method further includes:

[0012] The system receives interaction configuration information issued by the business platform. The interaction configuration information is the configuration information between the first intelligent agent and the second intelligent agent. The interaction configuration information includes model parameters of the action reasoning model and signal parameters of the integrated communication and perception signal between the two agents.

[0013] Based on the model parameters in the interactive configuration information, a current action inference model is generated. The current action inference model is used to identify action classification through the obtained action time-frequency data.

[0014] Specifically, the signal parameters are used to indicate the signal frame of the integrated communication sensing signal between the two.

[0015] The frame structure of the signal frame consists of a first subframe set for carrying communication information and a second subframe set for carrying sensing information.

[0016] Specifically, the first subframe set includes multiple subframes for indicating the interval at which the first agent sends communication signals, and multiple subframes for indicating the interval at which the second agent sends communication signals.

[0017] The subframe used to indicate that the first agent is sending a communication signal is an adjacent frame to the subframe used to indicate that the second agent is sending a communication signal.

[0018] The second set of subframes includes multiple subframes for indicating the interval at which the first agent sends perception signals, and multiple subframes for indicating the interval at which the second agent sends perception signals.

[0019] The subframe used to indicate that the first agent is sending a perception signal is an adjacent frame to the subframe used to indicate that the second agent is sending a perception signal.

[0020] Specifically, after configuring the signal parameters as described, the frequency division multiplexing modulation symbol with a specified modulation symbol number in the communication signal and the radar reference symbol with the same modulation symbol number in the sensing signal in any integrated communication and sensing signal are modulated onto the same specified subcarrier.

[0021] Specifically, the model parameters are used to indicate the parameters of the neural network machine model, which is used as the action reasoning model.

[0022] Specifically, both the first and second intelligent agents are configured with an action semantic library, which is used to store action semantic instructions corresponding to any action category.

[0023] Specifically, determining the business operation based on the action semantic instruction includes:

[0024] Based on the action semantic instructions, the business state in the business execution process of the first intelligent agent and the second intelligent agent is determined;

[0025] Determine whether the business status is a target status or an interrupted status to determine the business operation in the business execution process. The business operation includes: a stop operation or an operation to be executed corresponding to the target status or the interrupted status.

[0026] Specifically, after the action semantic instruction is determined, the agent information interaction method further includes:

[0027] Ignore the response of the second agent to the communication signal.

[0028] Secondly, embodiments of the present invention provide an intelligent agent information interaction method, applied to a second intelligent agent, the intelligent agent information interaction method comprising:

[0029] The integrated communication and sensing signal sent by the first intelligent agent is detected, the integrated communication and sensing signal carrying modulation information of communication signal and sensing signal;

[0030] Receive communication information carried by the integrated communication and sensing signal, wherein the communication information is a first instruction information from the first intelligent agent to the second intelligent agent;

[0031] In response to the first indication information, an action is initiated to execute the business execution process of the first intelligent agent and the second intelligent agent, wherein the action classification of the executed action corresponds to at least one action semantic instruction.

[0032] Thirdly, embodiments of the present invention provide an intelligent agent information interaction system, the intelligent agent information interaction system comprising:

[0033] The transmitting module is used to transmit an integrated communication and sensing signal to the second intelligent agent. The integrated communication and sensing signal carries modulation information of communication signals and sensing signals, and carries first indication information for the second intelligent agent.

[0034] The detection module is used to detect the sensing echo signal to obtain action time-frequency data. The action time-frequency data is the time-frequency data after the second intelligent agent starts to execute the action in response to the first instruction information. The action time-frequency data is used to describe the time-frequency characteristics of the sensing echo signal. The sensing echo signal includes the echo signal generated by the integrated communication and sensing signal under the action.

[0035] The identification module is used to identify the action classification of the action time-frequency data in order to determine the action semantic instructions. The first intelligent agent stores the action semantic instructions corresponding to any action classification.

[0036] The determination module is used to determine the business operation based on the action semantic instruction.

[0037] Fourthly, embodiments of the present invention provide an intelligent agent device, the intelligent agent device comprising:

[0038] At least one processor;

[0039] A memory connected to the at least one processor;

[0040] The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the aforementioned method by executing the instructions stored in the memory.

[0041] Fifthly, embodiments of the present invention provide a machine-readable storage medium storing machine instructions that, when executed on a machine, cause the machine to perform the aforementioned method.

[0042] In this invention, an integrated communication and sensing signal between intelligent agents is used to couple sensing and communication capabilities, transmitting information to the intelligent agents. Furthermore, for the actions performed by the receiving intelligent agent in the signal, after the action begins, the sensing echo signal of the integrated communication and sensing signal is detected to identify the action's time-frequency data, obtaining communication information that the receiving intelligent agent has not yet transmitted back via communication. This untransmitted communication information is the communication information that the receiving intelligent agent needs to transmit back to the sending intelligent agent after sensing the completion of the action, allowing for interaction between intelligent agents. This provides parallel information interaction capabilities, improves the interaction efficiency of intelligent agents in unmanned operations, and ensures the robustness of unmanned operations.

[0043] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0044] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used in conjunction with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0045] Figure 1 This is a schematic diagram of an exemplary interactive scenario according to an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the main method steps in an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram illustrating an exemplary information interaction application scenario according to an embodiment of the present invention;

[0048] Figure 4 This is an exemplary time-frequency division region diagram according to an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this specification clearer, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of the present invention.

[0050] As mentioned earlier, intelligent agents participating in unmanned operations can actively acquire target information using various sensing devices such as radar, cameras, ultrasound, and sensors, or directly exchange and share sensing information using communication methods. Intelligent agents can include unmanned vehicles, drones, unmanned ships, robots, and so on.

[0051] For example, when the intelligent agent is a robot, the robot performing unmanned tasks, after acquiring sensory information, interacts with other robots via communication information to form AI decisions, and then executes the business process, propelling the business state towards the target state. Information interaction is one of the key aspects of robots performing unmanned tasks. Current interaction modes can be divided into two types: indirect interaction and direct interaction. Figure 1 As shown.

[0052] The aforementioned indirect interaction mode can be a working mode in which intelligent agent 1 and intelligent agent 2 interact through a business platform. This can be implemented through a communication interface, which may include a wired communication interface or a wireless communication interface such as cellular network, Wi-Fi, or Bluetooth. The aforementioned direct interaction mode can be a working mode in which intelligent agent 1 and intelligent agent 2 communicate and interact directly through an intelligent interaction interface. This intelligent interaction interface may include a wireless communication interface, a data conversion communication interface based on sensing devices such as radar and cameras, and a data conversion communication interface based on ultrasound or voice. Both modes have their advantages and disadvantages and are complementary in application scenarios. For example, the indirect interaction mode can achieve a wider communication coverage and also supports interaction between intelligent agents and the business platform, offering strong functional scalability. The direct interaction mode, on the other hand, has shorter communication latency and is more efficient and secure than the indirect interaction mode.

[0053] In the aforementioned interaction model, the agent's perception and communication capabilities are separated. Specifically, the agent uses perception devices to perceive the business environment and target state, then exchanges key data and information through the interface of communication devices, and finally, based on the shared information, performs reasoning and decision-making to execute operations in the business process. Therefore, there are shortcomings in using this interaction model for agent interaction:

[0054] 1) The information interaction mode is a serial mode. The sending agent always needs to wait for the receiving agent to communicate after the perception is completed. The interaction time is extended. From the receiving agent's perception to the communication transmission, and then to the sending agent's decision-making, the response of multiple links affects the response time of the business, resulting in a poor user experience in general unmanned business.

[0055] 2) In a mobile communication network environment, the wireless communication performance between intelligent agents is affected by the business environment. Instability in communication between intelligent agents often occurs, and the interaction performance relying on the mobile communication network will be difficult to maintain, making it difficult to guarantee the quality and efficiency of unmanned business completion.

[0056] 3) The cost and size of communication and sensing devices that are separated from the intelligent agent are high, which is not conducive to robot integration design.

[0057] In view of this, this specification provides a scheme for intelligent agent information interaction that can couple the communication and perception capabilities of intelligent agents. The sending intelligent agent obtains the interactive information based on the integrated communication and perception signal without waiting for the receiving intelligent agent to communicate back after perception. As a first intelligent agent sending communication information, it can send a communication-sensing integrated signal to a second intelligent agent. This integrated signal carries modulation information of both communication and sensing signals, and includes a first instruction to the second intelligent agent. The first intelligent agent can detect sensing echo signals, which may include echo signals generated by the communication-sensing integrated signal under the influence of the executed action. The first intelligent agent can obtain action time-frequency data by detecting the sensing echo signals. This action time-frequency data may be time-frequency data after the second intelligent agent begins executing an action in response to the first instruction. The first intelligent agent can identify the action category of the action time-frequency data to determine action semantic instructions. The first intelligent agent stores action semantic instructions corresponding to any action category. Based on these action semantic instructions, the first intelligent agent can determine business operations, causing its business state in the business process between the first and second intelligent agents to progress towards a target state, or causing the first intelligent agent to progress towards / be in an interrupted state.

[0058] In the first aspect, please refer to Figure 2 This invention provides an intelligent agent information interaction method, which can be applied to a first intelligent agent. The first intelligent agent can interact with a second intelligent agent to conduct unmanned business processes. Each intelligent agent can make business operation decisions through interactive information, so that the business state of the intelligent agent progresses to the target state or interruption state in the business process. The aforementioned intelligent agent can be a device, also called an intelligent agent device. Intelligent agents all have communication capabilities and computing and instruction processing capabilities, and intelligent agents also have perception capabilities. In some possible implementations, the intelligent agent can include electronic devices and communication devices and perception devices connected to the electronic devices, or the intelligent agent can include integrated communication electronic devices and perception devices, or the intelligent agent can also be an integrated all-in-one device. The integration method can be, for example, through system-on-a-chip integration or bus integration of multiple chips on the same circuit board. The intelligent agent also has the ability to perform actions. For example, the intelligent agent also includes one or more robotic arms and / or one or more robotic legs, etc., intelligent mechanisms / robot mechanisms. In this embodiment of the invention, a robot can be described as an intelligent agent.

[0059] In this embodiment of the invention, the aforementioned intelligent agent information interaction method may include:

[0060] S1) Send a communication and sensing integrated signal to the second intelligent agent. The communication and sensing integrated signal carries modulation information of communication signals and sensing signals, and carries first instruction information for the second intelligent agent.

[0061] In some possible implementations, the integrated communication and sensing signal can be a signal coupling communication information and sensing information. A first intelligent agent can act as the sender of the integrated communication and sensing signal, and a second intelligent agent as the receiver. The modulation information of the sensing signal in the integrated communication and sensing signal can include the modulation information of wireless sensing (radar) signals, such as the modulation information of radar modulation symbols. Radar signals can be used to sense the service environment, particularly the action posture of the second intelligent agent. The modulation information of the communication signal in the integrated communication and sensing signal can include the modulation information of wireless / mobile communication signals, such as the modulation information of Orthogonal Frequency Division Multiplexing (OFDM) symbols. The first indication information can be communication information obtained by demodulating the communication signal. The first indication information can be used by the second intelligent agent to determine the service operation progressing towards the target state in the service flow between the first and second intelligent agents. The service operation in this embodiment of the invention can include one or more execution actions. Execution actions can include service actions or interactive actions (in some possible examples, a service action can simultaneously represent an interactive action), and there can be a sequence between the actions. The integrated communication and sensing signal enables intelligent agents to perform wireless sensing during wireless communication. The unit within the intelligent agent that transmits and receives the integrated communication and sensing signal can be called the integrated communication and sensing unit. This unit may include an integrated wireless signal processing module and an information processing module. In some possible implementations, when the intelligent agent communicates with the service platform, the integrated communication and sensing unit may only perform communication signal modulation, and its function may degenerate into a communication function, meaning that the emitted signal may only contain wireless communication signals.

[0062] In this embodiment of the invention, in order to achieve the coupling of the communication and perception capabilities of the intelligent agents, the first intelligent agent and the second intelligent agent may have the same system configuration before executing step S1).

[0063] In some possible implementations, an action semantic library can be configured in both the first and second intelligent agents. This action semantic library can be a semantic library of interactive actions, storing action categories for performing actions and the corresponding action semantic instructions for each action category. This allows the action semantic instructions corresponding to any action category to be retrieved during use, and the data can be recorded using a database. The intelligent agents can be configured with neural network models. The business platform can collect the robot's actions, define various action categories, and train an action recognition AI model based on action categories and actions labeled in an action semantic library. This action recognition AI model can have the same structure and parameter types as the neural network machine model configured within the intelligent agent. The trained model parameters can be sent to the intelligent agent upon request, and the untrained neural network machine model can be converted into a trained neural network machine model on the intelligent agent through parameter sharing (e.g., assignment according to an ordered dictionary). The trained neural network machine model can serve as the action inference model within the intelligent agent, thus eliminating the need for model training on the intelligent agent itself. This allows the intelligent agent to perform action recognition (classification) on the processed data and input the recognition results into the action semantic library to obtain the semantic representation of the action. Then, based on the action semantics and other relevant perceptual information, it decides on the next business operation (action to be executed). Other relevant perceptual information includes information from local cameras and sensors. Here, the unit in the intelligent agent that implements action inference can be called the AI ​​inference decision unit. It should be noted that the data format of the intelligent agent's actions can be images; action classification can be represented in the database using unique identifiers.

[0064] In some possible implementations, the intelligent agent is also configured with an (action) execution unit. The execution unit can select the corresponding action from the business action library and the interaction action library to execute based on the determined business operation (or the decision command corresponding to the business operation), thereby realizing the next business operation mentioned above in the business process, and being able to advance the business state or conduct information interaction.

[0065] In this embodiment of the invention, the aforementioned intelligent agent can be configured by a business platform. After configuration, the business platform can choose not to participate in the information interaction between the first and second intelligent agents. The aforementioned intelligent agent information interaction method may further include:

[0066] C1) Receives interaction configuration information issued by the business platform. The interaction configuration information is the configuration information between the first intelligent agent and the second intelligent agent. The interaction configuration information includes the model parameters of the action reasoning model and the signal parameters of the integrated communication and perception signal between the two agents.

[0067] C2) Based on the model parameters in the interactive configuration information, generate the current action inference model, which can be used to identify action classification through the obtained action time-frequency data.

[0068] In some possible implementations, depending on business needs, usage, and testing results, the intelligent agent can send a request to the business platform to obtain configuration information. This request may include the identification information of the intelligent agent that sent the request and the identification information of the intelligent agent that needs to interact, thereby enabling the business platform to send interaction configuration information to the first and second intelligent agents. This interaction configuration information can be used to identify function and communication resource configurations.

[0069] In some possible applications, the business platform sends information interaction configuration information to Agent 1 and Agent 2. This configuration information may include model parameters of the action inference model and signal (configuration) parameters of the integrated communication and sensing signal between the first and second agents. In some possible examples, after the interaction configuration information is sent, each agent can report a confirmation message to the business platform. The interaction configuration information can be stored in the agents indefinitely and may not be sent again unless the business platform updates the signal parameters and / or model parameters as needed. When the first and second agents are not within the coverage area of ​​the business platform, the first agent can directly send an information interaction request to the second agent. This request information can be carried by wireless communication signals, and the response action and / or response result can be determined by detecting the sensing echo signal, thus completing the communication and sensing capability confirmation before information interaction.

[0070] The aforementioned signal parameters may include time-frequency resource parameters, power, signal modulation method, and wireless signal encoding method for the integrated communication and sensing signals transmitted by both parties. Model parameters can be used to indicate the parameters of the neural network machine model, which includes all parameters of the trained neural network machine model and a buffer. The neural network machine model is used as the action inference model. Specifically, the time-frequency resource parameters in the signal parameters can be used to indicate the signal frame of the integrated communication and sensing signal between the two parties. The frame structure of the signal frame consists of a first subframe set for carrying communication information and a second subframe set for carrying sensing information.

[0071] The aforementioned first subframe set includes multiple subframes for indicating the interval at which the first agent sends communication signals, and multiple subframes for indicating the interval at which the second agent sends communication signals; the subframes for indicating that the first agent sends communication signals and the subframes for indicating that the second agent sends communication signals are adjacent frames.

[0072] The aforementioned second set of subframes includes multiple subframes for indicating the interval at which the first agent sends perception signals, and multiple subframes for indicating the interval at which the second agent sends perception signals; the subframes for indicating that the first agent sends perception signals and the subframes for indicating that the second agent sends perception signals are adjacent frames.

[0073] After configuring the signal parameters as described, the modulated integrated sensing signal can be transmitted on the configured resources. In any integrated sensing signal, the frequency division multiplexing modulation symbol with a specified modulation symbol number within the communication signal and the radar reference symbol with the same modulation symbol number within the sensing signal are modulated onto the same specified subcarrier. This enables integrated signal transmission and reception on the same frequency band using time-frequency multiplexing.

[0074] In this embodiment of the invention, the aforementioned intelligent agent information interaction method may further include:

[0075] S2) Detect the sensing echo signal to obtain action time-frequency data. The action time-frequency data is the time-frequency data after the second intelligent agent starts to execute the action in response to the first instruction information. The action time-frequency data is used to describe the time-frequency characteristics of the sensing echo signal. The sensing echo signal includes the echo signal generated by the integrated communication and sensing signal under the action.

[0076] In some possible implementations, after the second agent begins to execute an action in response to the first indication information, the action of the second agent will act on the integrated communication and sensing signal to generate a sensing echo signal. The time-frequency data of the integrated communication and sensing signal in the business environment of the second agent (or the first agent and the second agent) will change relative to when no action is executed. The time-frequency characteristics of the sensing echo signal will carry time-frequency characteristics representing this change, and the first agent will obtain the corresponding action time-frequency data, which can be represented by a time-frequency image (time-frequency diagram). It should be noted that this sensing echo signal is not the sensing signal sent by the second agent in step S2).

[0077] In this embodiment of the invention, the aforementioned intelligent agent information interaction method may further include:

[0078] S3) Identify the action classification of the action time-frequency data to determine the action semantic instruction. The first intelligent agent stores the action semantic instruction corresponding to any action classification.

[0079] In some possible implementations, the first intelligent agent identifies the action classification of the action time-frequency data through the aforementioned action reasoning model, and determines the action semantic instruction by querying the action semantic library based on the identified action classification.

[0080] In this embodiment of the invention, the aforementioned intelligent agent information interaction method may further include:

[0081] S4) Based on the action semantic instructions, determine the business operation, which can cause the business state of the first intelligent agent in the business process between the first intelligent agent and the second intelligent agent to progress towards the target state, or cause the first intelligent agent to progress towards the interruption state / be in the interruption state.

[0082] In some possible implementations, step S4) may include:

[0083] S401) Based on the action semantic instructions, determine the business state in the business execution process of the first intelligent agent and the second intelligent agent;

[0084] S402) Determine whether the business status is a target status or an interrupted status to determine the business operation in the business execution process. The business operation includes: a stop operation or an operation to be executed corresponding to the target status or the interrupted status.

[0085] In some possible implementations, the first intelligent agent can determine whether the business state has reached the target state or meets the interruption condition, and send an information interaction termination indication to the second intelligent agent, thus ending the information interaction. Furthermore, the business platform or the second intelligent agent can initiate the information interaction termination indication based on the business state.

[0086] In some possible examples, the business process could be a collaborative movement of objects by a first agent and a second agent. The first agent may have already placed the object in the object storage area of ​​the second agent, at which point the first agent can send a communication-sensing integrated signal to the second agent. The second agent can respond to the instruction information in the communication-sensing integrated signal by first swinging its right arm horizontally (an interactive action; the agent may have left and right robotic arms, and the horizontal swing can be a reciprocating motion along a single horizontal axis, indicating confirmation of the action), and then moving the object from the object storage area to the designated location (the horizontal swing of the right arm can be one of the movement actions, i.e., it can be one of the business actions, or it can be a separate interactive action). The first agent continuously detects and senses echo signals and obtains time-frequency data. It can use the time-frequency data after the second agent starts executing the right arm swinging motion (it is not necessary to determine whether the motion has started; it can be after a specified time or after the time-frequency data changes) as the motion time-frequency data (time-frequency data is relatively stable in a static business environment). The first agent inputs this motion time-frequency data into the motion inference model to obtain the motion classification, queries the motion semantic library to determine the motion semantic instructions, and obtains the business operation based on the motion semantic instructions. The first agent can perform a stop operation, such as temporarily stopping the robot arm's movement, or the first agent can perform a reset operation corresponding to the interrupted state, such as moving the robot arm to the initial position area and maintaining a specified posture, or the first agent can perform a pending operation corresponding to the business state, such as the robot arm moving the next object to the object temporary storage area.

[0087] After the action semantic instruction is determined, the aforementioned agent information interaction method further includes:

[0088] S5) Ignore the response result of the second agent to the communication signal, or record the information in the response result.

[0089] In some possible implementations, the response result can be completely consistent with the action semantic instruction, or it can be accompanied by perceptual information other than the action semantic instruction. For example, the action semantic instruction is information confirming the action, and the response result is information confirming the action and information that the designated location area is full. The first agent can, based on the record of information that the designated location area is full, place the new item in another item temporary storage area and interact with the second agent to carry out the information of the next designated location area item storage process.

[0090] In a second aspect, embodiments of the present invention also provide an intelligent agent information interaction method under the same inventive concept as the foregoing embodiments, which can be applied to the second intelligent agent in the foregoing embodiments to interact with the first intelligent agent in the foregoing embodiments. This intelligent agent information interaction method may include:

[0091] H1) Detects the integrated communication and sensing signal sent by the first intelligent agent, the integrated communication and sensing signal carrying modulation information of communication signal and sensing signal;

[0092] H2) Receives communication information carried by the integrated communication and sensing signal, wherein the communication information is a first instruction information from the first intelligent agent to the second intelligent agent;

[0093] H3) In response to the first instruction information, an action is started to be executed to carry out the business execution process of the first intelligent agent and the second intelligent agent, wherein the action classification of the action corresponds to at least one action semantic instruction.

[0094] In this embodiment of the invention, according to the needs of the business process, the second intelligent agent can execute the method executed by the first intelligent agent in the aforementioned embodiment and send a communication and sensing integrated signal to the first intelligent agent. The first intelligent agent can also execute the method executed by the second intelligent agent in this embodiment.

[0095] Please refer to some possible implementation methods. Figure 3 When the first intelligent agent (Agent 1) sends a communication-sensing integrated signal (hereinafter referred to as the integrated signal or integrated signal), the second intelligent agent (Agent 2) detects the communication signal in the integrated signal and receives the communication information, which can be the first indication information (obtained through information processing). Simultaneously, the first intelligent agent detects the sensing echo signal in the integrated signal to obtain the action time-frequency map of the second intelligent agent. The first intelligent agent inputs this action time-frequency map into the action recognition inference model to obtain the action classification result (action semantics), further determining the business operation (to achieve decision-making). Conversely, when the second intelligent agent sends the integrated signal, the first intelligent agent detects the communication signal in the integrated signal and receives the communication information, which can be the second indication information. Simultaneously, the second intelligent agent detects the sensing echo signal in the integrated signal to obtain the action time-frequency map of the first intelligent agent. The second intelligent agent inputs this action time-frequency map into the action recognition inference model to obtain the action classification result (action semantics), further determining the business operation (to achieve decision-making). Understandably, the first and second intelligent agents are not special types of devices, but rather distinguishable representations of the sender and receiver devices relative to each time an integrated signal is sent. The methods executed by the first intelligent agent can be executed by the second intelligent agent, and vice versa.

[0096] Based on the foregoing embodiments, this invention discloses an exemplary interactive application scenario implementation. In this scenario implementation, the agent information interaction method can be applied to agent 1 and agent 2, and the agent information interaction method may include:

[0097] P1) Configure the system.

[0098] In step P1), a semantic library of interactive actions for the intelligent agent (robot) is defined. A basic interactive action library can be set up, including {left arm upright, right arm upright, head shaking, left arm swinging horizontally, right arm swinging horizontally, right arm swinging, left arm swinging, left or right arm swinging forward, left or right arm swinging backward, left or right arm swinging downward, left or right arm swinging upward, arms crossed, arms hanging down}. The corresponding semantics are {interaction request, action instruction, request rejection, message confirmation, action confirmation, move right, move left, move forward, move backward, descend, stand up, rotate, stop}. Further, an extended interactive action library can be set up to supplement the basic interactive actions, such as left turn, right turn, identity inquiry, etc.

[0099] In some possible implementations, the business platform collects robot (tested) motion images and trains an AI model for motion recognition. Here, a Convolutional Neural Network (CNN) model is used, consisting of a motion feature extraction layer and a motion recognition layer. The feature extraction layer can be composed of 8 convolutional layers, 5 pooling layers, and 2 fully connected layers, with a ReLU activation function added after the convolutional layers to increase non-linearity. The extracted feature vectors are input to the motion recognition layer, which can then recognize interactive actions and business actions respectively, relative to the business actions.

[0100] During training, a loss function for multi-task recognition is designed for business actions and interactive actions, and the parameters of the training model are obtained through network optimization. Specifically, test agent 1 sends a communication-sensing integrated signal to test agent 2, test agent 2 selects and performs a specified interactive action, and test agent 1 obtains a time-frequency (distribution) map by detecting the echo signal, i.e., constructs a database of measured time-frequency images, selects m (e.g., m = 20) time-frequency images of the same action as the training set, and M (e.g., M = 10) time-frequency images as the test set.

[0101] In some possible examples, when designing the loss function, the weight of the interaction action loss function can be greater than that of the business action recognition loss function. Specifically, a center time-frequency image sample s is defined for the k-th interaction action. k Define a center loss function, which penalizes samples that are significantly distant from the action center sample in the feature space.

[0102]

[0103] Here, x i Let be the i-th sample out of m total samples, and w be the weight of the loss function. The total loss function is defined as L = wL 交互 +(1-w)L 业务 L here 业务 Identify loss functions for business actions.

[0104] The aforementioned intelligent agent information interaction method may also include:

[0105] P2) Agent 1 initiates a direct information interaction request to the business platform. The request includes the ID of the target agent 2.

[0106] The aforementioned intelligent agent information interaction method may also include:

[0107] The P3 business platform sends information interaction configuration information to Agent 1 and Agent 2, including at least action inference model parameters and synesthetic signal configuration parameters. Please refer to [link / reference here]. Figure 4 In some possible examples, the wireless signal uses OFDM modulation, and the frame structure of each signal frame can be divided into 10 subframes (each subframe can have multiple time slots). Subframes 0, 4, and 8 are used for agent 1 to send communication signal T1, subframes 1, 5, and 9 (subframes 0, 1, 4, 5, 8, and 9 in the first subframe set) are used for agent 2 to send communication signal T2, subframes 2 and 6 are used for agent 1 to send sensing signal S1, and subframes 3 and 7 (subframes 2, 3, 6, and 7 in the second subframe set) are used for agent 2 to send sensing signal S2. Subframes with a sequence number difference of 1 within the same subframe set can be adjacent frames.

[0108] The OFDM communication signal can be represented as:

[0109]

[0110] Where rect() is the rectangular pulse function, N is the number of subcarriers, and d n,k f is the k-th (positive integer) frequency domain communication modulation symbol modulated on the n-th (positive integer) subcarrier (k is the previously specified modulation symbol number). n Δf is the carrier frequency of the nth subcarrier, and Δf is the subcarrier modulation interval.

[0111] The sensed signal can be represented as:

[0112]

[0113] Where r n,k For the k-th radar reference symbol on the n-th subcarrier (the same modulation symbol number as k), each formula can be configured in the agent, and the sent signal parameters can include the parameters in formulas (2) and (3).

[0114] The aforementioned intelligent agent information interaction method may also include:

[0115] P4) Based on the interaction configuration information, agents 1 and 2 construct an action reasoning model, generate synesthetic signals, and send them on the configured resources;

[0116] P5) When agent 1 sends a synergistic signal, agent 2 detects the communication signal T1 in the synergy and receives the communication information d. At the same time, agent 1 detects the sensing echo signal in the synergy signal, obtains the time-frequency map of agent 2's actions, inputs it into the action recognition inference model to obtain the action classification result (action semantics), and further inputs it into the decision-making unit to determine the business operation.

[0117] P6) Agent 1 determines whether the business status has reached the target status or meets the interruption status, and sends an information interaction end indication message to Agent 2, thus ending the information interaction.

[0118] This invention realizes both communication interaction and perception interaction based on action coding (classifying action semantic commands according to action categories determined by time-frequency distribution characteristics). The communication and perception capabilities complement and enhance each other, improving the robustness of intelligent agent interaction. Furthermore, based on interactive action perception, it saves communication interaction processes and conserves bandwidth. It provides an information interaction foundation for intelligent agents such as multi-axis robotic arms, quadruped robots, and bipedal humanoid robots to perform unmanned business scenarios. It can also provide an information interaction foundation for intelligent agents such as autonomous vehicles and logistics robots to perform unmanned business scenarios (the action semantic library can define left lane change, right lane change, deceleration, etc.), and provides information interaction support for intelligent agents to enter households and become key assistants in daily life and production.

[0119] In a third aspect, embodiments of the present invention also provide an intelligent agent information interaction system under the same inventive concept as the foregoing embodiments, which can be applied to a first intelligent agent. This intelligent agent information interaction system may include:

[0120] The transmitting module is used to transmit an integrated communication and sensing signal to the second intelligent agent. The integrated communication and sensing signal carries modulation information of communication signals and sensing signals, and carries first indication information for the second intelligent agent.

[0121] The detection module is used to detect the sensing echo signal to obtain action time-frequency data. The action time-frequency data is the time-frequency data after the second intelligent agent starts to execute the action in response to the first instruction information. The action time-frequency data is used to describe the time-frequency characteristics of the sensing echo signal. The sensing echo signal includes the echo signal generated by the integrated communication and sensing signal under the action.

[0122] The identification module is used to identify the action classification of the action time-frequency data in order to determine the action semantic instructions. The first intelligent agent stores the action semantic instructions corresponding to any action classification.

[0123] The determination module is used to determine the business operation based on the action semantic instruction.

[0124] Specifically, the intelligent agent information interaction system also includes: a configuration module, which is used for:

[0125] The system receives interaction configuration information issued by the business platform. The interaction configuration information is the configuration information between the first intelligent agent and the second intelligent agent. The interaction configuration information includes model parameters of the action reasoning model and signal parameters of the integrated communication and perception signal between the two agents.

[0126] Based on the model parameters in the interactive configuration information, a current action inference model is generated. The current action inference model is used to identify action classification through the obtained action time-frequency data.

[0127] Specifically, the signal parameters are used to indicate the signal frame of the integrated communication sensing signal between the two.

[0128] The frame structure of the signal frame consists of a first subframe set for carrying communication information and a second subframe set for carrying perception information.

[0129] Specifically, the first subframe set includes multiple subframes for indicating the interval at which the first agent sends communication signals, and multiple subframes for indicating the interval at which the second agent sends communication signals.

[0130] The subframe used to indicate that the first agent is sending a communication signal is an adjacent frame to the subframe used to indicate that the second agent is sending a communication signal.

[0131] Specifically, the second set of subframes includes multiple subframes for indicating the interval at which the first agent sends perception signals, and multiple subframes for indicating the interval at which the second agent sends perception signals.

[0132] The subframe used to indicate that the first agent is sending a perception signal is an adjacent frame to the subframe used to indicate that the second agent is sending a perception signal.

[0133] Specifically, after configuring the signal parameters as described, the frequency division multiplexing modulation symbol with a specified modulation symbol number in the communication signal and the radar reference symbol with the same modulation symbol number in the sensing signal in any integrated communication and sensing signal are modulated onto the same specified subcarrier.

[0134] Specifically, the model parameters are used to indicate the parameters of the neural network machine model, which is used as the action reasoning model.

[0135] Specifically, both the first and second intelligent agents are configured with an action semantic library, which is used to store action semantic instructions corresponding to any action category.

[0136] Specifically, based on the action semantic instructions, the business operation is determined, including:

[0137] Based on the action semantic instructions, the business state in the business execution process of the first intelligent agent and the second intelligent agent is determined;

[0138] Determine whether the business status is a target status or an interrupted status to determine the business operation in the business execution process. The business operation includes: a stop operation or an operation to be executed corresponding to the target status or the interrupted status.

[0139] Specifically, the determining module is also used to ignore the response result of the second agent in response to the communication signal.

[0140] In a fourth aspect, embodiments of the present invention also provide an intelligent agent device under the same inventive concept as the foregoing embodiments. This intelligent agent device may include: at least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the aforementioned method by executing the instructions stored in the memory. The intelligent agent device may also possess the aforementioned action execution capability and communication sensing capability.

[0141] In a fifth aspect, embodiments of the present invention also provide a machine-readable storage medium under the same inventive concept as the foregoing embodiments, storing machine instructions that, when the machine instructions are run on a machine, cause the machine to perform the methods in the foregoing embodiments.

[0142] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention.

[0143] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not describe the various possible combinations separately.

[0144] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium may be non-transient and may include various media capable of storing program code, such as USB flash drives, hard disks, read-only memory (ROM), random access memory (RAM), flash memory, magnetic storage, and optical storage.

[0145] Furthermore, various different implementations of the present invention can be combined arbitrarily, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

1. An intelligent agent information interaction method, applied to a first intelligent agent, characterized in that, The intelligent agent's information interaction method includes: Send a communication and sensing integrated signal to the second intelligent agent. The communication and sensing integrated signal carries modulation information of communication signals and sensing signals, and carries first instruction information for the second intelligent agent. The sensing echo signal is detected to obtain action time-frequency data. The action time-frequency data is the time-frequency data after the second intelligent agent starts to execute the action in response to the first instruction information. The action time-frequency data is used to describe the time-frequency characteristics of the sensing echo signal. The sensing echo signal includes the echo signal generated by the integrated communication and sensing signal under the action. The action classification of the action time-frequency data is identified to determine the action semantic instructions. The first intelligent agent stores action semantic instructions corresponding to any action classification. Based on the action semantic instructions, the business operation is determined, specifically including: based on the action semantic instructions, determining the business state in the business execution process of the first intelligent agent and the second intelligent agent; determining whether the business state is a target state or an interrupted state, so as to determine the business operation in the business execution process, wherein the business operation includes: a stop operation, or an operation to be executed corresponding to the target state or the interrupted state.

2. The intelligent agent information interaction method according to claim 1, characterized in that, Before sending the integrated communication and sensing signal to the second agent, the agent information interaction method further includes: The system receives interaction configuration information issued by the business platform. The interaction configuration information is the configuration information between the first intelligent agent and the second intelligent agent. The interaction configuration information includes the model parameters of the action reasoning model and the signal parameters of the integrated communication and perception signal between the two agents. Based on the model parameters in the interactive configuration information, a current action inference model is generated. The current action inference model is used to identify action classification through the obtained action time-frequency data.

3. The intelligent agent information interaction method according to claim 2, characterized in that, The signal parameters are used to indicate the signal frame of the integrated communication sensing signal between the two; The frame structure of the signal frame consists of a first subframe set for carrying communication information and a second subframe set for carrying sensing information.

4. The intelligent agent information interaction method according to claim 3, characterized in that, The first subframe set includes multiple subframes for indicating the interval at which the first agent sends communication signals, and multiple subframes for indicating the interval at which the second agent sends communication signals. The subframe used to indicate that the first agent is sending a communication signal is an adjacent frame to the subframe used to indicate that the second agent is sending a communication signal. The second set of subframes includes multiple subframes for indicating the interval at which the first agent sends perception signals, and multiple subframes for indicating the interval at which the second agent sends perception signals. The subframe used to indicate that the first agent sends a sensing signal is an adjacent frame to the subframe used to indicate that the second agent sends a sensing signal. After the signal parameters are configured as described, the frequency division multiplexing modulation symbol with a specified modulation symbol number in the communication signal and the radar reference symbol with the same modulation symbol number in the sensing signal in any integrated communication and sensing signal are modulated onto the same specified subcarrier.

5. The intelligent agent information interaction method according to claim 3, characterized in that, The model parameters are used to indicate the parameters of the neural network machine model, which is used as the action reasoning model. Both the first and second intelligent agents are configured with an action semantic library, which is used to store action semantic instructions corresponding to any action category.

6. A method for intelligent agent information interaction, applied to a second intelligent agent, characterized in that, The intelligent agent's information interaction method includes: The integrated communication and sensing signal sent by the first intelligent agent is detected, the integrated communication and sensing signal carrying modulation information of communication signal and sensing signal; Receive communication information carried by the integrated communication and sensing signal, wherein the communication information is a first instruction information from the first intelligent agent to the second intelligent agent; In response to the first indication information, an action is initiated to execute the business execution flow of the first intelligent agent and the second intelligent agent. The action classification of the executed action corresponds to at least one action semantic instruction. The time-frequency data after the second intelligent agent initiates the action in response to the first indication information is used by the first intelligent agent to identify the action classification and determine the action semantic instruction. Based on the action semantic instruction, the business state in the business execution flow of the first intelligent agent and the second intelligent agent is determined. It is determined whether the business state is a target state or an interrupted state to determine the business operation in the business execution flow. The business operation includes: a stop operation or an operation to be executed corresponding to the target state or the interrupted state.

7. An intelligent agent information interaction system, characterized in that, The intelligent agent information interaction system includes: The transmitting module is used to transmit an integrated communication and sensing signal to the second intelligent agent. The integrated communication and sensing signal carries modulation information of communication signals and sensing signals, and carries first indication information for the second intelligent agent. The detection module is used to detect the sensing echo signal to obtain action time-frequency data. The action time-frequency data is the time-frequency data after the second intelligent agent starts to execute the action in response to the first instruction information. The action time-frequency data is used to describe the time-frequency characteristics of the sensing echo signal. The sensing echo signal includes the echo signal generated by the integrated communication and sensing signal under the action. The identification module is used to identify the action classification of the action time-frequency data in order to determine the action semantic instructions. The intelligent agent information interaction system stores the action semantic instructions corresponding to any action classification. The determination module is used to determine business operations based on the action semantic instructions, specifically including: determining the business state in the business execution process of the intelligent agent information interaction system and the second intelligent agent based on the action semantic instructions; determining whether the business state is a target state or an interrupted state, so as to determine the business operations in the business execution process, wherein the business operations include: a stop operation, or an operation to be executed corresponding to the target state or the interrupted state.

8. An intelligent agent device, characterized in that, The intelligent agent device includes: At least one processor; A memory connected to the at least one processor; The memory stores instructions executable by the at least one processor, which implements the method described in any one of claims 1 to 6 by executing the instructions stored in the memory.

9. A machine-readable storage medium, characterized in that, The machine contains machine instructions that, when executed on the machine, cause the machine to perform the method described in any one of claims 1 to 6.

Images