A robust cooperative sensing method against communication delay

Abstract

The application provides a robust cooperative perception method for communication delay, comprising data input coding network extraction perception features; cooperative communication is carried out based on the perception features, the received perception features with different delay times from other intelligent agents are aligned and fused at the master intelligent agent to obtain asynchronous features and edge feature fusion attention weights when the master intelligent agent receives; the master intelligent agent stores historical features based on the asynchronous features, and the historical features and the edge feature fusion attention weights pass through a delay compensation network to obtain a modulation result through feature weight symbiotic prediction and time modulation by using feature weight symbiotic prediction and time modulation; and the modulation result is decoded into final perception by using a decoder and output. A new delay compensation network is used to realize feature synchronization after communication. The network is based on a double-branch pyramid long short-term memory architecture, promotes synchronous evaluation of two types of key cooperative information (including features and cooperative attention weights), and enhances each other.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and image processing, and in particular, to a robust cooperative perception method for communication delay. BACKGROUND

[0002] In recent years, cooperative perception has shown great potential in improving perception ability, and has shown irreplaceable importance in many fields such as autonomous driving, emergency rescue, robot navigation, and UAV formation. Cooperative perception enables agents to share perception information through a communication network, fundamentally improving perception ability and overcoming physical limitations that have long plagued single-agent perception, such as occlusion and long-range problems. With the gradual development of deep neural networks and multi-instance learning in the field of cooperative perception, more and more research has begun to consider more realistic scenarios. Communication systems, as the core of cooperative perception, are being widely studied.

[0003] At present, most research has begun to consider some more realistic scenarios, such as how to improve the cooperative perception system to balance communication cost and perception performance. However, previous research has not considered the imperfect communication in the cooperative system. In practical applications, communication systems inevitably encounter delay problems, which will lead to potential performance degradation and pose high risks in safety-critical applications such as autonomous driving. SUMMARY

[0004] In view of the defects in the prior art, the purpose of the present application is to provide a robust cooperative perception method for communication delay.

[0005] According to one aspect of the present application, a robust cooperative perception method for communication delay is provided, comprising:

[0006] The data input encoding network extracts perception features;

[0007] Based on the perception features, cooperative communication is performed, and the received perception features with different delay times from other agents are aligned and fused at the master agent to obtain asynchronous feature and edge feature fusion attention weights at the time of receiving by the master agent;

[0008] Based on the asynchronous features, historical features are obtained, and the historical features and edge feature fusion attention weights are compensated through a delay compensation network, and through feature weight coexistence prediction and time modulation, a modulation result is obtained;

[0009] The modulation result is decoded into final perception by a decoder and output.

[0010] Preferably, the data input encoding network needs to be preprocessed before;

[0011] The preprocessing comprises: converting the original point cloud data obtained by each agent into bird's eye view data.

[0012] Preferably, the inputting data into the encoding network to extract perception features comprises:

[0013]

[0014] wherein, represents the original observation data of the i-th agent at the time stamp t, f encoder (·) represents an encoding network, represents the perception feature of the i-th agent at the time stamp t.

[0015] Preferably, the collaborative communication based on the perception features comprises: aligning and fusing the received perception features with different delay times from other agents at the master agent to obtain the asynchronous feature and the edge feature fusion attention weight when the master agent receives.

[0016]

[0017] wherein, wherein, represents the original observation data of the master agent i at the time stamp t (the superscript indicates the time stamp corresponding to the data), f encoder (·) represents an encoding network, represents the perception feature of the master agent i at the time stamp t (the superscript indicates the time stamp corresponding to the data), represents the time delay of transmitting data from the collaborative agent j to the master agent i; represents the perception feature of the collaborative agent j at the time stamp t-τ j→i generated by the collaborative agent j, i.e., the asynchronous feature when the master agent receives, represents the original observation data of the collaborative agent j at the time stamp t-τ j→i

[0018]

[0019] wherein, wherein, represents the perception feature of the master agent i at the time stamp t (the superscript indicates the time stamp corresponding to the data), f attention (·) represents an attention calculation network, represents the perception feature of the agent j at the time stamp t-τ j→i (the superscript indicates the time stamp corresponding to the data), represents the time delay of transmitting data from the collaborative agent j to the master agent i; represents the calculated edge feature fusion attention weight.

[0020] ​Preferably, the feature weight symbiotic prediction of the delay compensation network comprises sequentially performing feature estimation, cooperative attention weight estimation and feature-weight combination, wherein:

[0021] The feature estimation is to construct a time series by using historical features from the same agent to estimate the features at the current determination time;

[0022] The cooperative attention weight estimation is to estimate the cooperative attention weight at the determination time by using the real-time features perceived by the main agent and the historical features perceived by its cooperative agent, and to determine the cooperative region with the largest current information amount;

[0023] The feature-weight combination is to use the estimated features and cooperative attention weights in the previous timestamp as the input of the next timestamp of either branch of the feature estimation branch and the cooperative attention prediction branch of the double long short-term memory network, to mutually cooperate the features and the corresponding attention weights, and to obtain a feature-weight combination feature map.

[0024] Preferably, the feature estimation comprises:

[0025] The feature estimation is to construct a time series by using historical features from the same agent to estimate the features at the current determination time t;

[0026]

[0027] wherein each agent stores k frames of historical features in memory, represents the estimated feature of the cooperative agent j at timestamp t after synchronization, represents the perceived feature of the main agent i at timestamp t, represents the perceived feature of agent j at timestamp t-τ j→i -k, represents the time delay of transmitting data from cooperative agent j to main agent i, represents a time series composed of k frames of historical features stored by agent j, c F (·) represents the feature estimation branch of the long short-term memory estimation network.

[0028] Preferably, the cooperative attention weight estimation comprises:

[0029] The cooperative attention weight at time t is estimated by using the real-time features perceived by the main agent i and the historical features perceived by its cooperative agent j, and the cooperative region with the largest current information amount is determined, and the formula is as follows:

[0030]

[0031] wherein, These are the estimated collaborative attention weights between the main agent i and the nth agent at timestamp t. This represents the time delay in transmitting data from collaborating agent j to the master agent i. This represents the real-time characteristics of the perception of the main intelligent agent i at timestamp t. c represents the k-frame historical features stored by agent j. W (·) represents the collaborative attention prediction branch in the pyramid-long short-term memory estimation network.

[0032] Preferably, the time modulation of the delay compensation network includes:

[0033] Use the delay time as input;

[0034] The estimated features obtained in the dual long short-term memory network / Collaborative Attention Weight Asynchronous characteristics of the main intelligent agent receiving data / Edge feature fusion attention weight The two are merged, and the fusion process is as follows:

[0035]

[0036]

[0037]

[0038]

[0039] Among them, let This represents the confidence matrix reflecting the level of uncertainty in the estimation for each spatial region. Indicates the delay time The result of expansion and Delay tensors of the same shape Indicates the delay time The result of expansion and Delay tensors of the same shape, mF(·) and mW(·), represent convolutional neural networks with sigmoid activation functions. This represents a matrix in which all elements are 1;

[0040] The final fusion features are obtained as the modulation result:

[0041] Preferably, the detection result is decoded into a final sense result using a decoder and then output, including:

[0042]

[0043] wherein, represents the final perception output of the main agent i at the time stamp t, is the final fused feature, representing the estimated feature of the main agent i at the time stamp t after incorporating the predicted collaborative information, f decoder (·) represents the decoding network.

[0044] Preferably, the loss function of the perception method is:

[0045]

[0046] wherein, λ o , λ o , λ o , λ o respectively represent the weight of each loss, l output (·) represents the final perception loss, l fusion (·), l feature (·), l weight (·) respectively represent the loss in the fusion feature process, the intermediate estimated feature process and the estimated collaborative attention weight process. represents the true value of the final perception output of the main agent i at the time stamp t, represents the final perception output of the main agent i at the time stamp t; represents the true value of the estimated feature of the main agent i after aggregating real-time collaborative information at the time stamp t, represents the estimated feature of the main agent i after aggregating real-time collaborative information at the time stamp t; represents the true value of the feature map of the main agent i at the time stamp t, represents the feature map of the main agent i at the time stamp t; represents the true value of the collaborative attention weight of the collaborative agent j and the main agent i at the time stamp t; represents the collaborative attention weight of the collaborative agent j and the main agent i at the time stamp t.

[0047] Compared with the prior art, the present application has the following beneficial effects:

[0048] In an embodiment of the present application, a robust collaborative perception method for communication delay is proposed, which adopts a new delay compensation network to realize feature synchronization after communication.

[0049] The method for robust cooperative perception for communication delay proposed in another embodiment of the application is a delay compensation network based on a double-branch pyramid long short-term memory architecture, which promotes the synchronous evaluation of two types of key cooperative information (including features and cooperative attention weights) and mutual enhancement; the network promotes robust multi-agent perception by reducing the influence of inevitable communication delay, and enhances the performance and security of the cooperative perception system in a real communication scenario.

[0050] The method for robust cooperative perception for communication delay proposed in another preferred embodiment of the application is the first to solve the problem of communication delay in a cooperative perception task. BRIEF DESCRIPTION OF DRAWINGS

[0051] Other features, objects and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments with reference to the attached drawings:

[0052] Figure 1 A flowchart of the method for robust cooperative perception for communication delay in an embodiment of the application;

[0053] Figure 2 A framework diagram of the method for robust cooperative perception for communication delay in another embodiment of the application. DETAILED DESCRIPTION

[0054] The application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the application, but do not limit the application in any form. It should be noted that for those skilled in the art, without departing from the concept of the application, a number of modifications and improvements can be made. These all belong to the protection scope of the application.

[0055] The application provides an embodiment, a method for robust cooperative perception for communication delay, comprising:

[0056] S100, data input encoding network extracts perception features;

[0057] S200, cooperative communication based on the perception features obtained in S100, aligns and fuses the received perception features with different delay times from other agents at the master agent, and obtains asynchronous features and edge feature fusion attention weights at the time of receiving the master agent;

[0058] S300, obtain historical features based on the asynchronous features in S200, and pass the historical features and the edge feature fusion attention weights obtained in 200 through a delay compensation network, through feature weight coexistence prediction and time modulation, to obtain a modulation result;

[0059] S400, using a decoder to decode the modulation result obtained by S300 into final perception and output.

[0060] Figure 1 A flowchart of a robust cooperative perception method for communication delay based on the above embodiment for further optimization, comprising:

[0061] S11, data preprocessing: the original point cloud data obtained by each agent observation is processed to convert it into bird's eye view data.

[0062] S12, feature extraction: the bird's eye view data obtained by preprocessing is used to extract feature map using encoding network.

[0063] S13, cooperative communication: using the received perception features with different delay times from other agents, generate perception features at the master agent;

[0064] S14, through the delay compensation network, obtain the feature-weight combined feature map of the historical features of other agents received by the master agent;

[0065] S15, using the delay time as input, weighting and choosing between the fusion feature and the cooperative attention weight and the asynchronous feature and the cooperative attention when the master agent receives, to obtain the modulation result;

[0066] S16, detection result generation: using the decoder network to generate the final perception output.

[0067] In a preferred embodiment of the present application, S12 feature extraction is implemented. Specifically: using the encoding network to extract the feature map of the bird's eye view data obtained by preprocessing, the formula is as follows:

[0068]

[0069] Wherein, represents the original observation data of the i-th agent at time stamp t, f encoder (·) represents the encoding network, represents the perception feature of the i-th agent at time stamp t.

[0070] In a preferred embodiment of the present application, S13 is implemented based on the perception feature obtained in S12, and cooperative communication is performed. Specifically: using the received perception features with different delay times from other agents, generate perception features at the master agent, the formula is as follows:

[0071]

[0072] Wherein, represents the original observation data of the i-th agent at time stamp t, fencoder (·) represents a coding network, represents the perception feature of the main agent i at the time stamp t, represents the time delay of transmitting data from the cooperative agent j to the main agent i, represents the perception feature of the cooperative agent j at the time stamp t-τ j→i the perception feature generated by the cooperative agent j, that is, the asynchronous feature when the main agent receives.

[0073] Based on the above asynchronous feature, the edge feature fusion attention weight is generated, and the formula is as follows:

[0074]

[0075] wherein, wherein, represents the perception feature of the main agent i at the time stamp t (the superscript indicates the time stamp corresponding to the data), f attention (·) represents an attention calculation network, represents the perception feature of the agent j at the time stamp t-τ j→i (the superscript indicates the time stamp corresponding to the data), represents the time delay of transmitting data from the cooperative agent j to the main agent i; represents the edge feature fusion attention weight calculated.

[0076] In another preferred embodiment of the present application, S14 is implemented, and the historical features of other agents received by the main agent are compensated through a delay compensation network to obtain a feature-weight combined feature map.

[0077] In this embodiment, the delay compensation network includes two parts, one part is feature weight co-occurrence prediction based on a pyramid-long short-term memory estimation network, and the other part is time modulation. The pyramid-long short-term memory estimation network represents: a network obtained by replacing the matrix multiplication in the long short-term memory network with a multi-dimensional convolution structure, which is used to model a series of historical cooperation information and estimate the current state to capture spatially related cooperative features.

[0078] wherein, the feature weight co-occurrence prediction includes sequentially performed S141 feature estimation, S142 cooperative attention weight estimation and S143 feature-weight combination.

[0079] In a preferred embodiment, S141 is implemented, and the feature estimation is to use the historical features from the same agent to construct a time series to estimate the feature at the current determination time;

[0080] Further,

[0081]

[0082] where each agent can store k frames of history features in memory, denotes the estimated feature of the collaborative agent j at time stamp t after synchronization, denotes the perceived feature of the master agent i at time stamp t, denotes the perceived feature of the agent j at time stamp t - τ j→i -k, denotes the time delay of transmitting data from the collaborative agent j to the master agent i; c F denotes the feature estimation branch of the pyramid-LSTM estimation network.

[0083] In a preferred embodiment, S142 is implemented to estimate the collaborative attention weight at the current time using the real-time feature perceived by the master agent and the history features perceived by its collaborative agents, and to determine the collaborative region with the largest amount of current information;

[0084] Further,

[0085]

[0086] where, is the estimated collaborative attention weight of the master agent i and the nth agent at time stamp t, denotes the time delay of transmitting data from the collaborative agent j to the master agent i, denotes the perceived feature of the master agent i at time stamp t, c W denotes the collaborative attention prediction branch in the pyramid-LSTM estimation network.

[0087] In a preferred embodiment, S143 is implemented to combine the feature and the weight. Specifically, the two branches of the double-LSTM network share the same input, and the estimated feature map and the collaborative attention weight at the previous time stamp are used as the input of any branch at the next time stamp to realize mutual coordination of the estimation of the feature and the corresponding attention weight, and finally obtain the feature-weight combined feature map. The specific process is as follows:

[0088]

[0089] where, where τ denotes the delay time, k denotes the history frame, t0 denotes the current time, f attention denotes the attention calculation network, and denote the collaborative attention weight and the feature of the collaborative agent j to the master agent i at time stamp t, respectively, while and denote the estimated values of the collaborative attention weight and the feature of the collaborative agent j to the master agent i at time stamp t, respectively, e(t) This represents the input to the pyramid-shaped long short-term memory at timestamp t. represents the hidden state and unit state of the pyramid long short-term memory in each branch, respectively. pF and pw represent the feature extraction part in the feature and collaborative attention prediction network, respectively.

[0090] In another preferred embodiment of the present invention, the result is based on S14 in the above embodiment. and Implement S15. Specifically,

[0091] A trade-off is made between the fused features / cooperative attention weights obtained in the network using the delay time as input and the asynchronous features / cooperative attention weights received by the main agent. The principle is as follows:

[0092]

[0093]

[0094]

[0095]

[0096] Among them, let This represents the confidence matrix reflecting the level of uncertainty in the estimation for each spatial region. Indicates the delay time The result of expansion and Delay tensors of the same shape Indicates the delay time The result of expansion and Delay tensors of the same shape, m F (·) and m W (·) represents a lightweight convolutional neural network with a sigmoid activation function. This represents a matrix where all elements are 1. The confidence interval for feature estimation of each spatial region is obtained by concatenating the estimated collaborative features / collaborative attention weights, the latest asynchronous features / attention weights, and the delay tensor using a two-branch feature-weight synchronous delay compensation network. Based on the confidence matrix, the estimated collaborative features / collaborative attention weights and the latest asynchronous features / attention weights (edge ​​features fused with attention weights) are combined to output features that are more similar to those without delay.

[0097] Thus, we can make choices and obtain results. and Obtain the final fusion features

[0098]

[0099] In another preferred embodiment of the present application, S16 is implemented. That is, based on the modulation results obtained in the above embodiments, i.e., the final fusion features The final perception output is generated by the decoder network. The formula is as follows:

[0100]

[0101] wherein, represents the final perception output of the main agent i at the time stamp t, represents the estimated feature of the main agent i at the time stamp t after combining the predicted collaboration information, f decoder (·) represents the decoding network.

[0102] In another preferred embodiment of the present application,

[0103] The loss function is as follows

[0104]

[0105] wherein, λ o , λ o , λ o , λ o respectively represent the weight of each loss, l output (·) represents the final perception loss, l fusion (·), l feature (·), l weight (·) respectively represent the loss in the fusion feature process, the intermediate estimated feature process and the estimated collaboration attention weight process. represents the true value of the final perception output of the main agent i at the time stamp t, represents the final perception output of the main agent i at the time stamp t; represents the true value of the estimated feature of the main agent i at the time stamp t after aggregating the real-time collaboration information, represents the estimated feature of the main agent i at the time stamp t after aggregating the real-time collaboration information; represents the true value of the feature map of the main agent i at the time stamp t, represents the feature map of the main agent i at the time stamp t; represents the true value of the collaboration attention weight of the collaboration agent j and the main agent i at the time stamp t; represents the collaboration attention weight of the collaboration agent j and the main agent i at the time stamp t.

[0106] It should be explained that the steps in the method provided by the present application can be realized by the corresponding modules, devices, units, etc. in the system, and those skilled in the art can refer to the technical solutions of the system to realize the step flow of the method, that is, the embodiments in the system can be understood as preferred examples of realizing the method, and here will not be described in detail.

[0107] Those skilled in the art know that, in addition to implementing the system provided by the present application and each device thereof in a pure computer readable program code manner, the same functions can also be realized by logically programming the method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. Therefore, the system provided by the present application and each device thereof can be considered as a hardware component, and the devices included therein for realizing various functions can also be considered as structures within the hardware component; the devices for realizing various functions can also be considered as both software modules for realizing methods and structures within the hardware component.

[0108] The specific embodiments of the present application are described above. It should be understood that the present application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or changes within the scope of the claims, which does not affect the essential content of the present application. The above preferred features are combinable in any combination without conflict.

Claims

1. A robust cooperative sensing method for communication delay, characterized in that, include: The data is input into an encoding network to extract perceptual features; Based on the aforementioned perceptual features, collaborative communication is performed. Perceptual features with different delay times received from other agents are aligned and fused at the main agent to obtain the attention weight of asynchronous features and edge features received by the main agent. Historical features are obtained based on the asynchronous features, and the historical features and edge features are fused with attention weights and passed through a delay compensation network. Through feature weight co-occurrence prediction and time modulation, the modulation result is obtained. The modulation result is decoded into the final sensor image and output using a decoder. The feature weight co-occurrence prediction of the delay compensation network includes sequential feature estimation, cooperative attention weight estimation, and feature-weight combination, wherein: The feature estimation involves constructing a time series using historical features from the same agent and estimating the features at the current, determined moment. The collaborative attention weight estimation is to estimate and determine the collaborative attention weight at a given moment by using the real-time features perceived by the main agent and the historical features perceived by its collaborating agents, and to determine the collaborative region with the largest amount of information at the current moment. The feature-weight combination utilizes the features estimated in the previous timestamp and the collaborative attention weights as input to the next timestamp of either the feature estimation branch or the collaborative attention prediction branch of the dual long short-term memory network. The features and the corresponding attention weights are then coordinated to obtain the feature-weight combined feature map. The feature estimation includes: The feature estimation involves constructing a time series using historical features from the same agent and estimating the features at the current, determined time t. Each agent stores k frames of historical features in memory. This represents the estimated features of the cooperating agent j at timestamp t after synchronization. This indicates that agent j is at timestamp t-τ j→i Perceptual features at -k, τ j→i This represents the time delay in transmitting data from collaborating agent j to the master agent i. c represents the time series composed of k frames of historical features stored by agent j. F (·) represents the feature estimation branch of the Long Short-Term Memory estimation network; The collaborative attention weight estimation includes: The collaborative attention weight at time t is estimated by using the real-time features perceived by the main agent i and the historical features perceived by its collaborating agent j, and the collaborative region with the most information is determined, as shown in the following formula: in, τ is the estimated collaborative attention weight between the main agent i and the nth agent at timestamp t. j→i This represents the time delay in transmitting data from collaborating agent j to the master agent i. This represents the real-time characteristics of the perception of the main intelligent agent i at timestamp t. c represents the k-frame historical features stored by agent j. W (·) represents the collaborative attention prediction branch in the pyramid-long short-term memory estimation network.

2. The robust cooperative sensing method for communication delay according to claim 1, characterized in that, The data needs to be preprocessed before being input into the encoding network; The preprocessing includes converting the raw point cloud data acquired by each agent into bird's-eye view data.

3. The robust cooperative sensing method for communication delay according to claim 2, characterized in that, The step of inputting data into an encoding network to extract perceptual features includes: in, f represents the raw observation data of the i-th agent at timestamp t. encoder (·) represents the coding network. This represents the perceptual features of the i-th agent at timestamp t.

4. A robust cooperative sensing method for communication delay according to claim 2, characterized in that, The cooperative communication based on the perceived features involves aligning and fusing the perceived features with different delay times received from other agents at the main agent to obtain the asynchronous features and edge features received by the main agent, including: in, f represents the raw observation data of the main agent i at timestamp t. encoder (·) represents the coding network. τ represents the perceptual features of agent i at timestamp t. j→i This represents the time delay in transmitting data from the collaborating agent j to the master agent i. Indicates the time stamp t-τ j→i The perceptual features generated by the collaborative agent j are the asynchronous features received by the main agent. Indicates the time stamp t-τ j→i The raw observation data of the collaborative agent j; in, f represents the perceptual features of the main agent i at timestamp t. attention (·) indicates an attention computation network. This indicates that agent j is at timestamp t-τ j→i Perceptual features at location, τ j→i This represents the time delay in transmitting data from the collaborating agent j to the master agent i. This represents the calculated attention weights for edge feature fusion, and the superscript t indicates the timestamp corresponding to the data.

5. A robust cooperative sensing method for communication delay according to claim 1, characterized in that, The time modulation of the delay compensation network includes: Use the delay time as input; Estimated features obtained in the dual long short-term memory network Collaborative attention weights Asynchronous characteristics of the main intelligent agent receiving data Edge feature fusion attention weight The two are merged, and the fusion process is as follows: Among them, let This represents the confidence matrix reflecting the level of uncertainty in the estimation for each spatial region. Indicates the delay time The result of expansion and Delay tensors of the same shape Indicates the delay time The result of expansion and Delay tensors of the same shape, m F (·) and m W (·) represents a convolutional neural network with a sigmoid activation function. This represents a matrix in which all elements are 1; The final fusion features are obtained as the modulation result:

6. The robust cooperative sensing method for communication delay according to claim 5, characterized in that, The modulation result is decoded into the final sensing result and output using a decoder, including: in, This represents the final perceived output of the main agent i at timestamp t. For the final fused features, let f represent the estimated features of agent i at timestamp t after incorporating the predicted collaborative information. decoder (·) indicates the decoding network.

7. The robust cooperative sensing method for communication delay according to claim 6, characterized in that, The loss function of the perception method is: Where, λ o , λ f , λ fe , λ w These represent the weights of each type of loss. This represents the final perceived loss. These represent the losses during the feature fusion process, the intermediate feature estimation process, and the collaborative attention weight estimation process, respectively. This represents the truth value of the final perceived output of the main agent i at timestamp t. This represents the final perceived output of the main intelligent agent i at timestamp t; This represents the true value of the estimated features of agent i after aggregating real-time collaborative information at timestamp t. This represents the final fused feature of the main intelligent agent i after aggregating real-time collaborative information at timestamp t; Let represent the truth value of the feature map of the main agent i at timestamp t. This represents the feature map of the main agent i at timestamp t; This represents the truth value of the collaborative attention weights between the collaborating agent j and the lead agent i at timestamp t. This represents the collaborative attention weights between the collaborating agent j and the lead agent i at timestamp t.

Images