A collaborative control method for heterogeneous traffic cameras based on multi-agent

Abstract

The present invention proposes a multi-agent-based heterogeneous traffic camera collaborative control method, which belongs to the fields of reinforcement learning technology and edge computing technology. The method includes: in the target selection stage, the agent combines its available computing resources with the target uncertainty data to generate an embedding vector; in the participation control stage, the participation control module receives the embedding vector, extracts the coarse-grained features of the embedding vector through the multi-head self-attention block, and transmits the coarse-grained features to the participation action selection block; the participation action selection block decides whether the agent should participate in the task at the current time step to save computing time; in the rotation control stage, the rotation control module of the agent extracts fine-grained features through the dual self-attention block for the rotation action selection block to determine the optimal rotation action to reduce target uncertainty. The present invention simultaneously minimizes decision-making time and target uncertainty, intelligently controls the rotation of multiple traffic cameras, and tracks multiple moving targets.

Description

Technical Field

[0001] The present invention belongs to the field of reinforcement learning technology and edge computing technology, and in particular relates to a collaborative control method for heterogeneous traffic cameras based on multi-agents. Background Art

[0002] Traditional visual target tracking tasks mainly focus on estimating the bounding box of the target of interest in a series of image frames. In addition, the multi-target tracking task is dedicated to solving the challenge of simultaneously tracking multiple targets within the field of view. However, when the target is occluded or moves out of the field of view, the tracking accuracy will drop significantly. To alleviate this problem, active target tracking methods are proposed, which dynamically adjust the camera according to the visual scene to ensure that the target is always within the field of view. Active target tracking has been widely used in scenarios such as drones, intelligent robots, and sports events. The core of the target tracking problem is to maximize the number of targets observed or covered, or minimize the uncertainty of the target, using a limited number of cameras and robots as intelligent agents.

[0003] The decentralized coverage planner D2COPLAN generates a compact representation by predicting the map and shares it among multiple robots through a parameter-sharing graph neural network to achieve coverage. However, this method relies on data with real actions as training labels, which are often difficult to obtain or time-consuming. In recent years, advances in collaborative multi-agent reinforcement learning have significantly improved the efficiency of solving such collaborative problems, and can achieve better results without explicit labels. For example, the search tree is pruned by Monte Carlo tree search to confirm the optimal camera action, with the goal of achieving continuous target tracking and maximizing target coverage under real-time camera control. This method increases flexibility by allowing the camera to rotate and move along a predefined path. However, its fully centralized approach is limited by the curse of dimensionality, which can be alleviated by a distributed approach.

[0004] In a distributed system, each agent extracts features to select primitive actions. For example, in DARL1N, each agent makes independent decisions by interacting with its one-hop neighbors. The curse of dimensionality becomes more pronounced as the number of agents and targets increases. EPC addresses the scalability challenge of multi-agent and multi-target scenarios through a curriculum learning paradigm. In real-world scenarios, agents estimate target states from noisy observations using sensors (such as radar, lidar, or cameras). These observations are subject to measurement error, leading to observation uncertainty. PUARL addresses observation uncertainty in target tracking through exploration guidance based on probability estimation and policy adjustment based on uncertainty perception. However, these approaches fail to address the challenge of observation heterogeneity. Differences in field of view parameters (such as range and viewing angle) and measurement noise (including distance and angle noise) contribute to the uncertainty of target state estimates.

[0005] Furthermore, resource heterogeneity leads to differences in decision times among different agents, as a device's resource usage fluctuates dynamically due to the dynamics of other running services. Extreme edge computing further expands the application of edge computing by enabling real-time data processing directly on resource-constrained end devices (i.e., extreme edge devices). This paradigm creates new possibilities for industry, commerce, and individuals to utilize idle computing resources. However, because users can access their devices at any time and run intensive services, resource usage becomes highly dynamic. This volatile resource usage poses significant challenges for the real-time coordination of large-scale multi-agent systems in dynamic extreme edge computing environments.

[0006] Therefore, adopting effective control strategies is crucial in large-scale heterogeneous agent scenarios. Enhancing inter-agent collaboration in such environments is also a key challenge. To address these challenges, we propose a zero-shot generalization heterogeneous multi-agent reinforcement learning method that simultaneously accounts for heterogeneous computational and observational capabilities, aiming to minimize decision time and reduce target uncertainty. Our method utilizes multiple cameras as agents, controlling their rotation to track moving targets, such as pedestrians and vehicles at intersections. Summary of the Invention

[0007] The purpose of the present invention is to provide a multi-agent based collaborative control method for heterogeneous traffic cameras to solve the technical problems in the existing technology of limited and heterogeneous camera computing and observation capabilities.

[0008] In order to solve the above technical problems, the specific technical solutions of the present invention are as follows:

[0009] A multi-agent-based collaborative control method for heterogeneous traffic cameras consists of three main phases: target selection, participation control, and rotation control. Initially, each target is transformed to a new state according to a target motion model and its state is predicted by a Kalman filter. The agents then retrieve target information from these filters to guide their actions. Specifically, the method includes the following steps:

[0010] Step S1: In the target selection phase, each camera identifies the target within its viewing range and collects uncertainty data from the target. Each camera acts as an agent that combines its available computing resources with the target uncertainty data to generate an embedding vector.

[0011] Step S2: Participation control phase: The participation control module receives the embedding vector, extracts the coarse-grained features of the embedding vector through the multi-head self-attention block, and transmits the coarse-grained features to the participation action selection block; the participation action selection block decides whether the agent should participate in the task at the current time step to save computation time;

[0012] Step S3: In the rotation control stage, the rotation control module of the intelligent agent extracts fine-grained features through the dual self-attention block, which is used for the rotation action selection block decision to determine the optimal rotation action to reduce the target uncertainty.

[0013] Furthermore, in step S1, each camera acts as an agent, forming a multi-agent, multi-target environment. The goal of heterogeneous agents is to collaboratively cover targets; this target tracking problem can be naturally formulated as a decentralized partially observable Markov decision process, consisting of the tuple Indicates; among them, represents the state space, represents the agent action space, represents a local observation, represents the state transition probability, represents the joint reward function shared by the agents.

[0014] Furthermore, step S1 includes the following steps:

[0015] Step S11: Each camera observes the target within the line of sight through the intelligent observation model and collects uncertainty data from the target;

[0016] Step S12: Each agent uses a Kalman filter to predict and update the collected target uncertainty data;

[0017] Step S13: The agent combines its available computational resources with the target uncertainty data to generate an embedding vector.

[0018] Furthermore, step S2 includes the following steps:

[0019] Step S21: The participant control module receives the embedding vector, the multi-head self-attention block extracts coarse-grained features from the embedding vector, and passes the coarse-grained features to the participant action selection block;

[0020] Step S22: The participating action selection block receives the coarse-grained features, processes the coarse-grained features and decides whether the agent should participate in the task at the current time step;

[0021] Step S23: If the agent does not participate in the task at the current time step, the agent maintains its current direction and passes the newly observed target state directly to the agent observation model; if the agent participates in the task at the current time step, the agent enters the rotation control stage.

[0022] Furthermore, step S3 includes the following steps:

[0023] Step S31: The contribution estimation block of the rotation control module uses a long short-term memory network as its core to obtain an estimate of the contribution of each heterogeneous agent to the target within the line of sight, and ultimately obtains a target information vector;

[0024] Step S32: The related agent determination block identifies related agents based on the targets within the line of sight of each agent. If the same target is within the line of sight of multiple agents, these multiple agents are related agents and share observations.

[0025] Step S33: The dual self-attention block processes the target information vector from the contribution estimation block and the relevant agent observation from the relevant agent determination block, and uses the internal self-attention unit and the external self-attention unit to extract fine-grained features for the rotation action selection block decision;

[0026] Step S34: The rotation action selection block receives the fine-grained features transmitted by the dual self-attention block, processes the fine-grained features to obtain the optimal rotation angle, and then selects a specific rotation action;

[0027] Step S35: After executing the optimal rotation angle action obtained in step S34, each agent adjusts to the new direction and transmits the newly observed target state within the field of view to the agent observation model.

[0028] Compared with the prior art, the present invention has the following beneficial technical effects:

[0029] 1) This invention simultaneously addresses the problems of limited computing resources and target uncertainty. The environment supports access to a variety of mainstream algorithms, providing a flexible and diverse platform for subsequent research. A lightweight participant control mechanism designed in this paper dynamically selects the optimal subset of agents participating in a task, reducing decision time and alleviating the curse of dimensionality and resource constraints. This minimizes both decision time and target uncertainty, intelligently controlling the rotation of multiple traffic cameras, and tracking multiple moving targets.

[0030] 2) This invention achieves efficient decentralized decision-making with minimal information transfer. A fully parameter-shared rotation control mechanism designed in this invention utilizes a contribution estimation block based on a long short-term memory network to treat agent heterogeneity as a contribution estimation problem, eliminating the need for agents to rely on specific network layers and enabling zero-shot generalization in unknown scenarios without retraining. Unlike methods that extract features from all entities or limit themselves to a fixed number of targets, this invention integrates relevant agent determination blocks to focus on key observations, thereby improving the efficiency of feature extraction and action selection. This invention supports efficient training in small-scale scenarios and deployment in large-scale scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 Schematic diagram of the flow of the multi-agent-based heterogeneous traffic camera collaborative control method of the present invention. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0034] This paper proposes a multi-agent based collaborative control method for heterogeneous traffic cameras. Figure 1 As shown in Figure 3, the proposed method consists of three main phases: target selection, engagement control, and rotation control. Initially, each target is transformed to a new state according to the target motion model and its state is predicted by a Kalman filter. The agent then retrieves target information from these filters to guide its actions.

[0035] The present invention mainly makes decisions based on the multi-agent reinforcement learning algorithm. Each camera acts as an agent, forming a multi-agent, multi-target environment. The goal of heterogeneous agents is to collaboratively cover This target tracking problem can be naturally formulated as a decentralized partially observable Markov decision process, consisting of the tuple Indicates. Among them, represents the state space, represents the agent action space, represents a local observation, represents the state transition probability, represents the reward function shared by the agents.

[0036] is the state space of the environment at a given time step, It is the dimension of state. Representing an agent The action space of all agents is . Representing an agent local observations. Defines the current state To the next state The state transition probability of given joint action space The purpose of multi-agent reinforcement learning is to discover a set of optimal strategies, parameterization, , where the agent Use its strategy According to its local observation Determine its action .

[0037] The objective function is in a finite time range Maximize the discount cumulative reward:

[0038]

[0039] in, Representation Strategy The corresponding objective function is is the discount factor; Represents a state-action sequence Find the expectation of the probability distribution of ; Indicates time This aligns their individual goals with achieving the group goal.

[0040] The joint reward function aims to balance the performance of individual agents with the overall team goal. Indicates the assignment to the agent Rewards, is the individual reward of the agent and shared group rewards At each time step, the joint reward function (Right now ) is the average of all agent rewards and is shared among them, calculated as: .

[0041] The calculation formula is:

[0042]

[0043] in, It is a weighting factor that determines the balance of individual and team contributions to the reward.

[0044] Individual rewards The calculation formula is:

[0045]

[0046] in, Representing an agent The average certainty improvement of the contribution, It represents the proportion of unobserved targets, which is defined as the ratio of the number of unobserved targets within the viewing range to the total number of targets within the viewing range.

[0047] Team Rewards The calculation formula is:

[0048]

[0049] in, represents the average certainty improvement contributed by all agents, is the average time cost of all agents, including observation processing time and action selection time.

[0050] This invention also utilizes a multi-head self-attention mechanism to address the issue of varying input dimensions caused by changes in the number of agents and targets, and to extract features that significantly influence the agent's decision-making. Within the multi-head self-attention block, the self-attention mechanism has garnered significant attention due to its ability to learn to focus on the parts of the input most relevant to the output while handling variable-sized inputs in an order-invariant manner. The query and key are used to match relevance and derive attention scores between elements. These scores are then used to weight and aggregate the actual information provided by the values ​​to generate features.

[0051] The self-attention mechanism of the multi-head self-attention block can capture higher-order interactions by projecting the inner product into multiple subspaces. The outputs of all attention heads are concatenated and linearly transformed to produce the final output of the self-attention layer. The final output of the self-attention layer is calculated as follows:

[0052]

[0053]

[0054] in, Represents the final output of the self-attention layer; Represents a splicing operation; is the learnable weight matrix used in the output projection step; Indicates the The output of an attention head; Indicates the number of attention heads; Indicates the Query of 1 attention head; Indicates the The key of each attention head; Indicates the The value of an attention head; Indicates the Keys of attention heads Dimensions; Represents the Softmax function.

[0055] No. Query of attention heads ,key Sum By using the The query parameter matrix of the attention heads , key parameter matrix , value parameter matrix Embedding vector for the input The transformation is obtained as follows:

[0056]

[0057]

[0058]

[0059] The multi-head self-attention block enables each attention head to learn a different linear transformation of the input embedding vector, allowing the model to focus on different parts of the input data.

[0060] Specifically, the multi-agent-based heterogeneous traffic camera collaborative control method proposed in the present invention includes the following steps:

[0061] Step S1: In the target selection phase, each camera identifies the target within its viewing range (i.e., coverage radius) and collects uncertainty data from the target. Each camera acts as an agent that combines its available computing resources with the target uncertainty data to generate an embedding vector.

[0062] This invention introduces a heterogeneous design of agents in terms of computational power (available resources that vary over time) and observational capabilities (variation in field of view and observation noise), aiming to simultaneously reduce time costs and target uncertainty. Furthermore, by analyzing the rotational trajectories of the agents and the motion trajectories of the targets, the correlation between the agents and the targets can be effectively identified. Each agent can extract features not only from its selected target but also from other related agents, enabling more informed and strategic decision-making.

[0063] Each agent possesses various properties, including line-of-sight distance, viewing angle, and distance and angle measurement noise, which measure the agent's observation capabilities. Computational resources, expressed as available CPU frequency (GHz), account for the agent's data processing capabilities. Furthermore, the agent's movements are controlled by discrete angular velocities, defining its operational dynamics. These parameters enable the simulation of heterogeneous agents with computational, observational, and operational constraints, providing a powerful framework for collaboration and decision-making in multi-agent systems with constrained capabilities.

[0064] Step S11: Each camera observes the target within the line of sight through the intelligent observation model and collects uncertainty data from the target.

[0065] The agent observation model explains how the agent observes the position of each target relative to itself, taking into account the inherent uncertainty in the measurement process. The observation model is expressed as:

[0066]

[0067]

[0068] in, and Respectively represent the target The belief of the agent The polar diameter and polar angle of the polar coordinates are the target states observed by the agent observation model. Representing an agent The horizontal axis, Representing an agent The vertical coordinate of . Indicates the target The horizontal axis, Indicates the target The vertical coordinate of . Representing an agent The direction angle of and denote the distance and angle measurement noise respectively, and The mean is 0 and the variance is and Gaussian distribution, that is , .

[0069] The agent observation model observes the polar coordinate information of the target, taking into account the inherent uncertainty in the measurement process, so the polar coordinate information is uncertain data.

[0070] Step S12: Each agent uses a Kalman filter to predict and update the collected target uncertainty data.

[0071] Objects in the environment follow random motion models with Gaussian noise, resulting in inherent uncertainty. This, combined with the agent's heterogeneous observation capabilities, further increases measurement uncertainty. In partially observable object tracking tasks, the agent cannot obtain the true value of the target state. Each object is tracked using a separate Kalman filter, and each agent maintains a belief about the target state. This belief is updated via the Kalman filter as new observations are received. The prediction and update phases of each Kalman filter are as follows:

[0072] i) Prediction phase: The agent estimates the next state of each target using the target motion model, projecting the current belief forward.

[0073]

[0074]

[0075] in, Indicates time The measurement is based on the time The predicted target state. Represents the state transfer matrix, which converts time Status and time status is linked. Indicates time target state. Indicates time The measurement is based on the time The prediction covariance matrix of . Indicates time The covariance matrix of . express The transpose of . represents the process noise covariance matrix.

[0076] ii) Update phase: Use new observations to modify the predicted state. The updated state and covariance of is:

[0077]

[0078]

[0079]

[0080] in, Indicates time The predicted target state; Indicates time The Kalman gain matrix of , which weights the measurement residuals; Indicates time 's measurement innovation, i.e., the difference between actual and predicted measurements; Indicates time The updated covariance matrix of ; represents the identity matrix; represents the measurement matrix, which maps the state space to the measurement space; express The transpose of Represents the observation noise covariance matrix of the current agent.

[0081] Step S13: The agent combines its available computational resources with the target uncertainty data to generate an embedding vector.

[0082] Embedding vector By the agent The available computing resources and the uncertainty of the related goals are composed of the local observations of the agent, which are expressed as follows:

[0083]

[0084] in, Representing an agent Available computing resources, Indicates related targets uncertainty, It is calculated as follows:

[0085]

[0086] in, is the covariance matrix The logarithm of the determinant of . Related objectives The covariance of the belief It is expressed using the differential entropy formula. is the number of targets within the current agent's sight range, satisfying , is the total number of targets.

[0087] Step S2: In the participation control phase, the participation control module receives the embedding vector, extracts the coarse-grained features of the embedding vector through the multi-head self-attention block, and transmits the coarse-grained features to the participation action selection block; the participation action selection block decides whether the agent should participate in the task at the current time step to save computing time.

[0088] In traditional approaches, a central coordinator decides whether each agent should participate, while each agent independently determines its actions. However, the central coordinator requires information from all agents to make a decision, which can lead to the curse of dimensionality. Furthermore, if each agent independently decides its participation, decision-making time can become prohibitively long if the model parameters or input data are large.

[0089] To address the challenge of insufficient computing resources affecting action selection time, this paper uses a distributed, lightweight participation control module to determine whether an agent participates in a task at each time step. The participation control module processes less information but is more efficient. The participation control module consists of a multi-head self-attention block and a participating action selection block.

[0090] Step S21: The participant control module receives the embedding vector, the multi-head self-attention block extracts coarse-grained features from the embedding vector, and passes the coarse-grained features to the participant action selection block.

[0091] Step S22: The participating action selection block receives the coarse-grained features, processes the coarse-grained features and decides whether the agent should participate in the task at the current time step.

[0092] When the action selection block processes coarse-grained features, it is input into a linear fully connected layer to obtain the Q value of all actions. Furthermore, instead of simply selecting the one with the highest The action of the value (such as the greedy method), on the contrary, uses the LogSoftmax function to The values ​​are converted to a logarithmic probability distribution. Actions are then sampled from this distribution. This process smooths action selection and encourages exploration, allowing the agent to sometimes choose suboptimal actions rather than always the action with the highest Q value. The action space of the participating action selection block is discrete. It includes two types of actions: "participate" and "do not participate." The output is a binary value. A value of 1 indicates that the agent is participating in the task at the current time step, while a value of 0 indicates that it is not participating.

[0093] Step S23: If the agent is not participating in the task at the current time step, the agent maintains its current orientation and passes the newly observed target state directly to the agent observation model. If the agent is participating in the task at the current time step, the agent enters the rotation control phase.

[0094] Step S3: In the rotation control phase, the agent's rotation control module determines the optimal rotation action to reduce target uncertainty.

[0095] When all agents are homogeneous, implementing parameter-sharing models is simple and desirable, even when the task size increases. However, this approach becomes ineffective when dealing with agent heterogeneity. Conversely, using a separate model with heterogeneous parameters for each agent can lead to significant performance degradation when adapting to different tasks (such as when traffic cameras are deactivated or replaced due to failure, or when there is a significant increase in objects such as pedestrians and vehicles), requiring frequent retraining. Furthermore, as the number of agents and objects increases, the number of parameters also increases, introducing scalability challenges.

[0096] To address these issues, we designed a distributed and universal rotation control module that allows each agent to make independent decisions. Furthermore, all agents share the same model parameters within the rotation control module, eliminating the need for retraining and ensuring scalability even when moving to larger tasks or adapting to changes in agent configurations.

[0097] The rotation control module comprises a contribution estimation block, a related agent determination block, a dual self-attention block consisting of an internal self-attention unit and an external self-attention unit, and a rotation action selection block. The rotation control module facilitates efficient coordination among heterogeneous agents.

[0098] Step S31: The contribution estimation block of the rotation control module uses the long short-term memory network as the core to obtain the contribution estimate of each heterogeneous agent to the target within the line of sight, and finally obtain the target information vector.

[0099] The core of LSTM networks lies in memory cells and gating mechanisms. Memory cells retain information over time, while gating mechanisms regulate the flow of information. Contribution refers to the improvement in target certainty when an agent observes a target between two time steps.

[0100] The input vector of the contribution estimation block is composed of the current agent The attribute vector of the current target The input vector of the contribution estimation block is represented as:

[0101]

[0102] in, Representing an agent The attribute vector of Indicates the target The state vector of .

[0103] Agent The attribute vector is represented as ,in 、 、 and They represent the viewing distance, viewing angle, distance measurement noise and angle measurement noise respectively.

[0104] Target The state vector is represented as ;in, and Respectively represent the target The belief of the agent The polar diameter and polar angle of the polar coordinates; and They are and The derivative of Indicates related targets uncertainty; Representing an agent Whether the target is observed , if the agent Target observed ,but Set to 1 if yes, otherwise set to 0.

[0105] The input vector of the contribution estimation block is encoded by the long short-term memory network to estimate the contribution, obtaining the contribution estimate , Representing an agent Towards the target Contribution, i.e. goal The agent makes better action choices based on the contribution estimates.

[0106] After the output contribution of the LSTM network is estimated, the contribution estimation block converts the target The information vector is updated as:

[0107] .

[0108] Agent The target information vector of all observable targets is expressed as , Indicates the number of targets within the current agent's line of sight.

[0109] Step S32: The related agent determination block confirms the related agents through the targets within the sight range of each agent. If the same target is within the sight range of multiple agents, then these multiple agents are related agents and share observations.

[0110] The related agent determination block not only determines related agents through simple distance calculation, but also evaluates whether the agents are likely to cover the same target.

[0111] Specifically, the Related Agent Determination block evaluates each agent's relevance to all other agents. Agents are considered related if at least one target is within the line of sight of both the current agent and the agent being evaluated. If such a target is found, all target information collected by the evaluated agent is incorporated into the current agent's observations. Furthermore, agents marked as non-participating are skipped, reducing unnecessary time costs.

[0112] To illustrate, consider a scenario with three agents and three targets. Target 1 and Target 2 are both within the line of sight of Agent 1 and Agent 2. Therefore, Agent 1 and Agent 2 are considered related and can share observations. Similarly, Target 3 is within the line of sight of Agent 1 and Agent 3. As a result, Agent 1 and Agent 3 are also considered related, enabling them to share observations. If Agent 2 and Agent 3 are not directly related, they do not share observations. This approach effectively filters out irrelevant agents, optimizes computational efficiency, and focuses on important observations.

[0113] Step S33: The dual self-attention block processes the target information vector from the contribution estimation block and the relevant agent observation from the relevant agent determination block, and uses the internal self-attention unit and the external self-attention unit to extract fine-grained features for the rotation action selection block decision.

[0114] The dual self-attention block is designed to handle any number of targets and agents. Both the inner and outer self-attention units in the dual self-attention block are based on a multi-head self-attention mechanism. The inner self-attention unit processes the target information vector from the contribution estimation block, extracting key features from the target. The outer self-attention unit processes the relevant agent observations from the relevant agent determination block, extracting key features from the agents.

[0115] The target information vector of the internal self-attention unit is processed by the multi-head self-attention mechanism to generate the embedding feature of the current agent. Subsequently, the external self-attention unit aggregates the embedding features from all related agents and further refines these features to obtain fine-grained features to model the agent. Interactions with any number of other relevant agents. This approach effectively focuses on the most relevant features.

[0116] Each agent inputs its target information vector into the internal self-attention unit, and the target information vector serves as the embedding vector of the multi-head self-attention mechanism; The target information vector is embedded by extracting fine-grained features through the multi-head self-attention mechanism . Subsequently, all The relevant intelligent agents each extract fine-grained features from their own target information vectors to obtain corresponding embedded features, and then splice these embedded features together to obtain multi-dimensional embedded features. ,The multi-dimensional embedded features are used as the embedding vectors of the external self-attention unit, and the fine-grained features are further extracted through the multi-head self-attention mechanism. Is with the agent The number of related agents.

[0117] Step S34: The rotation action selection block receives the fine-grained features transmitted by the dual self-attention block, processes the fine-grained features to obtain the optimal rotation angle, and then selects a specific rotation action. The basic action space of the rotation action selection block is discrete, including rotation actions. rad / s, these actions can also be defined according to user needs. Specifically, the rotation action selection block processes the fine-grained features output by the dual self-attention block, that is, inputs the fine-grained features into a linear fully connected layer to obtain the Q value of all actions. The LogSoftmax function is further used to convert The values ​​are transformed into a log-probability distribution. Then actions are sampled from this distribution.

[0118] Step S35: After executing the optimal rotation angle action obtained in step S34, each agent adjusts to the new direction and transmits the newly observed target state within the field of view to the agent observation model.

[0119] Step S36: Using the Kalman filter described in step S12, the observation model is used to process the uncertainty data collected from the target to correct its prediction and reduce the overall target uncertainty, and the processing continues to the next time step.

[0120] This paper addresses the numerous challenges of optimizing target tracking by adjusting the orientation of distributed cameras, particularly when considering computational and observational capabilities. To address these challenges, a control method based on heterogeneous multi-agent reinforcement learning was developed specifically for heterogeneous extreme edge devices involved in multi-target tracking. Through a lightweight participatory control mechanism and an efficient rotational control mechanism, decision efficiency is improved and target uncertainty is reduced, eliminating the need for retraining in dynamic environments and achieving zero-shot generalization.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-agent based heterogeneous traffic camera collaborative control method, characterized in that: The proposed method consists of three main stages: target selection stage, engagement control stage, and rotation control stage. Initially, each target is transformed to a new state according to the target motion model and its state is predicted by the Kalman filter. The agent then retrieves target information from these filters to guide its actions. Specifically, the method includes the following steps: Step S1: In the target selection phase, each camera identifies the target within its viewing range and collects uncertainty data from the target. Each camera acts as an agent, which combines its available computing resources with the target uncertainty data to generate an embedding vector. The target uncertainty data is the polar coordinate information of the target observed by the agent. Step S2: Participation control phase: The participation control module receives the embedding vector, extracts the coarse-grained features of the embedding vector through the multi-head self-attention block, and transmits the coarse-grained features to the participation action selection block; the participation action selection block decides whether the agent should participate in the task at the current time step to save computation time; Step S3: In the rotation control phase, the agent's rotation control module extracts fine-grained features through the dual self-attention block, which is used in the rotation action selection block to determine the optimal rotation action to reduce target uncertainty. Step S3 includes the following steps: Step S31: The contribution estimation block of the rotation control module uses a long short-term memory network as its core to obtain an estimate of the contribution of each heterogeneous agent to the target within the line of sight, and ultimately obtains a target information vector; Step S32: The related agent determination block identifies related agents based on the targets within the line of sight of each agent. If the same target is within the line of sight of multiple agents, these multiple agents are related agents and share observations. Step S33: The dual self-attention block processes the target information vector from the contribution estimation block and the relevant agent observation from the relevant agent determination block, and uses the internal self-attention unit and the external self-attention unit to extract fine-grained features for the rotation action selection block decision; Step S34: The rotation action selection block receives the fine-grained features transmitted by the dual self-attention block, processes the fine-grained features to obtain the optimal rotation angle, and then selects a specific rotation action; Step S35: After executing the optimal rotation angle action obtained in step S34, each agent adjusts to the new direction and transmits the newly observed target state within the field of view to the agent observation model.

2. The multi-agent-based heterogeneous traffic camera collaborative control method according to claim 1 is characterized in that: Step S1 includes the following steps: Step S11: Each camera observes the target within the line of sight through the intelligent observation model and collects uncertainty data from the target; Step S12: Each agent uses a Kalman filter to predict and update the collected target uncertainty data; Step S13: The agent combines its available computational resources with the target uncertainty data to generate an embedding vector.

3. The multi-agent-based heterogeneous traffic camera collaborative control method according to claim 2 is characterized in that: Step S2 includes the following steps: Step S21: The participant control module receives the embedding vector, the multi-head self-attention block extracts coarse-grained features from the embedding vector, and passes the coarse-grained features to the participant action selection block; Step S22: The participating action selection block receives the coarse-grained features, processes the coarse-grained features and decides whether the agent should participate in the task at the current time step; Step S23: If the agent does not participate in the task at the current time step, the agent maintains its current direction and passes the newly observed target state directly to the agent observation model; if the agent participates in the task at the current time step, the agent enters the rotation control stage.

Images