Multi-agent-based heterogeneous traffic camera cooperative control method

Abstract

The invention provides a multi-agent-based heterogeneous traffic camera cooperative control method, and belongs to the technical field of reinforcement learning and edge computing. The method comprises the following steps: in a target selection stage, an agent combines available computing resources with target uncertainty data to generate an embedded vector; in the control participation stage, the control participation module receives the embedded vector, extracts the coarse-grained characteristics of the embedded vector through the multi-head self-attention block, and transmits the coarse-grained characteristics to the action participation selection block; the action participation selection block determines whether the agent should participate in the task at the current time step so as to save the calculation time; in the rotation control stage, a rotation control module of the intelligent agent extracts fine-grained features through double self-attention blocks, and the fine-grained features are used for decision making of a rotation action selection block, and the optimal rotation action is determined to reduce the target uncertainty. According to the invention, decision time and target uncertainty are minimized at the same time, rotation of a plurality of traffic cameras is intelligently controlled, and a plurality of moving targets are tracked.

Description

Technical Field

[0001] The present invention belongs to the technical fields of reinforcement learning and edge computing, and particularly relates to a multi-agent heterogeneous traffic camera collaborative control method. Background Art

[0002] Traditional visual object tracking tasks mainly estimate the bounding boxes of objects of interest in a series of picture frames. In addition, multi-object tracking tasks aim to address the challenge of tracking multiple objects simultaneously within the field of view. However, when an object is occluded or moves out of the field of view, the tracking accuracy will decrease significantly. To alleviate this problem, active object tracking methods have been proposed, which dynamically adjust the camera according to the visual scene to ensure that the object is always within the field of view. Active object tracking has been widely applied in scenarios such as unmanned aerial vehicles, intelligent robots, and sports events. The core of the object tracking problem lies in using a limited number of cameras and robots as agents to maximize the number of observed or covered objects, or to minimize the uncertainty of the objects.

[0003] The decentralized coverage planner D2COPLAN generates a compact representation through a predicted map and shares it among multiple robots through a graph neural network with parameter sharing to achieve coverage. However, this method relies on data with real actions as training labels, and these labels are usually difficult to obtain or require a large amount of time. In recent years, the progress of cooperative multi-agent reinforcement learning has significantly improved the efficiency of solving such cooperative problems and can achieve good results without explicit labels. For example, the best camera action is confirmed by pruning the search tree through Monte Carlo tree search, with the goal of achieving continuous object tracking under real-time camera control and maximizing object coverage. This method increases flexibility by allowing the camera to rotate and move along a predefined path. However, its fully centralized approach is limited by the curse of dimensionality, and the distributed method can alleviate this problem.

[0004] In a distributed system, each agent extracts features to select the original action. For example, in DARL1N, each agent makes an independent decision by interacting with its single-hop neighbors. When the number of agents and objects increases, the curse of dimensionality problem becomes more prominent. EPC addresses the scalability challenge of multi-agent and multi-object scenarios through a curriculum learning paradigm. In an actual scenario, an agent estimates the object state from noisy observations through sensors such as radar, lidar, or cameras, and these observation data are affected by measurement errors, thus causing observation uncertainty. PUARL solves the observation uncertainty problem in object tracking through exploration guidance based on probability estimation and policy adjustment based on uncertainty awareness. However, these methods fail to address the challenge of observation heterogeneity. Differences in field of view parameters (such as line of sight and viewing angle), as well as measurement noise (including distance and angle noise), will affect the uncertainty of object state estimation.

[0005] In addition, resource heterogeneity leads to differences in the decision-making times of different agents because the resource usage of devices changes due to the dynamic fluctuations of other running services. Extreme edge computing further expands the application of edge computing by directly implementing real-time data processing on resource-constrained terminal devices (i.e., extreme edge devices). This paradigm creates new possibilities for industries, businesses, and individuals to utilize idle computing resources. However, since users can access their devices at any time and run intensive services, this results in highly dynamic resource usage. This volatility in resource usage poses a significant challenge to the real-time coordination of large-scale multi-agent systems in dynamic extreme edge computing environments.

[0006] Therefore, in large-scale heterogeneous agent scenarios, it is crucial to adopt effective control strategies. Enhancing collaboration among agents is also a key challenge in such environments. To address these challenges, we propose a zero-shot generalization heterogeneous multi-agent reinforcement learning method that simultaneously considers heterogeneous computing capabilities and observation capabilities, aiming to minimize decision-making time and reduce target uncertainty. Our method uses multiple cameras as agents and controls their rotation to track moving targets, such as pedestrians and vehicles at intersections. Summary of the Invention

[0007] The object of the present invention is to provide a heterogeneous traffic camera collaborative control method based on multi-agents to solve the technical problems of limited and heterogeneous computing capabilities and observation capabilities of cameras in the prior art.

[0008] To solve the above technical problems, the specific technical solution of the present invention is as follows:

[0009] A heterogeneous traffic camera collaborative control method based on multi-agents, the method includes three main stages: a target selection stage, a participation control stage, and a rotation control stage; initially, each target is transformed into a new state according to the target motion model, and its state is predicted by a Kalman filter; then, the agent retrieves target information from these filters to guide its actions. Specifically, the method includes the following steps:

[0010] Step S1: In the target selection stage, each camera confirms the targets within its line of sight and collects uncertainty data from the targets. Each camera acts as an agent, and the agent combines its available computing resources with the target uncertainty data to generate an embedding vector;

[0011] Step S2: In the participation control stage, the participation control module receives the embedding vector, extracts the coarse-grained features of the embedding vector through a multi-head self-attention block, and transmits the coarse-grained features to the participation action selection block; the participation action selection block decides whether the agent should participate in the task at the current time step to save computing time;

[0012] Step S3: Rotation control phase. The rotation control module of the agent extracts fine-grained features through a dual self-attention block for the rotation action selection block to make a decision and determine the optimal rotation action to reduce target uncertainty.

[0013] Further, in step S1, each camera serves as an agent, forming a multi-agent and multi-target environment. A group of heterogeneous agents aim to collaboratively cover targets; this target tracking problem can be naturally formulated as a decentralized partially observable Markov decision process, represented by the tuple ; where, represents the state space, represents the agent action space, represents the local observation, represents the state transition probability, represents the joint reward function shared by the agents.

[0014] Further, step S1 includes the following steps:

[0015] Step S11: Each camera observes the targets within the line of sight through the agent observation model and collects uncertainty data from the targets;

[0016] Step S12: Each agent uses a Kalman filter to predict and update the collected target uncertainty data;

[0017] Step S13: The agent combines its available computing resources with the target uncertainty data to generate an embedding vector.

[0018] Further, step S2 includes the following steps:

[0019] Step S21: The participation control module receives the embedding vector. The multi-head self-attention block extracts coarse-grained features from the embedding vector and passes the coarse-grained features to the participation action selection block;

[0020] Step S22: The participation action selection block receives the coarse-grained features, processes the coarse-grained features and decides whether the agent should participate in the task at the current time step;

[0021] Step S23: If the agent does not participate in the task at the current time step, the agent maintains its current direction and directly passes the newly observed target state to the agent observation model; if the agent participates in the task at the current time step, the agent enters the rotation control phase.

[0022] Further, step S3 includes the following steps:

[0023] Step S31: The contribution estimation block of the rotation control module uses a long short-term memory network as the core to obtain the contribution estimation of each heterogeneous agent to the target within the line of sight range, and finally obtains the target information vector;

[0024] Step S32: The relevant agent determination block determines relevant agents through the targets within the line of sight of each agent. If the same target is within the line of sight of multiple agents, these multiple agents are relevant agents and share observations;

[0025] Step S33: The dual self-attention block processes the target information vector from the contribution estimation block and the relevant agent observations from the relevant agent determination block, and uses the internal self-attention unit and the external self-attention unit to extract fine-grained features for the rotation action selection block to make decisions;

[0026] Step S34: The rotation action selection block receives the fine-grained features transmitted by the dual self-attention block, processes the fine-grained features to obtain the optimal rotation angle, and then selects a specific rotation action;

[0027] After performing the optimal rotation angle action obtained in Step S34, each agent adjusts to a new direction and transmits the newly observed target state within the field of view to the agent observation model.

[0028] Compared with the prior art, the present invention has the following beneficial technical effects:

[0029] 1) The present invention solves the problems of limited computing resources and target uncertainty at the same time. This environment supports the access of a variety of mainstream algorithms, providing a flexible and diverse platform for subsequent research; a lightweight participation control mechanism designed by the present invention can dynamically select the optimal subset of agents participating in the task, thereby reducing the decision-making time and alleviating the curse of dimensionality and resource limitation problems. At the same time, minimizing the decision-making time and target uncertainty, intelligently controlling the rotation of multiple traffic cameras to track multiple moving targets.

[0030] 2) The present invention realizes efficient decentralized decision-making and only requires less information transmission; a rotation control mechanism with completely shared parameters designed by the present invention uses a contribution estimation block based on a long short-term memory network to handle agent heterogeneity as a contribution estimation problem, thereby eliminating the need for agents for specific network layers and achieving zero-shot generalization in unknown scenarios without retraining. Different from the method of extracting features from all entities or being limited to a fixed number of targets, the present invention integrates a relevant agent determination block, focuses on key observations, and thus improves the efficiency of feature extraction and action selection. The present invention supports efficient training on a small-scale scenario and deployment to a large-scale scenario. Brief Description of the Drawings

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0032] Figure 1 It is a schematic flowchart of the heterogeneous traffic camera cooperative control method based on multi-agent of the present invention. Detailed implementation manners

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0034] The present invention proposes a heterogeneous traffic camera cooperative control method based on multi-agent, as Figure 1 shown. The method includes three main stages: the target selection stage, the participation control stage, and the rotation control stage. Initially, each target is transformed into a new state according to the target motion model, and its state is predicted by the Kalman filter. Then, the agent retrieves the target information from these filters to guide its actions.

[0035] The present invention mainly makes decisions based on the multi-agent reinforcement learning algorithm. Each camera acts as an agent, forming a multi-agent, multi-target environment. A group of heterogeneous agents aim to collaboratively cover targets. This target tracking problem can be naturally formulated as a decentralized partially observable Markov decision process, represented by the tuple . Among them, represents the state space, represents the agent action space, represents the local observation, represents the state transition probability, represents the reward function shared by the agents.

[0036] is the state space of the environment at a given time step, is the dimension of the state. represents the action space of agent , and the joint action space of all agents is . represents agent Local observations. Defines the state transition probability from the current state to the next state given the joint action space . The intention of multi-agent reinforcement learning is to discover a set of optimal policies, parameterized by , where agent uses its policy to decide its action based on its local observations . .

[0037] The objective function is to maximize the discounted cumulative reward over a finite time horizon :

[0038]

[0039] where, denotes the objective function corresponding to policy , is the discount factor; denotes taking the expectation over the probability distribution of the state-action sequence ; denotes time and

[0040] is the joint reward function. This aligns their individual goals with the achievement of the group goal. denotes the reward assigned to agent , is the individual reward of the agent and the shared group reward weighted combination. At each time step, the joint reward function (i.e., ) is the average of all agent rewards and shared among them, calculated as: .

[0041] is calculated as:

[0042]

[0043] where, is the weight factor that determines the balance of the contributions of the individual and the team to the reward.

[0044] The individual reward is calculated as:

[0045]

[0046] Among them, represents the average increase in certainty contributed by the agent and represents the proportion of unobserved targets, defined as the ratio of the number of unobserved targets within the line of sight to the total number of targets within the line of sight.

[0047] Team reward is calculated as follows:

[0048]

[0049] Among them, represents the average increase in certainty contributed by all agents, is the average time cost of all agents, including observation processing time and action selection time.

[0050] In the present invention, a multi-head self-attention mechanism is also used to solve the problem of input dimension change caused by the change in the number of agents and targets, and can extract features that have a significant impact on the decision-making of agents. In the multi-head self-attention block, the self-attention mechanism has received extensive attention because it can learn to focus on the part of the input that is most relevant to the output and process inputs of variable sizes in a sequence-invariant manner. Among them, queries and keys are used to match correlations and obtain attention scores between elements, and then the actual information provided by the values is weighted and aggregated based on these scores to obtain features.

[0051] The self-attention mechanism of the multi-head self-attention block can capture higher-order interactions by projecting the inner product into multiple subspaces. The outputs of all attention heads are concatenated and linearly transformed to produce the final output of the self-attention layer, and the final output of the self-attention layer is calculated as follows:

[0052]

[0053]

[0054] Among them, represents the final output of the self-attention layer; represents the concatenation operation; is the learnable weight matrix used in the output projection step; represents the th output of the attention head; represents the number of attention heads; represents the th query of the attention head; represents the th key of the attention head; represents the th value of the attention head; represents the The key of one attention head The dimension of; Denote the Softmax function.

[0055] The The query of the -th attention head and key and value are obtained by transforming the input embedding vector using the query parameter matrix of the -th attention head the key parameter matrix

[0056]

[0057]

[0058]

[0059] The multi-head self-attention block enables each attention head to learn different linear transformations of the input embedding vector, so that the model can focus on different parts of the input data.

[0060] Specifically, the multi-agent heterogeneous traffic camera cooperative control method proposed by the present invention includes the following steps:

[0061] Step S1: In the target selection stage, each camera confirms the targets within its line of sight (i.e., coverage radius), and collects uncertainty data from the targets. Each camera acts as an agent, and the agent combines its available computing resources with the target uncertainty data to generate an embedding vector.

[0062] The present invention introduces the heterogeneous design of agents in terms of computing power (available resources that change over time) and observation ability (differences in field of view and observation noise), with the intention of reducing both the time cost and the uncertainty of the target. In addition, by analyzing the rotation trajectory of the agent and the movement trajectory of the target, the correlation between the agent and the target can be effectively confirmed. Each agent can not only extract features from its selected targets, but also extract features from other relevant agents, thus enabling more intelligent and strategic decision-making.

[0063] Each agent has diverse attributes, including line of sight, field of view, and distance and angle measurement noise, etc., which are used to measure the agent's observation ability. The computing resources are represented by the available CPU frequency (GHz) and are used for the agent's data processing ability. In addition, the agent's actions are controlled by discrete angular velocities, thus defining its operation dynamics. These parameters can simulate heterogeneous agents with different computing, observation, and operation constraints, providing a powerful framework for cooperation and decision-making in multi-agent systems with limited capabilities.

[0064] Step S11: Each camera observes the targets within the line of sight through the agent observation model and collects uncertainty data from the targets.

[0065] The agent observation model explains how the agent observes the position of each target relative to itself, taking into account the inherent uncertainty in the measurement process. The observation model of the agent is expressed as:

[0066]

[0067]

[0068] where and represent the radial distance and polar angle of the belief of target relative to the polar coordinates of agent , which are the target states observed by the agent observation model. represents the abscissa of agent , represents the ordinate of agent . represents the abscissa of target , represents the ordinate of target . represents the direction angle of agent ; and represent the distance and angle measurement noise respectively, and follow Gaussian distributions with a mean of 0 and variances of and respectively, that is , .

[0069] The polar coordinate information of the target observed by the agent observation model takes into account the inherent uncertainty in the measurement process, so the polar coordinate information is uncertainty data.

[0070] Step S12: Each agent uses the Kalman filter to predict and update the collected target uncertainty data.

[0071] Targets in the environment follow a random motion model with Gaussian noise, resulting in inherent uncertainty. Combining the heterogeneous observation capabilities of the agents further increases the measurement uncertainty. In a partially observable target tracking task, the agents cannot obtain the true values of the target states. Each target is tracked by an independent Kalman filter, and each agent maintains a belief based on the target state. This belief is updated through the Kalman filter as new observations are received. The prediction and update phases of each Kalman filter are as follows:

[0072] i) Prediction phase: The agent uses the target motion model to estimate the next state of each target, projecting the current belief forward.

[0073]

[0074]

[0075] where, represents the predicted target state at time based on the measurement at time . represents the state transition matrix, which relates the state at time to the state at time . represents the target state at time . represents the predicted covariance matrix at time based on the measurement at time . represents the covariance matrix at time . represents the transpose. represents the process noise covariance matrix.

[0076] ii) Update phase: The predicted state is corrected using the new observation. The updated state and covariance at time are:

[0077]

[0078]

[0079]

[0080] where, represents the predicted target state at time ; represents the The Kalman gain matrix weights the measurement residuals; represents the measurement innovation at time i.e., the difference between the actual measurement and the predicted measurement; represents the updated covariance matrix at time ; represents the identity matrix; represents the measurement matrix that maps the state space to the measurement space; represents the transpose of; represents the observation noise covariance matrix of the current agent.

[0081] Step S13: The agent combines its available computing resources with the target uncertainty data to generate an embedding vector.

[0082] The embedding vector is composed of the available computing resources of the agent and the uncertainty of the relevant target, representing the local observation of the agent, as follows:

[0083]

[0084] where represents the available computing resources of the agent ; represents the uncertainty of the relevant target ; is calculated as follows:

[0085]

[0086] where is the logarithm of the determinant of the covariance matrix . The covariance of the belief of the relevant target is expressed using the differential entropy formula. ; is the number of targets within the line of sight of the current agent, satisfying ; is the total number of targets.

[0087] Step S2: Participate in the control phase. The participation control module receives the embedding vector, extracts the coarse-grained features of the embedding vector through the multi-head self-attention block, and transmits the coarse-grained features to the participation action selection block; the participation action selection block decides whether the agent should participate in the task at the current time step to save computing time.

[0088] In traditional methods, the central coordinator can decide whether each agent participates, while each agent independently determines the actions it takes. However, the central coordinator needs information from all agents to make decisions, which may lead to the curse of dimensionality. Additionally, if each agent independently decides its participation, the decision-making time may become too long if the model parameters or input data are large.

[0089] To address the challenge that insufficient computing resources affect the action selection time, the present invention adopts a distributed lightweight participation control module to determine whether an agent participates in a task at each time step. The participation control module processes less but more effective information. The participation control module includes a multi-head self-attention block and a participation action selection block.

[0090] Step S21: The participation control module receives the embedding vector. The multi-head self-attention block extracts coarse-grained features from the embedding vector and passes the coarse-grained features to the participation action selection block.

[0091] Step S22: The participation action selection block receives the coarse-grained features, processes the coarse-grained features and decides whether the agent should participate in the task at the current time step.

[0092] When the participation action selection block processes the coarse-grained features, it inputs them into a linear fully connected layer to obtain the Q-values of all actions. Further, instead of simply selecting the action with the highest value (such as the greedy method), on the contrary, the LogSoftmax function is used to convert the value into a logarithmic probability distribution. Then an action is sampled from this distribution. This process makes the action selection smoother and encourages exploration, enabling the agent to sometimes select sub-optimal actions instead of always selecting the action with the highest Q-value. The action space of the participation action selection block is discrete. It includes two actions: "participate" and "not participate". The output is a binary value. A value of 1 indicates that the agent participates in the task at the current time step, while a value of 0 indicates non-participation.

[0093] Step S23: If the agent does not participate in the task at the current time step, the agent maintains its current direction and directly passes the newly observed target state to the agent observation model. If the agent participates in the task at the current time step, then the agent enters the rotation control stage.

[0094] Step S3: In the rotation control stage, the rotation control module of the agent determines the best rotation action to reduce the target uncertainty.

[0095] When all agents are homogeneous, even when the task scale increases, the method of implementing a parameter-sharing model is simply preferable. However, when dealing with agent heterogeneity, this method becomes ineffective. Instead, using independent models with heterogeneous parameters for each agent may lead to a significant drop in performance when adapting to different tasks (such as traffic cameras being deactivated due to faults, replaced, or a significant increase in targets such as pedestrians and vehicles), requiring frequent retraining. In addition, as the number of agents and targets increases, the number of parameters also increases, presenting challenges in terms of scalability.

[0096] To address these issues, the present invention designs a distributed and general rotation control module that allows each agent to make decisions independently. In addition, all agents share the same model parameters within the rotation control module, eliminating the need for retraining and ensuring scalability even when switching to larger-scale tasks or adapting to changes in agent configurations.

[0097] The rotation control module includes a contribution estimation block, a relevant agent determination block, a dual self-attention block, and a rotation action selection block. The dual self-attention block includes an internal self-attention unit and an external self-attention unit. The rotation control module promotes efficient coordination among heterogeneous agents.

[0098] Step S31: The contribution estimation block of the rotation control module uses a long short-term memory network as the core to obtain the contribution estimation of each heterogeneous agent to the targets within the line of sight, and finally obtains the target information vector.

[0099] The core of the long short-term memory network lies in the memory unit and the gating mechanism. The memory unit can retain information over time, and the gating mechanism regulates the flow of information. Contribution refers to the amount of increase in the certainty of a target when an agent observes the target between two time steps.

[0100] The input vector of the contribution estimation block is composed of the attribute vector of the current agent and the state vector of the current target , and the input vector of the contribution estimation block is expressed as:

[0101]

[0102] where, represents the attribute vector of agent , represents the state vector of target .

[0103] The attribute vector of agent is expressed as , where , , and respectively represent the viewing distance, viewing angle, distance measurement noise, and angle measurement noise.

[0104] Target The state vector of is represented as ; where and respectively represent the polar radius and polar angle of the target 's belief with respect to the polar coordinates of the agent ; and are respectively the derivatives of and ; represents the uncertainty of the relevant target ; represents whether the agent observes the target , if the agent observes the target , then is set to 1, otherwise it is set to 0.

[0105] The input vector of the contribution estimation block is encoded by a long short-term memory network to estimate the contribution, obtaining the contribution estimate , represents the contribution of the agent to the target , that is, the reduction in the uncertainty of the target . The agent makes better action choices based on the contribution estimate.

[0106] After the output contribution estimate of the long short-term memory network, the contribution estimation block updates the information vector of the target to:

[0107] .

[0108] The target information vectors of all targets that the agent can observe are represented as , represents the number of targets within the viewing distance of the current agent.

[0109] Step S32: The relevant agent determination block determines the relevant agents through the targets within the viewing distance of each agent. If the same target is within the viewing distance of multiple agents, then these multiple agents are relevant agents and share observations.

[0110] The relevant agent determination block not only determines the relevant agents through simple distance calculations, but also evaluates whether the agents are likely to cover the same target.

[0111] Specifically, the relevant agent determination block evaluates the relevance of each agent to all other agents. If at least one target is within the line of sight of the current agent and the evaluated agent, these agents are considered relevant. If such a target is found, all the target information collected by the evaluated agent will be incorporated into the observations of the current agent. Additionally, agents marked as not participating will be skipped, reducing unnecessary time costs.

[0112] For example: Consider a scenario with three agents and three targets. Target 1 and Target 2 are both within the line of sight of Agent 1 and Agent 2. Therefore, Agent 1 and Agent 2 are considered relevant and can share observations. Similarly, Target 3 is within the line of sight of Agent 1 and Agent 3. As a result, Agent 1 and Agent 3 are also considered relevant, enabling them to share observations. If Agent 2 and Agent 3 are not directly relevant, they do not share observations. This method effectively filters out irrelevant agents, optimizes computational efficiency, and focuses on observations of importance.

[0113] Step S33: The dual self-attention block processes the target information vector from the contribution estimation block and the relevant agent observations from the relevant agent determination block, and uses the internal self-attention unit and the external self-attention unit to extract fine-grained features for the rotation action selection block to make decisions.

[0114] The dual self-attention block is used to process any number of targets and agents. Both the internal self-attention unit and the external self-attention unit in the dual self-attention block are designed based on the multi-head self-attention mechanism. The internal self-attention unit processes the target information vector of the contribution estimation block to extract key features from the targets. The external self-attention unit processes the relevant agent observations of the relevant agent determination block to extract key features from the agents.

[0115] The target information vector of the internal self-attention unit is processed by the multi-head self-attention mechanism to generate the embedding features of the current agent. Subsequently, the external self-attention unit aggregates the embedding features from all relevant agents and further refines these features to obtain fine-grained features for modeling the interaction between the agent and any number of other relevant agents. This method effectively focuses on the most relevant features.

[0116] Each agent inputs its target information vector into the internal self-attention unit, and the target information vector serves as the embedding vector of the multi-head self-attention mechanism; the target information vector of the agent extracts fine-grained features through the multi-head self-attention mechanism to obtain embedding features . Subsequently, all agents The relevant agents respectively perform fine-grained feature extraction on their own target information vectors to obtain corresponding embedded features, and splice these embedded features together to obtain multi-dimensional embedded features. The multi-dimensional embedded features serve as the embedded vectors of the external self-attention unit, and further extract fine-grained features through the multi-head self-attention mechanism. is the number of agents related to the agent.

[0117] Step S34: The rotation action selection block receives the fine-grained features transmitted by the dual self-attention block, processes the fine-grained features to obtain the optimal rotation angle, and then selects a specific rotation action. The basic action space of the rotation action selection block is discrete, including rotation actions rad / s, and these actions can also be defined according to user requirements. Specifically, the rotation action selection block processes the fine-grained features output by the dual self-attention block, that is, inputs the fine-grained features into a linear fully connected layer to obtain the Q values of all actions. Further use the LogSoftmax function to convert the values into a logarithmic probability distribution. Then sample actions from this distribution.

[0118] After performing the optimal rotation angle action obtained in step S34, each agent adjusts to a new direction and transmits the newly observed target states within the field of view to the agent observation model.

[0119] Step S36: Use the Kalman filter described in step S12 to process the uncertainty data collected from the target using the observation model to correct its prediction and reduce the overall target uncertainty, and continue to process the next time step.

[0120] The main work of the present invention is to solve the many challenges faced when adjusting the orientation of distributed cameras to optimize target tracking, especially considering computing power and observation ability. To address these challenges, a control method based on heterogeneous multi-agent reinforcement learning is developed specifically for heterogeneous extreme edge devices in multi-target tracking. Through a lightweight participation control mechanism and an efficient rotation control mechanism, the decision-making efficiency is improved and the target uncertainty is reduced, without the need for retraining in a dynamic environment, achieving zero-shot generalization ability.

[0121] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it is still possible to modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-agent-based collaborative control method for heterogeneous traffic cameras, characterized in that The method includes three main stages: a target selection stage, a participation control stage, and a rotation control stage; initially, each target is transformed to a new state according to the target motion model, and its state is predicted by a Kalman filter; Then, the agent retrieves target information from these filters to guide its actions. Specifically, the method includes the following steps: Step S1: In the target selection stage, each camera confirms the targets within its line of sight and collects uncertainty data from the targets. Each camera acts as an agent, and the agent combines its available computing resources with the target uncertainty data to generate an embedding vector; Step S2: In the participation control stage, the participation control module receives the embedding vector, extracts the coarse-grained features of the embedding vector through a multi-head self-attention block, and transmits the coarse-grained features to the participation action selection block; the participation action selection block decides whether the agent should participate in the task at the current time step to save computing time; Step S3: In the rotation control stage, the rotation control module of the agent extracts fine-grained features through a double self-attention block for the rotation action selection block to make a decision and determine the best rotation action to reduce target uncertainty.

2. The multi-agent-based heterogeneous traffic camera collaborative control method according to claim 1, wherein Step S1 includes the following steps: Step S11: Each camera observes the targets within its line of sight through the agent observation model and collects uncertainty data from the targets; Step S12: Each agent predicts and updates the collected target uncertainty data using a Kalman filter; Step S13: The agent combines its available computing resources with the target uncertainty data to generate an embedding vector.

3. The heterogeneous traffic camera cooperative control method based on multi-agent according to claim 2, wherein Step S2 includes the following steps: Step S21: The participation control module receives the embedding vector, and the multi-head self-attention block extracts the coarse-grained features from the embedding vector and transmits the coarse-grained features to the participation action selection block; Step S22: The participation action selection block receives the coarse-grained features, processes the coarse-grained features, and decides whether the agent should participate in the task at the current time step; Step S23: If the agent does not participate in the task at the current time step, the agent maintains its current direction and directly transmits the newly observed target state to the agent observation model; if the agent participates in the task at the current time step, the agent enters the rotation control stage.

4. The multi-agent based heterogeneous traffic camera collaborative control method according to claim 3, characterized in that, Step S3 includes the following steps: Step S31: The contribution estimation block of the rotation control module uses a long short-term memory network as the core to obtain the contribution estimation of each heterogeneous agent to the targets within its line of sight, and finally obtains a target information vector; Step S32: The relevant agent determination block determines the relevant agents through the targets within the line of sight of each agent. If the same target is within the line of sight of multiple agents, these multiple agents are relevant agents and share observations; Step S33: The double self-attention block processes the target information vector from the contribution estimation block and the observations of the relevant agents from the relevant agent determination block, and uses an internal self-attention unit and an external self-attention unit to extract fine-grained features for the rotation action selection block to make a decision; Step S34: The rotation action selection block receives the fine-grained features transmitted by the double self-attention block, processes the fine-grained features to obtain the best rotation angle, and then selects a specific rotation action; Step S35: After performing the optimal rotation angle action obtained in step S34, each agent adjusts to a new direction and transmits the newly observed target states within the field of view to the agent observation model.

Images