Collaborative Target Tracking Method, Device, and Equipment Based on Adaptive Federated Learning

Abstract

The present application relates to a collaborative target tracking method, device and equipment based on adaptive federated learning in the field of visual monitoring technology. The method introduces federated learning into this field, determines follower nodes according to edge devices, selects candidate nodes participating in training from the follower nodes by using an adaptive candidate node selection method based on a reputation mechanism, selects aggregation nodes from the candidate nodes, then distributes the model to the candidate nodes for local partial training, transmits the updated parameters of each candidate node to the aggregation node for aggregation, and after several tenure trainings, obtains a final collaborative target tracking model, and uses the final collaborative target tracking model to track the target to be tracked in the target area to obtain a collaborative target tracking result. Compared with the most advanced existing methods, the present method has comparable or even better accuracy, while requiring less data transmission cost and less time consumption.

Description

Technical Field

[0001] The present application relates to the technical field of visual monitoring, and particularly to a collaborative target tracking method, device, and equipment based on adaptive federated learning. Background Art

[0002] Tracking moving targets is a basic task in many application fields, such as wildlife protection, area surveillance, battlefield reconnaissance, disaster relief, etc. With the rapid development of sensors, networks, and electronic devices in the Internet of Things, the tracking of moving targets mainly involves deploying multiple edge devices to collect images or videos of the target area from different angles and perspectives. The main tracking technology is based on neural networks such as deep learning, which must use a large amount of data to train the model. However, these edge devices may be reluctant to share their local data due to privacy concerns. Therefore, there is an urgent need for a secure and effective collaborative target tracking solution.

[0003] Currently, there are two mainstream collaborative target tracking technologies, including the centralized strategy and the distributed strategy. The centralized strategy uses data from other devices to train the model on a single node. The central node can independently select the data source and transmission path, saving bandwidth and energy. However, this strategy still has the risk of central node failure and a relatively high transmission cost. The distributed strategy, on the other hand, uses the local data of edge devices to sequentially and cyclically train a predefined model. Eventually, an aggregated model will be generated, but this training process cannot be parallelized, resulting in a slow convergence speed. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a collaborative target tracking method, device, and equipment based on adaptive federated learning.

[0005] A collaborative target tracking method based on adaptive federated learning, the method comprising:

[0006] Determine a list of follower nodes according to multiple edge devices deployed in the target area, the list of follower nodes including multiple follower nodes.

[0007] Initialize the dictionaries of the local training information and reputation values of each of the follower nodes.

[0008] Load an initial target tracking model on each of the follower nodes.

[0009] Set a term; where the term is an integer greater than 0 and less than the term threshold.

[0010] When training in the first term, term = 1, and set the accuracy of the global model in the current term to the accuracy of the initial target tracking model.

[0011] According to the list of follower nodes, the dictionary, and the accuracy of the global model of the current term, an adaptive candidate node selection method based on a reputation mechanism is adopted to obtain a list of candidate nodes.

[0012] If term = 1, set the aggregation node of the current term as the first candidate node in the list of candidate nodes.

[0013] According to the aggregation node of the current term and the list of candidate nodes, an aggregation node selection and model aggregation method is adopted to obtain the global model of the current term and the aggregation node of the next term.

[0014] Update the initial target tracking model to the global model of the current term, update the accuracy of the global model of the current term to the accuracy of the global model of the current term, update the aggregation node of the current term to the aggregation node of the next term, increase term by 1, enter the next term, and stop iterative training until term is greater than the term threshold to obtain the final collaborative target tracking model.

[0015] Use the final collaborative target tracking model to track the target to be tracked in the target area to obtain the collaborative target tracking result.

[0016] In one embodiment, according to the list of follower nodes, the dictionary, and the accuracy of the current global model, an adaptive candidate node selection method based on a reputation mechanism is adopted to obtain a list of candidate nodes; the specific steps of the adaptive candidate node selection method based on the reputation mechanism include:

[0017] Initialize a list of candidate nodes.

[0018] Determine whether each follower node in the list of follower nodes can be selected as a candidate node. If a node has never been selected as a candidate node, add the follower node as a candidate node to the list of candidate nodes; otherwise, if the number of times a node has not been selected is less than 3 times, add the follower node as a candidate node to the list of candidate nodes; if a node has not been selected as a candidate node more than 3 times, remove the follower node from the group.

[0019] According to the list of candidate nodes, the dictionary, the accuracy of the global model, and the number of training rounds, determine the reputation value of each candidate node in the list of candidate nodes.

[0020] Update the reputation value attribute in the dictionary with the obtained reputation value.

[0021] In one embodiment, according to the candidate node list, the dictionary, the accuracy of the global model, and the number of training rounds, determining the reputation value of each candidate node in the candidate node list includes:

[0022] According to the candidate node list, the dictionary, the accuracy of the global model, and the number of training rounds, determining the reputation value of each candidate node in the candidate node list as:

[0023]

[0024] Wherein, is the reputation value of candidate node i in the t-th round of training, is the reputation value of candidate node i in the (t - 1)-th round of training, and avg t are respectively the precision of the local model of candidate node i and the average precision of all candidate nodes in the t-th round of training; is the accuracy of the global model in the (t - 1)-th round of training, and α, w1 are global parameters.

[0025] In one embodiment, according to the current tenure aggregation node and the candidate node list, using an aggregation node selection and model aggregation method to obtain the global model of the current tenure and the aggregation node of the next tenure, the specific steps of the aggregation node selection and model aggregation method in the steps include:

[0026] According to the remaining computing power of each candidate node in the candidate node list and the communication delay when the candidate node serves as an aggregation node, obtaining the aggregation ability value of each candidate node.

[0027] Calculating the aggregation ability value of the current tenure aggregation node.

[0028] According to the aggregation ability value of each candidate node and the aggregation ability value of the current tenure aggregation node, selecting the candidate node corresponding to the node with the largest aggregation ability value as the current aggregation node.

[0029] Training the initial target tracking model with local data at each candidate node to obtain the local model of each candidate node,

[0030] Aggregating the local models of all candidate nodes at the current aggregation node to obtain the global model of the first round of global training.

[0031] Loading the global model of the first round of global training into each candidate node in the candidate node list, and continuing the next round of global training to obtain the global model of the second round of global training, and so on, until the number of global training rounds reaches a preset threshold, to obtain the global model of the current tenure.

[0032] In one embodiment, according to the remaining computing power of each candidate node in the candidate node list and the communication delay when the candidate node is an aggregation node, an aggregation ability value of each candidate node is obtained, including:

[0033] According to the remaining computing power of each candidate node in the candidate node list and the communication delay when the candidate node is an aggregation node, the aggregation ability value of each candidate node is:

[0034] Cap i = w2 * pow i + (1 - w2) * delay i

[0035] Wherein, pow i is the remaining available computing power of candidate node i, delay i is the communication delay when candidate node i is an aggregation node, and w2 is a global parameter.

[0036] In one embodiment, the remaining available computing power of candidate node i is:

[0037] pow i = route i * (1 - cpu i )

[0038] Where route i is the number of threads, and cpu i is the CPU usage.

[0039] In one embodiment, the communication delay when candidate node i is an aggregation node is:

[0040] delay i = max(d ij * X / v)

[0041] Where, d ij is the distance between node i and node j, X is the model size, and v is the network transmission rate.

[0042] In one embodiment, aggregating the local models of all candidate nodes at the current aggregation node to obtain the global model for the first round of global training, including:

[0043] Set the global aggregation learning rate.

[0044] Calculate the weights of the local models of each candidate node.

[0045] Update the global model of the previous round according to the global aggregation learning rate, the weights of the local models of the candidate nodes, and the aggregation weights of the local models of the candidate nodes to obtain the global model of the first round of global training. The update formula of the global model is as follows:

[0046] W epoch =W epoch-1 +w_acc[c]*w[c]*γ

[0047] where, W epoch is the global model of the current round of global training, W epoch-1 is the global model of the previous round of global training, γ is the global aggregation learning rate, w[c] is the weight of the local model of the c-th candidate node; w_acc[c] is the aggregation weight of the c-th candidate node.

[0048] A collaborative target tracking device for adaptive federated learning, the device includes:

[0049] A follower node list determination module, configured to determine a follower node list according to multiple edge devices deployed in a target area, where the follower node list includes multiple follower nodes.

[0050] An initialization module, configured to initialize the dictionaries of the local training information and reputation values of each of the follower nodes; load an initial target tracking model on each of the follower nodes; set a term, where term is an integer greater than 0 and less than a term threshold; when training in the first term, term = 1, and set the accuracy of the current global model to the accuracy of the initial target tracking model.

[0051] A candidate node list selection module, configured to obtain a candidate node list by using an adaptive candidate node selection method based on a reputation mechanism according to the follower node list, the dictionary, and the accuracy of the current global model.

[0052] An aggregation node selection and model aggregation module, configured to: if term = 1, set the aggregation node of the current term to the first candidate node in the candidate node list; according to the aggregation node of the current term and the candidate node list, use an aggregation node selection and model aggregation method to obtain the global model of the current term and the aggregation node of the next term; update the initial target tracking model to the global model of the current term, update the accuracy of the current global model to the accuracy of the global model of the current term, update the aggregation node of the current term to the aggregation node of the next term, increase term by 1, enter the next term, and stop iterative training until term is greater than the term threshold to obtain a final collaborative target tracking model.

[0053] A collaborative target tracking module, which is used to track the target to be tracked in the target area by using the final collaborative target tracking model to obtain the collaborative target tracking result.

[0054] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of any of the above methods are implemented.

[0055] In the above collaborative target tracking method, device and equipment based on adaptive federated learning, the method introduces federated learning into this field, determines follower nodes according to edge devices, uses an adaptive candidate node selection method based on a reputation mechanism to select candidate nodes participating in training from the follower nodes, selects aggregation nodes from the candidate nodes, then distributes the model to the candidate nodes for local training, and transmits the updated parameters of each candidate node to the aggregation node for aggregation. After several tenure trainings, the final collaborative target tracking model is obtained, and the final collaborative target tracking model is used to track the target to be tracked in the target area to obtain the collaborative target tracking result. Compared with the most advanced existing methods, this method has comparable or even better accuracy, and at the same time requires less data transmission cost and less time consumption. Description of the Drawings

[0056] Figure 1 It is a schematic flowchart of the collaborative target tracking method of adaptive federated learning in an embodiment;

[0057] Figure 2 It is the overall architecture of the collaborative target tracking method of adaptive federated learning in an embodiment;

[0058] Figure 3 It is a diagram showing how follower nodes, candidate nodes and aggregation nodes are converted into each other in an embodiment;

[0059] Figure 4 It is a structural block diagram of the collaborative target tracking device of adaptive federated learning in an embodiment;

[0060] Figure 5 It is the internal structure diagram of a computer device in an embodiment. Detailed Embodiments

[0061] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] In the present invention, by introducing federated learning into this field, the federated learning (FL) framework can fully meet the requirements of edge collaborative target tracking. On the one hand, FL greatly reduces the amount of data transmitted while providing privacy protection. On the other hand, FL can parallelize the training process and has a high convergence speed.

[0063] The present invention designs a collaborative target tracking method based on adaptive federated learning, called FedTrack. This method enables collaborative training of a large amount of data on edge devices without transmitting the original data. The core of FedTrack is to propose a reputation mechanism, that is, carefully select learning participants so that incapable or malicious clients will not participate in the learning process. By improving the reliability and quality of the input data, FedTrack ensures the accuracy of the model while accelerating the training process.

[0064] In the FedTrack method, three types of nodes are defined, namely follower nodes, candidate nodes, and aggregation nodes. Follower nodes are data owners willing to participate in collaborative training. Candidate nodes are those that participate in collaborative training and act as clients for local training. Aggregation nodes perform model aggregation and act as servers.

[0065] In one embodiment, as Figure 1 、 Figure 2 shown, a collaborative target tracking method based on adaptive federated learning is provided, and the method includes the following steps:

[0066] Step 100: According to multiple edge devices deployed in the target area, determine a list of follower nodes, and the list of follower nodes includes multiple follower nodes.

[0067] Specifically, the multiple edge devices deployed in the target area can be, but are not limited to, mobile devices, drones, cameras, sensors, and other edge devices.

[0068] Devices in the edge devices that are willing and able to collaborate in training the tracking model will become follower nodes. As Figure 2 shown, this camera is not a follower node because it cannot calculate and store. In addition, only follower nodes with a sufficient reputation value will be selected as candidate nodes allowed to participate in training, and aggregation nodes will be selected from the candidate nodes by evaluating the computing power and transmission capacity of the candidate nodes. The reputation of each follower is quantified by jointly considering the test accuracy of its local model and its contribution to the global model. For example, Figure 2 in, the drone is used as the aggregation node to complete the aggregation and distribution of the model.

[0069] Step 102: Initialize the dictionaries of the local training information and reputation values for each follower node; load the initial target tracking model on each follower node; set the term, where the term is an integer greater than 0 and less than the term threshold.

[0070] Specifically, the roles of follower nodes, candidate nodes, and aggregation nodes are updated regularly. A term is defined to describe the time during which a node can act as the current role. For simplicity, it is defined that N rounds of global training is a term, where N is an integer.

[0071] Preferably, the loaded initial target tracking model is the initial MDNet model. The initial MDNet model is loaded on the follower nodes, but local training is not directly performed on the follower nodes. Instead, local training is performed on the candidate nodes selected after screening the follower nodes to obtain local models.

[0072] Step 104: During the training in the first term, when term = 1, set the accuracy of the global model in the current term to the accuracy of the initial target tracking model.

[0073] Step 106: According to the follower node list, the dictionary, and the accuracy of the global model in the current term, use an adaptive candidate node selection method based on the reputation mechanism to obtain the candidate node list.

[0074] Specifically, in the edge environment, nodes are highly heterogeneous, which means that some data sources may be unreliable or of low quality. Therefore, it is important to select candidate nodes as clients. Doing so brings two benefits. On the one hand, the pruning of malicious nodes and low-performance nodes avoids their negative impact on the global model. On the other hand, since fewer local models are aggregated, both the communication overhead and the training time are reduced.

[0075] The adaptive candidate node selection method based on the reputation mechanism is used to select from the follower node list according to the attribute values of the nodes in the follower node list and the dictionary, determine the candidate node list, and calculate and update the reputation value of each candidate node.

[0076] Step 108: If term = 1, then set the aggregation node in the current term to the first candidate node in the candidate node list.

[0077] Step 110: According to the aggregation node in the current term and the candidate node list, use the aggregation node selection and model aggregation method to obtain the global model in the current term and the aggregation node in the next term.

[0078] Specifically, the aggregation node is selected from the candidate nodes. The aggregation node should be able to aggregate local models effectively and economically. For example, in Figure 2In this, the UAV acts as an aggregator node to complete the aggregation and distribution of the model.

[0079] The aggregator node selection and model aggregation method is used to select the candidate node with the largest aggregation ability value from the candidate node list by calculating the aggregation ability values of all candidate nodes as the aggregator node. Then, local training of the initial target tracking model is performed on all candidate nodes in the candidate node list using local data to obtain the local module of each candidate node. All local models are aggregated in the aggregator node to obtain the global model for one round of global training. The global model for this round of global training is loaded into all candidate nodes in the candidate node list to continue local training. When the number of rounds of global training reaches the preset value, the training stops, and the global model for the current tenure is obtained.

[0080] Step 112: Update the initial target tracking model to the global model for the current tenure, update the accuracy of the global model for the current tenure to the accuracy of the global model for the current tenure, update the aggregator node for the current tenure to the aggregator node for the next tenure, increment term by 1, enter the next tenure, and stop iterative training until term is greater than the tenure threshold to obtain the final collaborative target tracking model.

[0081] Specifically, as shown in Algorithm 1, the roles of each node do not change within each tenure, but when a tenure ends, both the aggregator node and candidate nodes will be re-elected. Therefore, in the new tenure, the aggregator node may degenerate into a candidate node, and unqualified candidate nodes will be adjusted to follower nodes. In this way, this method can prune malicious and low-end nodes and use the most suitable aggregator node to aggregate local models. The pseudocode of Algorithm 1 is as follows.

[0082]

[0083]

[0084] Figure 3 Describes how follower nodes, candidate nodes, and aggregator nodes convert to each other. When the reputation of a follower node is high enough, it will be selected as a candidate node; by evaluating its computing and transmission capabilities, it is upgraded to an aggregator node.

[0085] Step 114: Use the final collaborative target tracking model to track the target to be tracked within the target area to obtain the collaborative target tracking result.

[0086] In the above collaborative target tracking method for adaptive federated learning, the method introduces federated learning into this field, determines follower nodes according to edge devices, uses an adaptive candidate node selection method based on a reputation mechanism to select candidate nodes participating in training from the follower nodes, selects an aggregation node from the candidate nodes, then distributes the model to the candidate nodes for local training, transmits the updated parameters of each candidate node to the aggregation node for aggregation, and after several rounds of training, obtains a final collaborative target tracking model, and uses the final collaborative target tracking model to track the target to be tracked in the target area to obtain a collaborative target tracking result. Compared with the state-of-the-art existing methods, this method has comparable or even better accuracy, while requiring less data transmission cost and less time consumption.

[0087] In one embodiment, the specific steps of the adaptive candidate node selection method based on the reputation mechanism in step 106 include:

[0088] Step 200: Initialize a list of candidate nodes.

[0089] Step 202: Determine whether each follower node in the follower node list can be selected as a candidate node. If a node has never been selected as a candidate node, add the follower node as a candidate node to the candidate node list; otherwise, if a node has been unselected less than 3 times, add the follower node as a candidate node to the candidate node list; if a node has not been selected as a candidate node more than 3 times, remove the follower node from the group.

[0090] Step 204: Determine the reputation value of each candidate node in the candidate node list according to the candidate node list, the dictionary, the accuracy of the global model, and the number of training rounds.

[0091] Step 206: Update the reputation value attribute in the dictionary with the obtained reputation value.

[0092] In one embodiment, step 204 includes: determining the reputation value of each candidate node in the candidate node list according to the candidate node list, the dictionary, the accuracy of the global model, and the number of training rounds as:

[0093]

[0094] Where, is the reputation value of candidate node i in the t-th round of training, is the reputation value of candidate node i in the (t - 1)-th round of training, and avg t are respectively the accuracy of the local model of candidate node i and the average accuracy of all candidate nodes in the t-th round of training; is the accuracy of the global model in the (t - 1)-th round of training, where α and w1 are global parameters. In Equation (1), represents the gap between the test accuracy of the local model of node i and the average test accuracy of all nodes in this round of training; represents how much the accuracy of node i can be improved through this round of training, and is also affected by its reputation in the previous round of training. By adding the above parts with different weights, the value of can be finally obtained.

[0095] In a specific embodiment, according to the definition of reputation, Algorithm 2 details the candidate node selection strategy. The first part of this algorithm will create and maintain a list of candidate nodes. As shown in lines 2 - 10, if a follower node has never been selected, it will directly become a candidate node. On the contrary, once a follower node has not been selected three times, it will be kicked out of the group. The second part is the basis for whether a node is selected, corresponding to calculating its reputation in lines 11 - 14. If the reputation value of a node reaches 60% of the aggregator, it is considered to have good performance and can participate in the training. Otherwise, it will not be selected as a candidate.

[0096]

[0097]

[0098] In one of the embodiments, the specific steps of the aggregator node selection and model aggregation method in step 110 include:

[0099] Step 300: Obtain the aggregation ability value of each candidate node according to the remaining computing power of each candidate node in the candidate node list and the communication delay when the candidate node serves as the aggregator node.

[0100] Step 302: Calculate the aggregation ability value of the aggregator node in the current term.

[0101] Step 304: Select the candidate node corresponding to the node with the largest aggregation ability value as the current aggregator node according to the aggregation ability value of each candidate node and the aggregation ability value of the aggregator node in the current term.

[0102] Step 306: Each candidate node trains the initial target tracking model with local data to obtain the local model of each candidate node,

[0103] Step 308: Aggregate the local models of all candidate nodes at the current aggregator node to obtain the global model for the first round of global training.

[0104] Step 310: Load the global model of the first-round global training into each candidate node in the candidate node list, continue with the next round of global training to obtain the global model of the second-round global training, and so on until the number of global training rounds reaches a preset threshold, thereby obtaining the global model of the current term.

[0105] The pseudocode of the global aggregation mechanism is as follows. The global aggregation mechanism mainly consists of two parts. One part is to select the aggregator node, corresponding to lines 2 - 7. Based on Formula 2, each candidate node calculates the aggregation ability value. Algorithm 3 specifies our global aggregation mechanism. It mainly consists of two parts. One part is to select the aggregator node, corresponding to lines 2 - 7. Based on Formula 2, each candidate node calculates the aggregation ability value, constructs a token with its reputation value, and then broadcasts it with the aggregation ability value. Each node compares and iterates the received token with the stored token, retains the token with the highest aggregation ability value, and maintains a global list of reputation values. The node will sign and broadcast the finally stored token. If more than half of the election results select the same token, the master node is the node corresponding to that token, and the election is completed. The other part is the complete process, mainly reflected in lines 8 - 15. According to Formula 1, the reputation value is associated with the contribution of the candidate node to the global model. Therefore, in each round of global aggregation, according to the reputation value list maintained by the aggregator node, aggregation weights are assigned to other participating nodes.

[0106]

[0107]

[0108] In one embodiment, Step 300 includes: obtaining the aggregation ability value of each candidate node according to the remaining computing power of each candidate node in the candidate node list and the communication delay when the candidate node is an aggregation node, as follows:

[0109] Cap i =w2*pow i +(1 - w2)*delay i (2)

[0110] where, pow i is the remaining available computing power of candidate node i, delay i is the communication delay when candidate node i is an aggregation node, and w2 is a global parameter.

[0111] In one embodiment, the remaining available computing power of candidate node i is:

[0112] pow i =route i *(1 - cpu i) (3)

[0113] where route i is the number of threads, and cpu i is the CPU usage.

[0114] In one embodiment, the communication delay when the candidate node i is an aggregation node is:

[0115] delay i = max(d ij * X / v) (4)

[0116] where d ij is the distance between node i and node j, X is the model size, and v is the network transmission rate.

[0117] In one embodiment, step 308 includes: setting a global aggregation learning rate; calculating the weights of the local models of each candidate node; updating the previous round of the global model according to the global aggregation learning rate, the weights of the local models of the candidate nodes, and the aggregation weights of the local models of the candidate nodes to obtain the global model of the first round of global training, where the update formula of the global model is:

[0118] W epoch = W epoch-1 + w_acc[c] * w[c] * γ (5)

[0119] where W epoch is the global model of the current round of global training, W epoch-1 is the global model of the previous round of global training, γ is the global aggregation learning rate, w[c] is the weight of the local model of the c-th candidate node; w_acc[c] is the aggregation weight of the c-th candidate node.

[0120] Specifically, providing reasonable rewards for the participants in the collaborative training is beneficial to the continuous and stable operation of the incentive system. For these three different nodes, the system has different incentive measures. Among them, the follower nodes do not make actual contributions to the system, but consume their own computing power through local training. For such nodes, only the computing power consumption needs to be compensated, and the data they contribute should also be rewarded. For the candidate nodes that support the global model, the rewards should be distributed according to the result of normalization of the reputation values (that is, each reputation value is divided by their sum). For the aggregation nodes, in addition to local training, they also undertake global aggregation work and should receive additional rewards. The rewards will be distributed at the end of each round of global training.

[0121] It should be understood that although Figure 1The steps in the flowchart are shown in sequence according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least a part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in rotation with at least a part of other steps or sub-steps or stages of other steps.

[0122] In one embodiment, as Figure 4 shown, a collaborative target tracking device for adaptive federated learning is provided, including: a follower node list determination module, a B module, and a C module, where:

[0123] The follower node list determination module is used to determine a follower node list according to multiple edge devices deployed in the target area. The follower node list includes multiple follower nodes.

[0124] The initialization module is used to initialize the dictionary of local training information and reputation values for each follower node; load the initial target tracking model on each follower node; set the term, where term is an integer greater than 0 and less than the term threshold; when training in the first term, term = 1, and set the accuracy of the current global model to the accuracy of the initial target tracking model.

[0125] The candidate node list selection module is used to obtain a candidate node list by using an adaptive candidate node selection method based on the reputation mechanism according to the follower node list, the dictionary, and the accuracy of the current global model.

[0126] The aggregation node selection and model aggregation module is used to, if term = 1, set the current term aggregation node to the first candidate node in the candidate node list; according to the current term aggregation node and the candidate node list, use the aggregation node selection and model aggregation method to obtain the global model of the current term and the aggregation node of the next term; update the initial target tracking model to the global model of the current term, update the accuracy of the current global model to the accuracy of the global model of the current term, update the current term aggregation node to the aggregation node of the next term, increase term by 1, enter the next term, and stop iterative training until term is greater than the term threshold to obtain the final collaborative target tracking model.

[0127] The collaborative target tracking module is used to track the target to be tracked in the target area by using the final collaborative target tracking model to obtain the collaborative target tracking result.

[0128] In one embodiment, the adaptive candidate node selection method based on the reputation mechanism in the candidate node list selection module is used to initialize a list of candidate nodes; determine whether each follower node in the follower node list can be selected as a candidate node. If a node has never been selected as a candidate node, then add the follower node as a candidate node to the candidate node list; otherwise, if the number of times a node has not been selected is less than 3 times, then add the follower node as a candidate node to the candidate node list; if a node has not been selected as a candidate node more than 3 times, then remove the follower node from the group; determine the reputation value of each candidate node in the candidate node list according to the candidate node list, the dictionary, the accuracy of the global model, and the number of training rounds; update the reputation value attribute in the dictionary with the obtained reputation value.

[0129] In one embodiment, the candidate node list selection module is further used to determine the expression of the reputation value of each candidate node in the candidate node list as shown in Equation (1) according to the candidate node list, the dictionary, the accuracy of the global model, and the number of training rounds.

[0130] In one embodiment, the aggregator node selection and model aggregation method in the aggregator node selection and model aggregation module is used to obtain the aggregation ability value of each candidate node according to the remaining computing power of each candidate node in the candidate node list and the communication delay when the candidate node is an aggregator node; calculate the aggregation ability value of the current term aggregator node; select the candidate node corresponding to the node with the largest aggregation ability value as the current aggregator node according to the aggregation ability value of each candidate node and the aggregation ability value of the current term aggregator node; train the initial target tracking model with local data at each candidate node to obtain the local model of each candidate node; aggregate the local models of all candidate nodes at the current aggregator node to obtain the global model of the first round of global training; load the global model of the first round of global training into each candidate node in the candidate node list and continue the next round of global training to obtain the global model of the second round of global training, and so on until the number of global training rounds reaches a preset threshold to obtain the global model of the current term.

[0131] In one embodiment, the aggregator node selection and model aggregation module is further used to determine the expression of the aggregation ability value of each candidate node according to the remaining computing power of each candidate node in the candidate node list and the communication delay when the candidate node is an aggregator node as shown in Equation (2).

[0132] In one embodiment, the expression of the remaining available computing power of candidate node i in the aggregator node selection and model aggregation module is as shown in Equation (3).

[0133] In one embodiment, the expression of the communication delay when the candidate node i is the aggregation node in the aggregation node selection and model aggregation module is shown in Equation (4).

[0134] In one embodiment, the aggregation node selection and model aggregation module is further configured to set a global aggregation learning rate; calculate the weights of the local models of each candidate node; and update the previous-round global model according to the global aggregation learning rate, the weights of the local models of the candidate nodes, and the aggregation weights of the local models of the candidate nodes, to obtain the global model for the first-round global training, where the update formula of the global model is shown in Equation (5).

[0135] For the specific limitations of the collaborative target tracking device for adaptive federated learning, reference can be made to the limitations of the collaborative target tracking method for adaptive federated learning in the foregoing, which will not be elaborated herein. Each module in the above-mentioned collaborative target tracking device for adaptive federated learning can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in the form of hardware or independent of it, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0136] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a collaborative target tracking method for adaptive federated learning. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0137] Those skilled in the art can understand that Figure 5 the structure shown in

[0138] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps in the above method embodiment are implemented.

[0139] Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or an external cache. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0140] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0141] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A collaborative target tracking method for adaptive federated learning, characterized in that, The method includes: Determine a list of follower nodes according to multiple edge devices deployed in the target area, where the list of follower nodes includes multiple follower nodes; Initialize the dictionaries of the local training information and reputation values of each of the follower nodes; Load an initial target tracking model on each of the follower nodes; Set a term; where the term is an integer greater than 0 and less than the term threshold; During the training of the first term when term = 1, set the accuracy of the global model of the current term to the accuracy of the initial target tracking model; According to the list of follower nodes, the dictionary, and the accuracy of the global model of the current term, use an adaptive candidate node selection method based on the reputation mechanism to obtain a list of candidate nodes; If term = 1, set the aggregation node of the current term to the first candidate node in the list of candidate nodes; According to the aggregation node of the current term and the list of candidate nodes, use an aggregation node selection and model aggregation method to obtain the global model of the current term and the aggregation node of the next term; Update the initial target tracking model to the global model of the current term, update the accuracy of the global model of the current term to the accuracy of the global model of the current term, update the aggregation node of the current term to the aggregation node of the next term, increase term by 1, enter the next term, and stop iterative training until term is greater than the term threshold to obtain the final collaborative target tracking model; Use the final collaborative target tracking model to track the target to be tracked in the target area to obtain a collaborative target tracking result.

2. The method according to claim 1, characterized in that, According to the list of follower nodes, the dictionary, and the accuracy of the current global model, use an adaptive candidate node selection method based on the reputation mechanism to obtain a list of candidate nodes; The specific steps of the adaptive candidate node selection method based on the reputation mechanism described above include: Initialize a list of candidate nodes; Judge whether each follower node in the list of follower nodes can be selected as a candidate node. If a node has never been selected as a candidate node, add the follower node as a candidate node to the list of candidate nodes; otherwise, if the number of times a node has not been selected is less than 3 times, add the follower node as a candidate node to the list of candidate nodes; if a node has not been selected as a candidate node more than 3 times, remove the follower node from the group; Determine the reputation value of each candidate node in the list of candidate nodes according to the list of candidate nodes, the dictionary, the accuracy of the global model, and the number of training rounds; Update the reputation value attribute in the dictionary with the obtained reputation value.

3. The method according to claim 2, wherein Determine the reputation value of each candidate node in the list of candidate nodes according to the list of candidate nodes, the dictionary, the accuracy of the global model, and the number of training rounds, including: Determine the reputation value of each candidate node in the list of candidate nodes as: Among them, is the reputation value of candidate node i in the t-th round of training, is the reputation value of candidate node i in the (t-1)-th round of training, and avg t are the accuracy of the local model of candidate node i and the average accuracy of all candidate nodes in the t-th round of training, respectively; is the accuracy of the global model in the (t-1)-th round of training, and α, w1 are global parameters.

4. The method according to claim 1, wherein Aggregate the nodes according to the current term and the list of candidate nodes, and adopt the aggregate node selection and model aggregation method to obtain the global model of the current term and the aggregate node of the next term. The specific steps of the aggregate node selection and model aggregation method in the steps include: Obtain the aggregation ability value of each candidate node according to the remaining computing power of each candidate node in the candidate node list and the communication delay when the candidate node is the aggregate node; Calculate the aggregation ability value of the aggregate node of the current term; According to the aggregation ability value of each candidate node and the aggregation ability value of the aggregate node of the current term, select the candidate node corresponding to the node with the largest aggregation ability value as the current aggregate node; Train the initial target tracking model with local data at each candidate node to obtain the local model of each candidate node; Aggregate the local models of all candidate nodes at the current aggregate node to obtain the global model of the first round of global training; Load the global model of the first round of global training into each candidate node in the candidate node list, continue the next round of global training, and obtain the global model of the second round of global training, and so on until the number of global training rounds reaches the preset threshold to obtain the global model of the current term.

5. The method according to claim 4, characterized in that, Obtain the aggregation ability value of each candidate node according to the remaining computing power of each candidate node in the candidate node list and the communication delay when the candidate node is the aggregate node, including: Obtain the aggregation ability value of each candidate node according to the remaining computing power of each candidate node in the candidate node list and the communication delay when the candidate node is the aggregate node as: Cap i = w2 * pow i + (1 - w2) * delay i Among them, pow i is the remaining available computing power of candidate node i, and delay i is the communication delay when candidate node i is the aggregation node, and w2 is a global parameter.

6. The method according to claim 5, characterized in that, The remaining available computing power of candidate node i is: pow i = route i *(1 - cpu i ) Among them, route i is the number of threads, cpu i is the CPU usage.

7. The method according to claim 5, wherein The communication delay when candidate node i is the aggregate node is: delay i = max(d ij *X / v) where d ij is the distance between node i and node j, X is the model size, and v is the network transmission rate.

8. The method according to claim 4, wherein Aggregate the local models of all candidate nodes at the current aggregate node to obtain the global model of the first round of global training, including: Set the global aggregation learning rate; Calculate the weights of the local models of each candidate node; Update the global model of the previous round according to the global aggregation learning rate, the weights of the local models of the candidate nodes, and the aggregation weights of the local models of the candidate nodes to obtain the global model of the first round of global training, where the update formula of the global model is: W epoch = W epoch-1 + w_acc[c] * w[c] * γ Among them, W epoch is the global model for the current round of global training, and W epoch-1 is the global model for the previous round of global training. γ is the global aggregation learning rate, and w[c] is the weight of the local model of the c-th candidate node; w_acc[c] is the aggregation weight of the c-th candidate node.

9. A collaborative target tracking device for adaptive federated learning, characterized in that, The device includes: A follower node list determination module, configured to determine a follower node list according to a plurality of edge devices deployed in a target area, where the follower node list includes a plurality of follower nodes; An initialization module, configured to initialize the local training information and the dictionary of reputation values of each follower node; load the initial target tracking model on each follower node; set the term, where term is an integer greater than 0 and less than the term threshold; when training in the first term, term = 1, and set the accuracy of the current global model to the accuracy of the initial target tracking model; A candidate node list selection module, configured to obtain a candidate node list by using an adaptive candidate node selection method based on a reputation mechanism according to the follower node list, the dictionary, and the accuracy of the current global model; The aggregation node selection and model aggregation module is used to, if term = 1, set the current term aggregation node as the first candidate node in the candidate node list; according to the current term aggregation node and the candidate node list, use the aggregation node selection and model aggregation method to obtain the global model of the current term and the aggregation node of the next term; update the initial target tracking model to the global model of the current term, update the accuracy of the current global model to the accuracy of the global model of the current term, update the current term aggregation node to the aggregation node of the next term, increase term by 1, enter the next term, and stop iterative training until term is greater than the term threshold to obtain the final collaborative target tracking model; The collaborative target tracking module is used to track the target to be tracked in the target area by using the final collaborative target tracking model to obtain the collaborative target tracking result.

10. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 8.

Images