A person re-identification method based on federated split learning

Abstract

The present invention discloses a pedestrian re-identification method based on federated split learning, including: S1. loading a pedestrian re-identification network; S2. training the pedestrian re-identification network, dividing the pedestrian re-identification network model as a local local model according to the number of layers to obtain more than one first sub-model, splitting the model parameters to be uploaded, and uploading the split first sub-model to the parameter server; S3. assembling and aggregating all the first sub-models to generate a global model; S4. splitting the global model to obtain more than one second sub-model, and sending each second sub-model to a target training node; S5. The target training node assembles the second sub-model with the local local model to obtain an initialization model and load parameters, and uses the initialization model as a new local local model; S6. looping S2-S5 until the local local model completes convergence. The present invention has high security and accuracy, and effectively improves the communication efficiency of the network.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more specifically to a pedestrian re-identification method based on federated split learning. Background Art

[0002] Traditional pedestrian re-identification methods require the collection of pedestrian data and the labeling of user data for training. However, since pedestrian data is usually private data, the pedestrian data held by each data owner cannot be made public, and data cannot be shared between communities, resulting in data silos.

[0003] Ordinary federated learning implements the pedestrian re-identification algorithm without sharing the user's private data, solving the problem of data islands and allowing all data owners to participate in the model training process. However, the model or gradient information needs to be transferred during the training process. Although the transfer of these parameters reduces the possibility of data leakage caused by direct data transmission, there is still a risk of data leakage. That is, malicious attack nodes can capture the gradient information during the transmission process multiple times, infer the original data distribution based on the change of gradient, and cause data leakage.

[0004] In addition, the existing federated learning methods require uploading the model or gradient information after the training node completes the training. When the model parameters are relatively large, frequent model transmission will bring large communication overhead.

[0005] Therefore, how to provide a safe, accurate and simple pedestrian re-identification method based on federated split learning is an urgent problem that technicians in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a pedestrian re-identification method based on federated split learning

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A person re-identification method based on federated split learning includes the following steps:

[0009] S1. Load the person re-identification network built according to the federated learning task;

[0010] S2. Train the person re-identification network, use the person re-identification network model after completing several rounds of local training as the local model, split the local local model according to the model splitting group to which it belongs to obtain more than one first sub-model, split the model parameters of the first sub-model, obtain the model parameters that need to be uploaded after the splitting, and upload them to the parameter server;

[0011] S3. The parameter server obtains and assembles all first sub-models, aggregates the first sub-models belonging to the same model split group, and generates a global model for each model split group;

[0012] S4. Split the global model to obtain more than one second sub-model, and send each second sub-model to the target training node;

[0013] S5. The target training node receives the second sub-model, assembles the second sub-model with the local partial model, obtains an initialization model, loads the initialization model parameters, and uses the initialization model as a new local partial model;

[0014] S6. Loop through S2-S5 until the local model converges. The convergence condition is that the change in the loss value for several consecutive rounds is less than a threshold.

[0015] Preferably, the process of model splitting includes:

[0016] (1) Generating computing levels; generating one or more computing power levels by grouping nodes based on their computing power;

[0017] (2) Define the splitting mode; based on the generated level grouping, allocate different parameter amounts for different computing power levels. Levels with strong computing power should allocate more parameters for training, while nodes with weak computing power should allocate fewer parameters;

[0018] (3) According to the parameter quantity divided by the computing power level, the model parameters are divided, the model splitting process is executed, and different split models are generated; one of the split models corresponds to a computing power level, and each computing power level corresponds to a model splitting group;

[0019] (4) Node claiming model: Each training node selects the corresponding model split group based on its own machine computing power, and claims the corresponding split model to perform model training tasks.

[0020] Preferably, the process of model fusion assembly includes:

[0021] (1) Upload the split model: After the training node is completed, it uploads its corresponding split model;

[0022] (2) Split model query: Based on the claimed split model of each training node, query the split group to which the current split model belongs;

[0023] (3) Split model fusion: the fusion process of the split models in the same group;

[0024] (4) Assembling the split models: sorting and splicing the model split groups in order of computing power, arranging the model parameters in order, and synthesizing a new model.

[0025] Preferably, in S5, a model regularization term is added to constrain the model optimization process. For the second sub-model W received by the training node s , and the local local model W ns , the constraints are:

[0026]

[0027] in Indicates the parameters that need to be federated in the model at time t, represents the model parameter that does not perform the splitting process at time t, λ1 and λ2 represent the weights of the two parts;

[0028] The loss function is:

[0029]

[0030] Among them, L represents the loss of the original network.

[0031] Preferably, before S1, the following contents are also included:

[0032] When the federated learning task initiator initiates the federated learning task, the training node obtains the specified federated learning task and initializes the federated learning environment;

[0033] The training node performs data preprocessing on the pedestrian re-identification image data.

[0034] Preferably, the content of initializing the federated learning environment includes: initializing the network architecture model, the fusion model method, the number of fusion iterations, the learning rate and the optimizer.

[0035] Preferably, preprocessing includes data augmentation.

[0036] It can be seen from the above technical solution that compared with the prior art, the present invention discloses a pedestrian re-identification method based on federated split learning. First, federated split learning is used to integrate the pedestrian re-identification algorithm. Federated split learning is used to only transfer the model during the training process, and the training model is split at the same time to avoid the possibility of inferring the original data from the gradient information, thereby enhancing the protection of data privacy without the need to share pedestrian data, thereby protecting the data security of users' pedestrian re-identification.

[0037] Secondly, due to the above-mentioned protection of pedestrian data security, more users can participate in the model training process, further expanding the pedestrian data set, solving the data island problem, and thus improving the accuracy and generalization ability of the model.

[0038] Finally, due to the method of federated split learning, each model is split, and each training node only has a part of the model. During the transmission process, only the part of the model parameters it owns is transmitted, which effectively improves the communication efficiency of each training participant. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0040] Figure 1 The accompanying drawing is a flow chart of a pedestrian re-identification method based on federated split learning provided by the present invention;

[0041] Figure 2 The accompanying drawing is a schematic diagram of the process of model splitting and model fusion assembly in a pedestrian re-identification method based on federated split learning provided by the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The embodiment of the present invention discloses a pedestrian re-identification method based on federated split learning, such as Figure 1 As shown, the following steps are included:

[0044] S1. Load the person re-identification network built according to the federated learning task;

[0045] S2. Train the person re-identification network, use the person re-identification network model after completing several rounds of local training as the local model, split the local local model according to the model splitting group to which it belongs to obtain more than one first sub-model, split the model parameters of the first sub-model, obtain the model parameters that need to be uploaded after the splitting, and upload them to the parameter server;

[0046] S3. The parameter server obtains and assembles all first sub-models, aggregates the first sub-models belonging to the same model split group, and generates a global model for each model split group;

[0047] S4. Split the global model to obtain more than one second sub-model, and send each second sub-model to the target training node;

[0048] S5. The target training node receives the second sub-model, assembles the second sub-model with the local partial model, obtains an initialization model, loads the initialization model parameters, and uses the initialization model as a new local partial model;

[0049] S6. Loop through S2-S5 until the local model converges. The convergence condition is that the change in the loss value for several consecutive rounds is less than a threshold.

[0050] It should be noted that:

[0051] (1) The present invention adopts a secure distributed learning method of federated learning to protect data privacy. This feature makes it possible to be used for various privacy data, thereby solving the data island phenomenon and expanding the dataset for pedestrian re-identification, so that more image data can be used for model training and improving the accuracy of the model.

[0052] (2) The present invention adopts a federated split learning algorithm, which splits the model during training. Each training participant only owns a part of the model architecture. Even if its information is leaked, the overall data distribution cannot be inferred from the local information, thereby improving data security.

[0053] (3) The present invention adopts the method of model structure splitting in federated split learning to split the structure of the model. In the entire communication process, it is not necessary to transmit the entire model, but only part of the model parameters, thereby reducing the communication overhead of the model.

[0054] In order to further implement the above technical solution, Figure 2 As shown in the figure, the model splitting process includes:

[0055] (1) Generating computing levels; generating one or more computing power levels by grouping nodes based on their computing power;

[0056] (2) Define the splitting mode; based on the generated level grouping, allocate different parameter amounts for different computing power levels; the level with strong computing power should allocate more parameters for training, and the nodes with weak computing power should allocate fewer parameters; each computing power level can be represented by a specific number, and its size represents the strength of computing power, so the proportion of each computing power level in the total can be obtained. At the same time, the size of the parameter amount of each layer in the calculation model is calculated, and a number of layers are allocated to each level from small to large according to the above generated ratio. The ratio of the parameter amount allocated to each level to the total parameter amount should be roughly the same as its level ratio.

[0057] (3) According to the number of parameters divided by the computing power level, the model parameters are divided, the model splitting process is executed, and different split models are generated; one split model corresponds to one computing power level, and each computing power level corresponds to one model splitting group; for example, if there are three computing power levels, the model parameters are divided into three parts, and each computing power level corresponds to a part of the model parameters split above. That is, one split model corresponds to one computing power level and one model splitting group.

[0058] (4) Node claiming model: Each training node selects the corresponding model split group based on its own machine computing power, and claims the corresponding split model to perform model training tasks. After the model is split, a series of split models belonging to different model split groups are obtained. According to the computing power of each training node's own machine, the model split group closest to it is selected, so as to ensure that the model transmitted in the subsequent model upload and parameter server distribution process is the split model of this part, and the training, fusion and other processes only modify the model parameters of this part.

[0059] In order to further implement the above technical solution, Figure 2 As shown in the figure, the process of model fusion assembly includes:

[0060] (1) Upload the split model: After the training node is completed, it uploads its corresponding split model;

[0061] (2) Split model query: Based on the claimed split model of each training node, query the split group to which the current split model belongs;

[0062] (3) Split model fusion: the fusion process of the split models in the same group;

[0063] (4) Assembling the split models: sorting and splicing the model split groups in order of computing power, arranging the model parameters in order, and synthesizing a new model.

[0064] It should be noted that:

[0065] In step (1), each training node belongs to a different model split group, and the model parameters uploaded should also belong to the model split group; in the split model query in step (2), the model split group to which each node belongs is recorded in the aforementioned node claim model; and in step (3), the split models belonging to the same group are fused to obtain each group of fused models, which can be obtained by weighted average; in step (4), the models obtained by the above aggregation are assembled and spliced ​​in the order corresponding to the model network structure, thereby completing the entire fusion assembly process.

[0066] In order to further implement the above technical solution, in S5, a model regularization term is added to constrain the model optimization process. For the second sub-model W received by the training node s , and the local local model W ns , the constraints are:

[0067]

[0068] in Indicates the parameters that need to be federated in the model at time t, represents the model parameter that does not perform the splitting process at time t, λ1 and λ2 represent the weights of the two parts;

[0069] The loss function is:

[0070]

[0071] Among them, L represents the loss of the original network.

[0072] In order to further implement the above technical solution, S1 also includes the following contents:

[0073] When the federated learning task initiator initiates the federated learning task, the training node obtains the specified federated learning task and initializes the federated learning environment;

[0074] The training node performs data preprocessing on the pedestrian re-identification image data.

[0075] In order to further implement the above technical solution, the content of initializing the federated learning environment includes: initializing the network architecture model, fusion model method, fusion iteration number, learning rate and optimizer.

[0076] In order to further implement the above technical solution, preprocessing includes data enhancement.

[0077] It should be noted that:

[0078] Split regularization is used in the training process. Different split regularization weights are used for the model parameters of the split model and the remaining part, and added to the training nodes to focus on ensuring the accuracy of the split model.

[0079] The present invention can also adopt a group learning method to replace the federated learning method. That is, since a parameter service node is required in federated learning to continuously integrate the models of each training participant, and group learning is based on the decentralized characteristics of blockchain, the aggregation of the model can occur on each training participant.

[0080] Group learning can be understood as a decentralized federated learning solution, which removes the parameter server from the entire learning process and completely implements a point-to-point training fusion method. Since group learning is decentralized federated learning, it differs from ordinary federated learning solutions only in the way of fusion model, so this part is explained. The specific implementation process is as follows:

[0081] After the training node completes the training task, it obtains the latest global model parameters from the blockchain that stores the model parameters.

[0082] The training node obtains this part of the model and performs the fusion process locally. The fusion process is the same as the ordinary federated learning method.

[0083] After executing the above process, a new global model is obtained, and the new global model is uploaded to the blockchain, and the fusion process is completed.

[0084] Since only one training node can perform the fusion process at the same time in the above process, all training nodes are required to perform the fusion in sequence.

[0085] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0086] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A person re-identification method based on federated split learning, characterized in that: The following steps are involved: S1. Load the person re-identification network built according to the federated learning task; S2. Train the person re-identification network, use the person re-identification network model after completing several rounds of local training as the local model, and split it according to the model splitting group to which it belongs to obtain more than one first sub-model, split the model parameters of the first sub-model, obtain the model parameters that need to be uploaded after the splitting, and upload them to the parameter server; S3. The parameter server obtains and assembles all first sub-models, aggregates the first sub-models belonging to the same model split group, and generates a global model for each model split group; S4. Split the global model to obtain more than one second sub-model, and send each second sub-model to the target training node; S5. The target training node receives the second sub-model, assembles the second sub-model with the local partial model, obtains an initialization model, loads the initialization model parameters, and uses the initialization model as a new local partial model; S6. Loop through S2-S5 until the local model converges. The convergence condition is that the change in the loss value for several consecutive rounds is less than a threshold.

2. According to the method of person re-identification based on federated split learning in claim 1, it is characterized in that: The model splitting process includes: (1) Generating computing levels; generating one or more computing power levels by grouping nodes based on their computing power; (2) defining a splitting mode; based on the generated level grouping, dividing different parameter quantities for different computing power levels; (3) According to the parameter quantity divided by the computing power level, the model parameters are divided, the model splitting process is executed, and different split models are generated; one of the split models corresponds to one computing power level, and each computing power level corresponds to one model splitting group; (4) Node claiming model: Each training node selects the corresponding model split group based on its own machine computing power, and claims the corresponding split model to perform model training tasks.

3. The method for person re-identification based on federated split learning according to claim 1, characterized in that: The process of model fusion assembly includes: (1) Upload the split model: After the training node is completed, it uploads its corresponding split model; (2) Split model query: Based on the claimed split model of each training node, query the split group to which the current split model belongs; (3) Split model fusion: the fusion process of the split models in the same group; (4) Assembling the split models: sorting and splicing the model split groups in order of computing power, arranging the model parameters in order, and synthesizing a new model.

4. The method for person re-identification based on federated split learning according to claim 1, characterized in that: In S5, a model regularization term is added to constrain the model optimization process. For the second sub-model W received by the training node s , and the local local model W ns , the constraints are: in Indicates the parameters that need to be federated in the model at time t, represents the model parameter that does not perform the splitting process at time t, λ1 and λ2 represent the weights of the two parts; The loss function is: Among them, L represents the loss of the original network.

5. The method for person re-identification based on federated split learning according to claim 1, characterized in that: S1 also includes the following: When the federated learning task initiator initiates the federated learning task, the training node obtains the specified federated learning task and initializes the federated learning environment; The training node performs data preprocessing on the pedestrian re-identification image data.

6. The method for person re-identification based on federated split learning according to claim 5, characterized in that: The contents of initializing the federated learning environment include: initializing the network architecture model, fusion model method, fusion iteration number, learning rate and optimizer.

7. The method for person re-identification based on federated split learning according to claim 5, characterized in that: Preprocessing includes data augmentation.

Images