Heterogeneous federal learning method and system based on historical model double knowledge distillation

Abstract

The invention relates to a heterogeneous federated learning method and system based on historical model double knowledge distillation, and belongs to the technical field of federated learning and knowledge distillation. The method comprises the steps that a server initializes a global model and constructs a historical global model storage list, and then the server generates a historical fusion model through weighted aggregation of multiple rounds of historical global models and sends related models to a client; after the client receives the global model and the historical fusion model of the round, local model training is carried out through a double knowledge distillation strategy based on a local private data set, the updated local model of the round is stored, global knowledge distillation takes the historical fusion model as a teacher model, and local knowledge distillation takes the historical local model of the client as a teacher model; global and local knowledge migration is balanced through hyper-parameters; the server averagely aggregates the model parameters updated by the client to generate and store a new round of global model; and iteratively optimizing to a convergent or preset training round number. According to the method, the performance of the model in a heterogeneous data environment is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning and knowledge distillation, and specifically relates to a heterogeneous federated learning method and system based on dual knowledge distillation of historical models. Background Art

[0002] Federated learning, a privacy-preserving distributed machine learning framework, enables multi-client collaborative modeling without sharing raw data. In recent years, it has been widely used in data-sensitive fields such as healthcare, finance, and smart devices. However, in real-world scenarios, client data often exhibits non-independent and identically distributed (IID) characteristics. For example, data across different user devices is highly heterogeneous due to differences in geography and behavioral habits. This data heterogeneity can lead to client model drift, global knowledge loss, and degraded model performance, severely hindering the performance and practical application of federated learning. Designing efficient and stable heterogeneous federated learning algorithms has become a research hotspot for both academia and industry.

[0003] Existing methods mainly alleviate data heterogeneity problems through federated distillation technology, such as using global model soft labels or knowledge transfer between clients to enhance model generalization. Although existing methods have proposed a variety of federated distillation methods to alleviate the challenges brought by data heterogeneity, there are still many limitations. First, in scenarios where data distribution differences are significant, existing methods lack effective mechanisms to deal with the problems of global knowledge forgetting and personalized knowledge retention, which leads to the gradual degradation of model performance over multiple rounds of training. Secondly, most methods rely only on the current global model or a single round of distillation process, ignoring the potential knowledge accumulation in historical global models and historical local models, and failing to fully utilize historical information to improve the stability and generalization ability of the model. In response to the above problems, the present invention proposes a heterogeneous federated learning method and system based on dual knowledge distillation of historical models. By combining the dual knowledge distillation mechanism of historical global models and historical local models, it synergistically optimizes global generalization and local personalized performance, breaking through the performance bottleneck of existing federated learning methods in data heterogeneous environments. Summary of the Invention

[0004] The purpose of the present invention is to provide a heterogeneous federated learning method and system based on dual knowledge distillation of historical models. This method and system effectively solve the problems of global generalized knowledge forgetting and local personalized knowledge degradation caused by non-independent and identically distributed data in federated learning, and significantly improve the performance of the model in a heterogeneous data environment.

[0005] To achieve the above objectives, the technical solution of the present invention is: a heterogeneous federated learning method based on dual knowledge distillation of historical models, comprising:

[0006] The server initializes the global model and builds a historical global model storage list. Then, the server generates a historical fusion model by weighted aggregation of multiple rounds of historical global models and sends the relevant model to the client.

[0007] After receiving the current global model and the historical fusion model, the client trains the local model based on the local private dataset using a dual knowledge distillation strategy, namely, collaborative optimization of global knowledge distillation and local knowledge distillation. The client also saves the updated local model. Global knowledge distillation uses the historical fusion model as the teacher model, while local knowledge distillation uses the client's historical local model as the teacher model. Hyperparameters are used to balance global and local knowledge transfer.

[0008] The server averages and aggregates the model parameters updated by the client to generate a new round of global model and stores it;

[0009] Iterate and optimize until convergence or a preset number of training rounds.

[0010] Furthermore, the server S initializes the global model and constructs a historical global model storage list. The specific steps are as follows:

[0011] Step A1: Server S randomly initializes the global model

[0012] Step A2: Server S builds a historical global model storage list Used to record the global model parameters of the first m rounds.

[0013] Furthermore, the server S generates a historical fusion model through weighted aggregation and sends the relevant model to the client C1, C1..., C n , the specific steps are as follows:

[0014] Step B1: Server S maintains the historical global model storage list Based on various historical models in the public dataset D public The performance on the model is calculated to obtain the model weight α, which is calculated as follows: Among them L j is the validation loss of the jth historical global model on the public dataset, and β is a hyperparameter that adjusts the smoothness of the weight distribution;

[0015] Step B2: Server S performs weighted aggregation based on the weights α calculated by each historical model to generate the historical fusion model of this round. The polymerization process is where w j represents the j-th historical global model;

[0016] Step B3: Server S sends the global model of this round and historical fusion models Sent to the clients C1, C1..., C participating in this round of federated learning training n .

[0017] Furthermore, client C k Receive the model from server S, combine the dual knowledge distillation mechanism to train and optimize the local model, and save the updated local model. The specific steps are as follows:

[0018] Step C1, Client C k Receive the current round global model from server S and historical fusion models And update the local model

[0019]

[0020] Step C2, Client C k Using local private dataset D k For local models Perform training and calculate cross entropy loss Optimize the model's ability to fit local data;

[0021] Step C3, Client C k With the historical local model w hist As a teacher model, it minimizes the KL divergence loss of all categories through local knowledge distillation, transfers local personalized knowledge, and optimizes the personalized performance of the local model. The objective function of local knowledge distillation is: where n k Represents client C k The number of training samples, KL represents the Kullback-Leibler divergence, and Represents the historical local model and the current local model for sample x i Generated soft label. Soft label q τ Depend on Calculated, where z(w,x) is the raw score logits output by the model, representing the model output that has not been normalized and contains rich model knowledge. w and x represent the model parameters and input data respectively, and temperature τ is a hyperparameter used to control the smoothness of the softmax output probability distribution.

[0022] Step C4, Client C k Historical fusion model As a teacher model, since the information of non-real categories in local data is often insufficient and easily forgotten during training, global knowledge distillation transfers global generalization knowledge by minimizing the KL divergence loss of non-real categories and optimizes the generalization performance of the local model. The objective function of global knowledge distillation is: and Represents the historical fusion model and the client local model for sample x i Generated soft labels. Indicates that global knowledge distillation is performed only between soft labels of non-target classes;

[0023] Step C5: Client C k Through the dual knowledge distillation strategy, global knowledge distillation and local knowledge distillation are combined to simultaneously distill global knowledge and personalized knowledge during local training, promoting comprehensive knowledge transfer. The overall objective function is: Among them, λ is a weight hyperparameter used to control the ratio between global knowledge distillation and local knowledge distillation, balance global and local knowledge transfer, and update the local model after this round of training is completed. Where η represents the learning rate, which controls the step size of each update and determines the magnitude of the weight adjustment along the gradient descent direction;

[0024] Step C6: Client C k Get the local model after this round of update After that, save As the historical local model w hist , and upload To server S.

[0025] Furthermore, the server S sends k The uploaded model parameters are aggregated to generate a new round of global models and saved to the historical global model storage list The specific steps are as follows:

[0026] Step D1: Server S receives data from each client C. k Uploaded local model parameters Average and aggregate the model parameters of each client to generate a new round of global model in is the total size of all client datasets, For client C k The size of the local dataset;

[0027] Step D2: Server S saves the new global model To the historical global model storage list , used for the next round of historical global model aggregation.

[0028] Furthermore, in the iterative optimization phase, the historical global model aggregation, i.e. steps B1-B3, client local training, i.e. steps C1-C6, and global model aggregation and storage, i.e. steps D1-D2, are repeatedly executed until the global model reaches the preset convergence condition or the preset number of training rounds is reached, and the server S outputs the final global model.

[0029] The present invention also provides a heterogeneous federated learning system based on dual knowledge distillation of historical models, including:

[0030] Server S is responsible for initializing the global model, weighted aggregation of historical models, and training process control. It initializes the global model and maintains a list of historical models. It calculates the weights of each historical model based on its performance on a public dataset, generates a fusion model, and distributes it to the client. After client training is complete, the server aggregates all local model parameters to generate a new generation of global model, and loops through the above process until the convergence conditions or the preset number of training rounds are met, ultimately outputting the optimized global model.

[0031] Client C1, C1..., C n , receives the global model and fusion model sent by server S, and performs training based on local private data: first, the model's fitting to local data is optimized through cross-entropy loss, and then global generalized knowledge and local personalized knowledge are transferred respectively using global knowledge distillation and local knowledge distillation; after training is completed, the client saves the updated model as the historical local model and uploads the parameters to the server to participate in global aggregation.

[0032] The present invention also provides a computer system, characterized in that it includes a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, it can implement the steps of any of the above-mentioned methods.

[0033] The present invention also provides a computer-readable storage medium on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the steps of any of the above methods can be implemented.

[0034] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any of the above methods.

[0035] Compared with the existing technology, the present invention has the following beneficial effects: the method and system of the present invention, which utilize the dynamic weighted aggregation of the historical global model and the personalized knowledge retention mechanism of the client's historical local model, through the dual knowledge distillation collaborative optimization of global knowledge distillation and local knowledge distillation, compared with the existing federated distillation method, the present invention effectively solves the problems of global generalized knowledge forgetting and local personalized knowledge degradation caused by non-independent and identically distributed data in federated learning, and significantly improves the performance of the model in heterogeneous data environments. The present invention can be directly deployed in a privacy-sensitive distributed computing environment, and realizes efficient knowledge transfer through a lightweight client-server architecture. Its dual distillation mechanism is compatible with a variety of deep learning models and supports low-computing power adaptation of edge devices, and has broad application prospects and commercial potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flowchart of a method implementation in an embodiment of the present invention.

[0037] Figure 2 It is a system implementation diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0039] This invention aims to address the challenges brought about by heterogeneous federated learning of data. It applies a dual knowledge distillation strategy to federated learning and uses historical global models and historical local models to respectively transfer global generalized knowledge and local personalized knowledge accumulated over multiple rounds. While effectively alleviating the problems of global knowledge forgetting and local personalized knowledge degradation caused by non-independent and identically distributed data, it significantly improves the performance of the model in a heterogeneous data environment.

[0040] Based on the above ideas, the present invention provides a heterogeneous federated learning method based on dual knowledge distillation of historical models, such as Figure 1 As shown, the server first initializes the global model and builds a historical global model storage list. Then, the server generates a historical fusion model by weighted aggregation of multiple rounds of historical global models and sends the relevant model to the client. After the client receives the global model and fusion model of this round, it trains the local model based on the local private data set through a dual knowledge distillation strategy (i.e., collaborative optimization of global knowledge distillation and local knowledge distillation), and saves the updated local model of this round. The global knowledge distillation uses the historical fusion model as the teacher model, and the local knowledge distillation uses the client's historical local model as the teacher model, and balances the global and local knowledge transfer through hyperparameters. Finally, the server averages the model parameters updated by the client to generate a new round of global model and stores it, and iteratively optimizes the above steps until convergence or the preset number of training rounds.

[0041] In this embodiment, the server S initializes the global model and builds a historical global model storage list. The specific steps of the initialization phase are as follows:

[0042] Step A1: Server S randomly initializes the global model

[0043] Step A2: Server S builds a historical global model storage list Used to record the global model parameters of the first m rounds.

[0044] In this embodiment, the server S generates a historical fusion model through weighted aggregation and sends the relevant model to the client C1, C1..., C n ,The specific steps of the historical global model aggregation stage are as follows:

[0045] Step B1: Server S maintains the historical global model storage list Based on various historical models in the public dataset D public The performance on the model is calculated to obtain the model weight α, which is calculated as follows: Among them L j is the validation loss of the j-th historical global model on the public dataset, and β is a hyperparameter that adjusts the smoothness of the weight distribution.

[0046] Step B2: Server S performs weighted aggregation based on the weights α calculated by each historical model to generate the historical fusion model of this round. The polymerization process is where w j represents the j-th historical global model;

[0047] Step B3: Server S sends the global model of this round and fusion model Sent to the clients C1, C1..., C participating in this round of federated learning training n .

[0048] In this embodiment, the client C k Receive the model from server S, combine the dual knowledge distillation mechanism to train and optimize the local model, and save the updated local model. The specific steps of the local model training phase are as follows:

[0049] Step C1, Client C k Receive the current round global model from server S and fusion model And update the local model

[0050] Step C2, Client C kUsing local private dataset D k For local models Perform training and calculate cross entropy loss Optimize the model's ability to fit local data;

[0051] Step C3, Client C k With the historical local model w hist As a teacher model, it minimizes the KL divergence loss of all categories through local knowledge distillation, transfers local personalized knowledge, and optimizes the personalized performance of the local model. The objective function of local knowledge distillation is: where n k Represents client C k The number of training samples, KL represents the Kullback-Leibler divergence, and Represents the historical local model and the current local model for sample x i Generated soft label. Soft label q τ Depend on Calculated, where z(w,x) is the raw score logits output by the model, representing the model output that has not been normalized and contains rich model knowledge. w and x represent the model parameters and input data respectively, and temperature τ is a hyperparameter used to control the smoothness of the softmax output probability distribution.

[0052] Step C4, Client C k Historical fusion model As a teacher model, since the information of non-real categories in local data is often insufficient and easily forgotten during training, global knowledge distillation transfers global generalization knowledge by minimizing the KL divergence loss of non-real categories and optimizes the generalization performance of the local model. The objective function of global knowledge distillation is: and Represents the historical fusion model and the client local model for sample x i Generated soft labels. Indicates that global knowledge distillation is performed only between soft labels of non-target classes;

[0053] Step C5: Client C k Through the dual knowledge distillation strategy, global knowledge distillation and local knowledge distillation are combined to simultaneously distill global knowledge and personalized knowledge during local training, promoting comprehensive knowledge transfer. The overall objective function is: Among them, λ is a weight hyperparameter used to control the ratio between global knowledge distillation and local knowledge distillation, balance global and local knowledge transfer, and update the local model after this round of training is completed. Where η represents the learning rate, which controls the step size of each update and determines the magnitude of the weight adjustment along the gradient descent direction;

[0054] Step C6: Client C k Get the local model after this round of update After that, save As the historical local model w hist , and upload To server S.

[0055] In this embodiment, the server S sends a k The uploaded model parameters are aggregated to generate a new round of global models and saved to the historical global model storage list The specific steps of the global model aggregation and storage phase are as follows:

[0056] Step D1: Server S receives data from each client C. k Uploaded local model parameters Average and aggregate the model parameters of each client to generate a new round of global model in is the total size of all client datasets, For client C k The size of the local dataset.

[0057] Step D2: Server S saves the new global model To the historical global model storage list , used for the next round of historical global model aggregation.

[0058] In this embodiment, the historical global model aggregation (steps B1-B3), client local training (steps C1-C6) and global model aggregation and storage (steps D1-D2) processes are iteratively executed until the global model reaches the preset convergence condition or reaches the preset number of training rounds, and the server S outputs the final global model.

[0059] In order to verify the effectiveness of the embodiments of the present invention, the inventors conducted a variety of heterogeneous partitioning experiments on multiple image classification datasets such as MNIST, CIFAR-10, and CIFAR-100, and compared them with related work, as shown in Table 1. The details of the experimental setting are as follows: the experiment adopts a federated learning environment with n = 100 clients, the number of communication rounds is set to T = 200, and 10% of the clients are randomly sampled from them to participate in training in each round. During the local training process, each selected client performs E = 5 rounds of local updates, and the batch size is set to B = 64. All experiments are trained using the SGD optimizer, with a learning rate of 0.01, a momentum factor of 0.90, a learning rate decay coefficient of 0.99, and a weight decay coefficient of 1×10 -5 To simulate the non-IID characteristics of federated learning in real-world scenarios, the experiment used the Dirichlet distribution Dir(α) for data partitioning to control the degree of data heterogeneity between different clients. CIFAR-10 used α∈{0.1, 0.5, 1.0} for data partitioning, while the other datasets used α=0.1. The weight hyperparameter λ∈{0.6, 0.7, 0.8}, the temperature parameter τ∈{1, 3}, and the historical model list size m∈1, 3, 5, 7, 9} were used. Experimental results show that compared with the existing technology, the present invention exhibits superior global model accuracy and more stable convergence under different data sets and data heterogeneity conditions. In particular, in the α=0.1 scenario of the CIFAR-10 dataset, it improves by 4.59%, 2.55%, 2.90% and 0.64% respectively compared with other related works, demonstrating its effectiveness and robustness in non-independent and identically distributed scenarios, further verifying the advantages of this method in data heterogeneous federated learning environments, and effectively alleviating data heterogeneity without leaking privacy.

[0060] Table 1 Global model accuracy of different works on MNIST, CIFAR-10 and CIFAR-100 datasets

[0061]

[0062] The present invention also provides a data heterogeneous federated learning system for implementing the above method, such as Figure 2 As shown, it includes a central server S, multiple clients C1, C1..., C n .

[0063] Server S is responsible for initializing the global model, weighting and aggregating historical models, and controlling the training process. It initializes the global model and maintains a list of historical models. It calculates weights based on the performance of each historical model on a common dataset, generates a fused model, and distributes it to the client. After client training is complete, the server aggregates all local model parameters to generate a new global model. The above process is repeated until convergence conditions are met or the preset number of training rounds are reached, ultimately outputting the optimized global model.

[0064] Client C1, C1..., C n The client receives the global model and fusion model from server S and performs training based on local private data. First, the model's fit to the local data is optimized using cross-entropy loss. Global generalized knowledge and local personalized knowledge are then transferred using global knowledge distillation and local knowledge distillation, respectively. After training is complete, the client saves the updated model as the historical local model and uploads its parameters to the server for global aggregation.

[0065] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions and effects do not exceed the scope of the technical solution of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A heterogeneous federated learning method based on dual knowledge distillation of historical models, characterized by: include: The server initializes the global model and builds a historical global model storage list. Then, the server generates a historical fusion model by weighted aggregation of multiple rounds of historical global models and sends the relevant model to the client. After receiving the current global model and the historical fusion model, the client trains the local model based on the local private dataset using a dual knowledge distillation strategy, namely, collaborative optimization of global knowledge distillation and local knowledge distillation. The client also saves the updated local model. Global knowledge distillation uses the historical fusion model as the teacher model, while local knowledge distillation uses the client's historical local model as the teacher model. Hyperparameters are used to balance global and local knowledge transfer. The server averages and aggregates the model parameters updated by the client to generate a new round of global model and stores it; Iterate and optimize until convergence or a preset number of training rounds.

2. The heterogeneous federated learning method based on historical model dual knowledge distillation according to claim 1 is characterized in that: Server S initializes the global model and builds a historical global model storage list. The specific steps are as follows: Step A1: Server S randomly initializes the global model Step A2: Server S builds a historical global model storage list Used to record the global model parameters of the first m rounds.

3. The heterogeneous federated learning method based on historical model dual knowledge distillation according to claim 2 is characterized in that: The server S generates a historical fusion model through weighted aggregation and sends the relevant model to the client C1, C1..., C n , the specific steps are as follows: Step B1: Server S maintains the historical global model storage list Based on various historical models in the public dataset D public The performance on the model is calculated to obtain the model weight α, which is calculated as follows: Among them L j is the validation loss of the jth historical global model on the public dataset, and β is a hyperparameter that adjusts the smoothness of the weight distribution; Step B2: Server S performs weighted aggregation based on the weights α calculated by each historical model to generate the historical fusion model of this round. The polymerization process is where w j represents the j-th historical global model; Step B3: Server S sends the global model of this round and historical fusion models Sent to the clients C1, C1..., C participating in this round of federated learning training n .

4. The heterogeneous federated learning method based on historical model dual knowledge distillation according to claim 3 is characterized in that: Client C k Receive the model from server S, combine the dual knowledge distillation mechanism to train and optimize the local model, and save the updated local model. The specific steps are as follows: Step C1, Client C k Receive the current round global model from server S and historical fusion models And update the local model Step C2, Client C k Using local private dataset D k For local models Perform training and calculate cross entropy loss Optimize the model's ability to fit local data; Step C3, Client C k With the historical local model w hist As a teacher model, it minimizes the KL divergence loss of all categories through local knowledge distillation, transfers local personalized knowledge, and optimizes the personalized performance of the local model. The objective function of local knowledge distillation is: where n k Represents client C k The number of training samples, KL represents the Kullback-Leibler divergence, and Represents the historical local model and the current local model for sample x i Generated soft label, soft label q τ Depend on Calculated, where z(w,x) is the raw score logits output by the model, representing the model output that has not been normalized and contains rich model knowledge. w and x represent the model parameters and input data respectively, and temperature τ is a hyperparameter used to control the smoothness of the softmax output probability distribution. Step C4, Client C k Historical fusion model As a teacher model, since the information of non-real categories in local data is often insufficient and easily forgotten during training, global knowledge distillation transfers global generalization knowledge by minimizing the KL divergence loss of non-real categories and optimizes the generalization performance of the local model. The objective function of global knowledge distillation is: and Represents the historical fusion model and the client local model for sample x i Generated soft labels, Indicates that global knowledge distillation is performed only between soft labels of non-target classes; Step C5: Client C k Through the dual knowledge distillation strategy, global knowledge distillation and local knowledge distillation are combined to simultaneously distill global knowledge and personalized knowledge during local training, promoting comprehensive knowledge transfer. The overall objective function is: Among them, λ is a weight hyperparameter used to control the ratio between global knowledge distillation and local knowledge distillation, balance global and local knowledge transfer, and update the local model after this round of training is completed. Where η represents the learning rate, which controls the step size of each update and determines the magnitude of the weight adjustment along the gradient descent direction; Step C6: Client C k Get the local model after this round of update After that, save As the historical local model w hist , and upload To server S.

5. The heterogeneous federated learning method based on historical model dual knowledge distillation according to claim 4 is characterized in that: Server S to each client C k The uploaded model parameters are aggregated to generate a new round of global models and saved to the historical global model storage list The specific steps are as follows: Step D1: Server S receives data from each client C. k Uploaded local model parameters Average and aggregate the model parameters of each client to generate a new round of global model in is the total size of all client datasets, For client C k The size of the local dataset; Step D2: Server S saves the new global model To the historical global model storage list , used for the next round of historical global model aggregation.

6. The heterogeneous federated learning method based on historical model dual knowledge distillation according to claim 5 is characterized in that: In the iterative optimization phase, the historical global model aggregation (steps B1-B3), client local training (steps C1-C6), and global model aggregation and storage (steps D1-D2) are repeatedly executed until the global model reaches the preset convergence condition or the preset number of training rounds is reached, and the server S outputs the final global model.

7. A heterogeneous federated learning system based on dual knowledge distillation of historical models, characterized by: include: Server S is responsible for initializing the global model, weighted aggregation of historical models, and training process control; It initializes the global model and maintains a list of historical models. It calculates the weights of each historical model based on its performance on a public dataset, generates a fusion model, and distributes it to the client. After the client training is complete, the server aggregates all local model parameters to generate a new generation of global model. The above process is repeated until the convergence condition is met or the preset number of training rounds is reached, and finally the optimized global model is output. Client C1, C1..., C n , receiving the global model and fusion model sent by server S, and performing training based on local private data: first, the model’s fitting to local data is optimized through cross entropy loss, and then global knowledge distillation and local knowledge distillation are used to transfer global generalized knowledge and local personalized knowledge respectively; After training is completed, the client saves the updated model as a historical local model and uploads the parameters to the server to participate in global aggregation.

8. A computer system, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the steps of the method according to any one of claims 1 to 6 can be implemented.

9. A computer-readable storage medium storing computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, the steps of the method according to any one of claims 1 to 6 can be implemented.

10. An electronic device comprising a processor and a memory, wherein: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 6.

Images