A robust aggregation method and system for federated learning based on reinforcement learning
By adopting a geometric median dynamic robust aggregation method based on reinforcement learning in the federated learning system, dynamically adjusting the client node weights, the problems of data imbalance and Byzantine attacks in distributed scenarios are solved, and the robustness and accuracy of the model are improved.
Patent Information
- Application Number
- CN202410960826.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2025-08-12
- Estimated Expiration
- 2044-07-17
AI Technical Summary
When facing data imbalances of each participant and Byzantine attacks in distributed scenarios, it is difficult to effectively eliminate the biases and malicious attacks of individual nodes, resulting in a decline in global model performance.
A robust aggregation method based on reinforcement learning is adopted to enhance model robustness by calculating geometric median vectors and dynamically adjusting client node weights, combining deep deterministic policy gradient networks, identifying and weakening the impact of outliers, and enhancing the robustness of the model.
Effectively reduce the impact of abnormal nodes on the global model, improve the robustness and model accuracy of the federated learning system, and enhance the resistance to malicious attacks.
Smart Images
Figure CN119066605B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of federated learning technology, and in particular to a federated learning robustness aggregation method and system based on reinforcement learning. Background Art
[0002] Federated learning is a new paradigm for distributed machine learning, enabling multiple parties to collaborate and improve AI model performance without sharing local data. However, in distributed scenarios, the data scale and quality of each participant often vary, and the local model trained by each participant's node may deviate significantly from the final global model. Furthermore, in the actual application of federated learning, individual nodes may be limited by factors such as client status, network conditions, and physical equipment. Model parameters of participating nodes may be compromised during training or transmission, thus affecting the performance of the global model.
[0003] Furthermore, Byzantine attacks on distributed systems pose a significant security threat to federated learning systems. Malicious actors can manipulate the training process by directly modifying local model parameters sent from local devices to the coordination server, a phenomenon known as model poisoning. Alternatively, they can bias the local model by altering samples in the local training set. This can alter the data of individual nodes, resulting in poor training results for those nodes. Existing techniques typically aggregate nodes uniformly, making it difficult to eliminate the impact of poor individual nodes. Summary of the Invention
[0004] In view of this, an embodiment of the present invention provides a federated learning robustness aggregation method based on reinforcement learning to eliminate or improve one or more defects in the prior art.
[0005] One aspect of the present invention provides a federated learning robustness aggregation method based on reinforcement learning. The method is applied to a federated network, wherein the federated network includes a server node and a client node. The method includes the following steps:
[0006] The server node receives the model parameters uploaded by the client node after completing local training;
[0007] Constructing the model parameters uploaded by each client node into an initial parameter vector, constructing the current model parameters of the server node into a current round parameter vector, and calculating the geometric median vector based on the initial parameter vector, the current current round parameter vector, and the current weight value of the client node;
[0008] Calculate the distance between the current initial parameter vector of each client node and the geometric median vector, and construct it as a state vector to input into a preset reinforcement learning model. The reinforcement learning model outputs an updated weight value corresponding to each client node;
[0009] The model parameter vector is calculated based on the updated weight value output by each client node, the initial parameter vector and the current round parameter vector, where the values of each dimension in the model parameter vector are all model parameters.
[0010] Adopting the above scheme, this scheme first calculates the geometric median vector of all nodes through the model parameters uploaded by each node, outputs the updated weight value corresponding to each client node through the reinforcement learning model, and calculates the updated model parameters through the updated weight value. When a single node is attacked, there is a large error in the uploaded model parameters, and the distance between the model parameters and the geometric median vector is far, then the updated weight value will also become smaller relative to other client nodes, and the impact on the final calculation will also become smaller. Then, this scheme can eliminate the influence of poor individual nodes.
[0011] In some embodiments of the present invention, in the step of calculating the geometric median vector based on the initial parameter vector, the current round parameter vector and the current weight value of the client node,
[0012] Calculate the transition weight of each client node based on the initial parameter vector and weight value corresponding to each client node and the parameter vector of this round;
[0013] A geometric median vector is calculated based on the current round parameter vector and the transition weight of each client node.
[0014] In some embodiments of the present invention, in the step of calculating the transition weight of each client node based on the initial parameter vector and weight value corresponding to each client node and the current round parameter vector, the transition weight of the client node is calculated based on the following formula:
[0015]
[0016] in, represents the transition weight of client node k; α k represents the weight value of client node k; w represents the parameter vector of this round; w k Represents the initial parameter vector corresponding to client node k.
[0017] In some embodiments of the present invention, in the step of calculating the geometric median vector based on the current round parameter vector and the transition weight of each client node, the geometric median vector is calculated based on the following formula:
[0018]
[0019] Where GM represents the geometric median vector; represents the transition weight of client node k; w krepresents the initial parameter vector corresponding to client node k; K represents the total number of client nodes.
[0020] In some embodiments of the present invention, in the step of calculating the distance between the current initial parameter vector of each client node and the geometric median vector, the Euclidean distance between the current initial parameter vector of each client node and the geometric median vector is calculated.
[0021] In some embodiments of the present invention, in the step of calculating the model parameter vector based on the updated weight value output by each client node, the initial parameter vector and the current round parameter vector, the transition weight of each client node is recalculated based on the updated weight value output by each client node, the initial parameter vector and the current round parameter vector, and the geometric median vector is recalculated based on the current round parameter vector and the transition weight of each client node, and the recalculated geometric median vector is used as the model parameter vector.
[0022] In some embodiments of the present invention, the method further includes the server node feeding back the calculated model parameter vector to each client node.
[0023] In some embodiments of the present invention, the method further includes: treating each process in which the server node receives the model parameters uploaded by the client node after completing local training and calculates the model parameter vector as a federated training round, and recording the training data of multiple federated training rounds.
[0024] In some embodiments of the present invention, the steps of the method further include: using the state vectors of adjacent federated training rounds in the training data as the state of the previous round and the state of the next round, respectively; combining the updated weight values output by each client node in the training data as an action; using the accuracy of the model constructed by the model parameter vector obtained through each federated training round in the training data as a reward; training the reinforcement learning model and updating the reinforcement learning model.
[0025] The second aspect of the present invention also provides a federated learning robustness aggregation system based on reinforcement learning, which includes a computer device, wherein the computer device includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented by the method described above.
[0026] The third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps implemented by the aforementioned reinforcement learning-based federated learning robustness aggregation method.
[0027] Additional advantages, objects, and features of the present invention will be described in part in the following description and will become apparent to those skilled in the art after studying the following or may be learned by practice of the present invention. The objects and other advantages of the present invention may be particularly pointed out and attained in the description and drawings.
[0028] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.
[0030] Figure 1 Schematic diagram of a first embodiment of the federated learning robustness aggregation method based on reinforcement learning of the present invention;
[0031] Figure 2 Schematic diagram of a second embodiment of the federated learning robustness aggregation method based on reinforcement learning of the present invention;
[0032] Figure 3 Schematic diagram of a third embodiment of the federated learning robustness aggregation method based on reinforcement learning of the present invention;
[0033] Figure 4 FIG. 1 is a schematic diagram of the processing architecture of the present invention. DETAILED DESCRIPTION
[0034] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.
[0035] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.
[0036] Federated learning is an emerging distributed machine learning paradigm that enables multiple participants to train models without sharing local data. The Federated Averaging Algorithm (FedAvg) is a representative approach. Clients train local models using local data and then send these models to the server, which receives and averages the model parameters, completing a round of federated training. However, traditional Federated Averaging Aggregation Algorithms are less resilient to abnormal nodes and malicious attacks.
[0037] To address the problem of traditional federated average aggregation algorithms being weak against the aforementioned abnormal nodes and malicious attacks, existing technologies offer robust strategies that reduce the impact of outliers on the global model by aggregating the model parameters of each participating node. These strategies include distributed gradient descent algorithms based on the coordinate median and the coordinate clipped mean, as well as robust aggregation methods based on the geometric median. However, these robust aggregation calculation methods are typically completely independent in each round and do not affect each other. The calculation results are only related to the current model parameters of that round and do not take into account the historical process and parameters of the federation.
[0038] Introduction to existing technology:
[0039] Prior art 1
[0040] Mathematical methods such as coordinate clipping, median or geometric median are used to replace the traditional average aggregation strategy to reduce the impact of outliers on the global aggregation model from the data distribution level.
[0041] Disadvantages of the prior art 1
[0042] The existing technology only uses mathematical calculations to reduce the impact of outliers and improve robustness, and does not involve other parameters of the federated process.
[0043] Prior Art 2
[0044] Based on the different data quality of each client, the weight of each client is calculated, and the weighted geometric median of each client is calculated as the global model to improve the robustness of the model to abnormal nodes.
[0045] Disadvantages of the second prior art
[0046] In the second existing technology, each round of calculation runs independently, and its result depends entirely on the current model parameters of the client in that round of calculation, without taking into account the process parameters and historical results of federated learning.
[0047] This solution uses a model aggregation method based on geometric median to ensure initial robustness. It also integrates federated learning parameters and historical processes into a reinforcement learning framework. It uses federated historical data to train reinforcement learning to identify malicious clients and dynamically adjusts the geometric median calculation weights to reduce the impact of outliers.
[0048] This solution utilizes reinforcement learning strategies, comprehensively considering the parameters and processes of the federated system, to design a dynamic robust aggregation strategy. Reinforcement learning focuses on how intelligent agents interact with their environments and improve their actions by maximizing cumulative rewards. Specifically, agents learn how to take actions in specific tasks to maximize cumulative rewards through interaction with the federated learning system environment. Several elements, such as the agent's state and actions, can be represented in the federated learning system, enabling autonomous adjustments to the federated learning strategy in response to changes in the data environment.
[0049] like Figure 1 As shown, the present invention proposes a federated learning robustness aggregation method based on reinforcement learning. The method is applied to a federated network, which includes a server node and a client node. The method includes the following steps:
[0050] This method is applied to the server nodes of the federated network.
[0051] Step S100: The server node receives the model parameters uploaded by the client node after completing local training;
[0052] During the specific implementation process, each of the client nodes can perform local machine learning training on the local model through the data it receives, and the client node sends the model parameter items obtained through training to the server node.
[0053] Step S200: constructing the model parameters uploaded by each client node into an initial parameter vector, constructing the current model parameters of the server node into a current round parameter vector, and calculating a geometric median vector based on the initial parameter vector, the current current round parameter vector, and the current weight value of the client node;
[0054] In some embodiments of the present invention, in the step of constructing the model parameters uploaded by each client node into an initial parameter vector, the model parameters uploaded by the client node are used as the values of each dimension of the initial parameter vector; in the step of constructing the current model parameters of the server node into the current round parameter vector, the current model parameters of the server node are used as the values of each dimension of the current round parameter vector, and the current round parameter vectors of each client node are the same.
[0055] In a specific implementation process, the current weight value of the client node is the weight value of the client node obtained in the previous round of training or the weight value calculated based on the number of client nodes. If the weight value is calculated based on the number of client nodes, it is calculated based on the following formula:
[0056]
[0057] Among them, α k represents the weight value of client node k, and K represents the total number of client nodes.
[0058] In a specific implementation process, if the weight value is calculated based on the number of client nodes, then in the process of initially calculating the transition weight of the client nodes, the weight value of each client node is the same.
[0059] Step S300: Calculate the distance between the current initial parameter vector of each client node and the geometric median vector, construct the distance as a state vector, and input it into a preset reinforcement learning model. The reinforcement learning model outputs an updated weight value for each client node.
[0060] Reinforcement learning is a method of learning by interacting with the environment, adjusting its actions based on the reward signals it receives. Its goal is to discover a strategy that allows the agent to maximize its total reward, thereby achieving the optimal strategy in a given environment.
[0061] During the specific implementation process, this solution constructs a robust calculation strategy by setting weight values. Through the robust calculation strategy, the parameters of each federated client can be aggregated to reduce the impact of outliers on the global model.
[0062] Step S400 , calculating a model parameter vector based on the updated weight value output by each client node, the initial parameter vector and the current round parameter vector, wherein the values of each dimension in the model parameter vector are all model parameters.
[0063] During the specific implementation process, this solution calculates the updated weight value. When an abnormal client node appears, its influence on the global situation is weakened by controlling the weight of the client node.
[0064] Adopting the above scheme, this scheme first calculates the geometric median vector of all nodes through the model parameters uploaded by each node, outputs the updated weight value corresponding to each client node through the reinforcement learning model, and calculates the updated model parameters through the updated weight value. When a single node is attacked, there is a large error in the uploaded model parameters, and the distance between the model parameters and the geometric median vector is far, then the updated weight value will also become smaller relative to other client nodes, and the impact on the final calculation will also become smaller. Then, this scheme can eliminate the influence of poor individual nodes.
[0065] In some embodiments of the present invention, in the step of calculating the geometric median vector based on the initial parameter vector, the current round parameter vector and the current weight value of the client node,
[0066] Calculate the transition weight of each client node based on the initial parameter vector and weight value corresponding to each client node and the parameter vector of this round;
[0067] A geometric median vector is calculated based on the current round parameter vector and the transition weight of each client node.
[0068] In some embodiments of the present invention, in the step of calculating the transition weight of each client node based on the initial parameter vector and weight value corresponding to each client node and the current round parameter vector, the transition weight of the client node is calculated based on the following formula:
[0069]
[0070] in, represents the transition weight of client node k; α k represents the weight value of client node k; w represents the parameter vector of this round; w k Represents the initial parameter vector corresponding to client node k.
[0071] In some embodiments of the present invention, in the step of calculating the geometric median vector based on the current round parameter vector and the transition weight of each client node, the geometric median vector is calculated based on the following formula:
[0072]
[0073] Where GM represents the geometric median vector; represents the transition weight of client node k; w k represents the initial parameter vector corresponding to client node k; K represents the total number of client nodes.
[0074] In some embodiments of the present invention, in the step of calculating the distance between the current initial parameter vector of each client node and the geometric median vector, the Euclidean distance between the current initial parameter vector of each client node and the geometric median vector is calculated.
[0075] In some embodiments of the present invention, in the step of calculating the model parameter vector based on the updated weight value output by each client node, the initial parameter vector and the current round parameter vector, the transition weight of each client node is recalculated based on the updated weight value output by each client node, the initial parameter vector and the current round parameter vector, and the geometric median vector is recalculated based on the current round parameter vector and the transition weight of each client node, and the recalculated geometric median vector is used as the model parameter vector.
[0076] In the specific implementation process, based on the updated weight value output by each client node, the initial parameter vector and the current round parameter vector, the transition weight of the client node is recalculated according to the following formula:
[0077]
[0078] Recalculate the geometric median vector according to the following formula;
[0079]
[0080] The recalculated geometric median vector is used as the final model parameter vector.
[0081] The above scheme is adopted, in which the client nodes and server nodes are the main process of federated learning. There are malicious nodes using poisonous data among the client nodes, which are randomly selected to participate in training in each round. The server adopts geometric median aggregation of each client parameter, which can preliminarily enhance the ability to resist outliers. The reinforcement learning agent is used to adjust the weight of each client, aiming to reduce the weight of outlier nodes and further enhance the robustness of the system. The present invention adopts a deep deterministic policy gradient network (DDPG) as a reinforcement learning agent, which is designed to process continuous action spaces. The network runs within the actor-critic framework and uses deep neural networks to approximate the policy network and action-value function. This network structure uses strategy and value functions, combines the online network and the target network, greatly improving the stability of the algorithm and accelerating the convergence speed.
[0082] like Figure 2 As shown, in some embodiments of the present invention, the method further includes: step S500, the server node feeds back the calculated model parameter vector to each client node.
[0083] During the specific implementation process, the values of each dimension in the model parameter vector correspond to the parameters of the model. The server node will calculate the model parameter vector and feed it back to each client node. The client node will apply the values of each dimension in the model parameter vector to the local model.
[0084] In the specific implementation process, the server node uses the updated model with the final model parameter vector, evaluates it on the test dataset, and calculates the accuracy.
[0085] In some embodiments of the present invention, the method further includes: treating each process in which the server node receives the model parameters uploaded by the client node after completing local training and calculates the model parameter vector as a federated training round, and recording the training data of multiple federated training rounds.
[0086] like Figure 3 and 4 As shown, in some embodiments of the present invention, the steps of the method further include: step S600, using the state vectors of adjacent federal training rounds in the training data as the state of the previous round and the state of the next round respectively; combining the updated weight values output by each client node in the training data as an action; using the accuracy of the model constructed by the model parameter vector obtained through each federal training round in the training data as a reward; training the reinforcement learning model and updating the reinforcement learning model.
[0087] In the specific implementation process, reinforcement learning is a machine learning method based on Markov decision process.
[0088] Specifically, the training of the reinforcement learning agent is based on an experience replay mechanism, which maintains a replay buffer and samples training data from it. This buffer stores four-tuple data (state, action, reward, next state) sampled from the environment after each round of federated training. During the training phase, the reinforcement learning agent randomly extracts data from the replay buffer to train its network. Since the next state requires the completion of the next round of federated training, this solution stores the four-tuple data in the replay buffer. Once the size of the replay buffer exceeds the set batch size, the reinforcement learning agent will begin network training.
[0089] The beneficial effects of this program include:
[0090] 1. This proposal proposes a dynamic robust aggregation strategy based on the geometric median, using a deep deterministic policy gradient network as a reinforcement learning agent to adjust the median aggregation weights. This adaptive approach allows for more efficient integration of federated learning parameters and enhances the model's robustness against outliers.
[0091] 2. This approach considers federated parameters and historical processes to design a reinforcement learning strategy, including state space, action space, and reward signals. The goal is to actively improve the accuracy of the global model in each round of trials, thereby improving the robustness of the model while maintaining its overall performance.
[0092] 3. This solution uses the geometric median to reduce the impact of outliers on the global model. It also uses a reinforcement learning agent to adaptively adjust the weights of the federated learning median aggregation to identify malicious clients and enhance the robustness of the model.
[0093] 4. This solution comprehensively considers the process parameters and historical results of federated learning and designs a reinforcement learning strategy to improve the performance of the global model while maintaining system robustness.
[0094] An embodiment of the present invention also provides a federated learning robustness aggregation system based on reinforcement learning, which includes a computer device, wherein the computer device includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented by the method described above.
[0095] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps implemented by the aforementioned reinforcement learning-based federated learning robustness aggregation method. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the art.
[0096] It should be understood by those skilled in the art that the various exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.
[0097] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.
[0098] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.
[0099] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations to the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.
Claims
1. A robust aggregation method for federated learning based on reinforcement learning, characterized by: This method is applied to a federated network, wherein the federated network includes a server node and a client node. The method includes the following steps: The server node receives the model parameters uploaded by the client node after completing local training; Constructing the model parameters uploaded by each client node into an initial parameter vector, constructing the current model parameters of the server node into a current round parameter vector, calculating a geometric median vector based on the initial parameter vector, the current current round parameter vector, and the current weight value of the client node, calculating a transition weight of each client node based on the initial parameter vector and weight value corresponding to each client node and the current round parameter vector, and calculating a geometric median vector based on the current round parameter vector and the transition weight of each client node; The transition weight of the client node is calculated based on the following formula: in, represents the transition weight of client node k; α k represents the weight value of client node k; w represents the parameter vector of this round; w k represents the initial parameter vector corresponding to the client node k; The geometric median vector is calculated based on the following formula: Where GM represents the geometric median vector; represents the transition weight of client node k; w k represents the initial parameter vector corresponding to the client node k; K represents the total number of client nodes; Calculate the distance between the current initial parameter vector of each client node and the geometric median vector, and construct it as a state vector to input into a preset reinforcement learning model. The reinforcement learning model outputs an updated weight value corresponding to each client node; The model parameter vector is calculated based on the updated weight value output by each client node, the initial parameter vector and the current round parameter vector, where the values of each dimension in the model parameter vector are all model parameters.
2. The method for robust aggregation of federated learning based on reinforcement learning according to claim 1, characterized in that: In the step of calculating the distance between the current initial parameter vector of each client node and the geometric median vector, the Euclidean distance between the current initial parameter vector of each client node and the geometric median vector is calculated.
3. The method for robust aggregation of federated learning based on reinforcement learning according to claim 1, characterized in that: In the step of calculating the model parameter vector based on the updated weight value output by each client node, the initial parameter vector and the current round parameter vector, the transition weight of each client node is recalculated based on the updated weight value output by each client node, the initial parameter vector and the current round parameter vector, and the geometric median vector is recalculated based on the current round parameter vector and the transition weight of each client node, and the recalculated geometric median vector is used as the model parameter vector.
4. The method for robust aggregation of federated learning based on reinforcement learning according to claim 1, characterized in that: The method further includes the step of the server node feeding back the calculated model parameter vector to each client node.
5. The method for robust aggregation of federated learning based on reinforcement learning according to any one of claims 1 to 4, characterized in that: The method further includes the steps of: treating each process in which the server node receives the model parameters uploaded by the client node after completing local training and calculates the model parameter vector as a federated training round, and recording the training data of multiple federated training rounds.
6. The method for robust aggregation of federated learning based on reinforcement learning according to claim 5, characterized in that: The method further includes: using the state vectors of adjacent federated training rounds in the training data as the state of the previous round and the state of the next round, respectively; combining the updated weight values output by each client node in the training data as an action; using the accuracy of a model constructed using the model parameter vectors obtained through each federated training round in the training data as a reward; and training and updating the reinforcement learning model.
7. A federated learning robust aggregation system based on reinforcement learning, characterized by: The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions. The processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented by the method according to any one of claims 1 to 6.