Multi-view domain-based multi-modal heterogeneous data fusion method in federated learning

Abstract

The invention relates to a multi-view domain-based multi-modal heterogeneous data fusion method in federated learning, and belongs to the field of multi-modal data fusion. The method comprises the following steps: firstly, designing a multi-mode heterogeneous data fusion model based on a multi-view domain and designing a local decision boundary adaptive adjustment strategy at a cloud end; secondly, learning the multi-modal fusion model based on an adaptive loss strategy to obtain a local model with local features; and finally, aggregating the local model at the cloud in an average aggregation mode. According to the method, distributed training of the shared model is realized on the premise of protecting user privacy, and the problem of complex domain drift and long tail caused by coupled modals and category isomerism in a heterogeneous multi-modal federated learning scene is solved. Model divergence caused by complex data isomerism in a multi-modal federation learning scene is effectively relieved, the reliability of the model on different view domains is enhanced, and meanwhile the generalization ability of all view domains is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of multi-modal data fusion, and particularly relates to a multi-modal heterogeneous data fusion method based on multi-view domains in federated learning. BACKGROUND

[0002] With the popularity of ubiquitous sensing, ubiquitous interconnection and ubiquitous intelligent connection, and the coverage of the next generation wireless network in the global scope, the explosive growth of intelligent networking devices provides wireless networks with more massive and more modal sensing data. However, since the sensing information may contain user privacy information such as behavior preference, geographic location, etc., directly collecting all the multi-modal data sensed on the devices into the cloud server for fusion and analysis seriously violates the principle of privacy protection.

[0003] Federated learning is a kind of distributed machine learning technology applied to decentralized edge devices (such as industrial devices, mobile phones, etc.). In federated learning, edge devices train local models with local features using local data sets, and upload the trained local models to the cloud server for aggregation to update the parameters of the global model, so as to obtain a global shared model with global features. This technology effectively avoids the risk of data leakage caused by the transmission of user data. Therefore, the research on multi-modal fusion based on federated learning has attracted more and more attention. Most of the existing works focus on integrating multi-modal complementary information by using traditional centralized multi-modal fusion operations to improve model performance. However, these methods do not consider the modal-based data heterogeneity in the multi-modal federated learning scenario. Although some federated learning works on data heterogeneity have considered the long-tail class distribution or domain shift problem between devices, most of them do not consider the coupled modal-based and class-based data heterogeneity problem in the multi-modal federated learning scenario. In fact, in the multi-modal federated learning scenario, due to the introduction of modal-based data heterogeneity, the long-tail distribution and domain shift problem become more complex, resulting in more serious model divergence.

[0004] Specifically, domain shift (same label, different features) is ubiquitous in real-world federated learning applications. For example, cameras and scanners on different devices cause domain shift within the same image modality due to the adoption of different imaging protocols. However, in the heterogeneous multi-modal federated learning scenario, domain shift not only exists within the same modality, but also exists between different modalities. In addition, due to fixed geographical location or behavioral preference, long-tail distribution is difficult to avoid, which not only makes the data between nodes present non-identically distributed characteristics, but also further causes the global data to present long-tail distribution. Existing researches focus on the class imbalance problem, but ignore the modality imbalance phenomenon that may occur in the multi-modal federated learning scenario. In fact, different data modalities provide different complementary information, which may lead to long-tail distribution of samples even within a single class. Therefore, in the multi-modal federated learning scenario, to effectively improve the model performance, how to solve the complex challenges caused by the coupling of modal and class data heterogeneity has extremely high research value. SUMMARY

[0005] The purpose of the present application is to provide a multi-modal heterogeneous data fusion method based on multi-view domain in federated learning, which realizes multi-user information sharing under the premise of protecting user privacy, solves the problem of multi-modal information fusion of heterogeneous nodes in federated learning, and improves the reliability and generalization of the model.

[0006] To achieve the above purpose, the technical scheme of the present application is: a multi-modal heterogeneous data fusion method based on multi-view domain in federated learning, comprising:

[0007] designing a multi-modal heterogeneous data fusion model based on multi-view domain and designing a local decision boundary adaptive adjustment strategy in the cloud;

[0008] learning the multi-modal heterogeneous data fusion model based on the local decision boundary adaptive adjustment strategy to obtain a local model with local characteristics;

[0009] aggregating the local model by averaging in the cloud.

[0010] In an embodiment of the present application, the method specifically comprises the following steps:

[0011] Step S1, designing a multi-modal heterogeneous data fusion model based on multi-view domain in the cloud;

[0012] Step S2, designing a local decision boundary adaptive adjustment strategy;

[0013] Step S3, initializing the multi-modal heterogeneous data fusion model based on multi-view domain in the cloud, and distributing the initialized global model to the edge nodes participating in training;

[0014] Step S4, the edge node participating in the training utilizes the local data set and learns the multi-modal heterogeneous data fusion model designed in step S1 according to the local decision boundary adaptive adjustment strategy designed in step S2 to obtain a local model with local characteristics;

[0015] Step S5, the local models are aggregated in a cloud end through an average aggregation manner.

[0016] In an embodiment of the present application, in step S1, the multi-modal heterogeneous data fusion model of the multi-view domain comprises a feature extraction module a feature fusion module and a feature prediction module

[0017] In an embodiment of the present application, the specific implementation process of step S1 is as follows:

[0018] Step S11, assuming that the outputs of M modal feature sub-networks and the fusion tensor are respectively and wherein the output of the feature fusion module is represented as:

[0019]

[0020] wherein, the jth element of the output is represented as:

[0021]

[0022] Step S12, a feature fusion module Φ(·) of the multi-view domain is constructed; any modality or modality combination is referred to as a view domain; in the constructed multi-view domain, the joint feature of the ith modality and the jth modality is:

[0023]

[0024] wherein, the core tensor is the factor matrix set is the dimension of each feature extraction is I, and the mapping dimension corresponding to the s th modality is r s ;

[0025] is represented as using R matrices and is represented as wherein * represents element-wise multiplication;

[0026] Therefore, the output of the feature fusion module of the multi-view domain is represented as:

[0027]

[0028] wherein, Intrinsic information and complementary information i.e.

[0029] Step S13, integrating the information of different modal views by weighted sum of the prediction results of the multi-view domain, wherein is a feature prediction module:

[0030]

[0031] In an embodiment of the present application, the specific implementation process of step S2 is as follows:

[0032] Step S21, the softmax loss function based on the cosine distance is as follows:

[0033]

[0034] wherein C is the total number of classes, s is a scaling factor for controlling the scale of loss, θ i is the angle between the feature vector and the true value, θ j is the angle between the feature vector and the weight vector of the jth class in the classifier.

[0035] Step S22, based on the loss function of step S21, a class-aware adjustment function M(·) is added to increase the inter-class distance and enhance the intra-class compactness by adjusting the decision boundary between positive samples and negative samples; at this time, the local adaptive loss function is as follows:

[0036]

[0037] wherein,

[0038]

[0039] wherein n i and n j are the sample numbers of class i and class j on the edge node respectively, and α and ε are hyperparameters for preventing the denominator from being zero and controlling the adjustment strength of the decision boundary respectively.

[0040] In an embodiment of the present application, in step S3, after receiving the connection request of the edge node in the cloud, the initialized global model is distributed to the edge nodes participating in the training.

[0041] In an embodiment of the present application, in step S4, the edge node is a heterogeneous edge node, and the heterogeneous edge node is equipped with different types / quantities of sensors.

[0042] In an embodiment of the present application, the specific implementation process of step S5 is as follows:

[0043] The edge node sends the local model to the cloud after the training is completed, and the cloud aggregates the local models by averaging aggregation after receiving the local models:

[0044]

[0045] wherein, is the local model on the kth edge node in the t+1th round of communication, n k Corresponding to the local data on the kth edge node, K is the total number of edge nodes, n is the global data, and after the aggregation is completed, the updated global model is returned to the edge node uploading the local model for a new round of training.

[0046] The application also provides a multi-modal heterogeneous data fusion system based on multi-view domains in federated learning, which comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor, and when the processor executes the computer program instructions, the method steps as described above can be realized.

[0047] The application also provides a computer readable storage medium having computer program instructions capable of being executed by a processor stored thereon, and when the processor executes the computer program instructions, the method steps as described above can be realized.

[0048] Compared with the prior art, the application has the following beneficial effects: the application uses tensor decomposition theory to propose a multi-modal heterogeneous data fusion method based on multi-view domains in federated learning, and realizes full utilization of multi-modal complementary information collected by heterogeneous devices. Secondly, for the model divergence caused by more complex domain drift in the multi-modal federated learning scene, an effective global decision boundary alignment scheme is proposed. Finally, the application proposes a local decision boundary adaptive adjustment strategy, which adjusts the decision boundary of the local category based on the global decision boundary according to the local data distribution, reduces the suppression effect of the head category on the tail category, and at the same time, relieves the model divergence caused by the long-tail distribution between heterogeneous devices. This method effectively improves the performance of the model and improves the generalization ability of the model in any view domain (including the view domain appearing in the training set and the view domain not appearing in the training set). BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 It is an embodiment of the application a federated learning multi-modal heterogeneous data fusion framework based on multi-view domains. DETAILED DESCRIPTION

[0050] The technical solutions of the application will be specifically described below with reference to the drawings.

[0051] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0052] It is also important to note that the terms "or" and "and" as used herein, unless otherwise indicated, are used to mean either phonetic "or", that is, any or all possible combinations of one or more items, or they are used to mean phonetic "and" that is, one and only one of an item. As used herein, the following terms have the following meanings.

[0053] The application provides a multi-modal heterogeneous data fusion method based on multi-view domains in federated learning, comprising:

[0054] A multi-modal heterogeneous data fusion model based on multi-view domains is designed in the cloud, and a local decision boundary adaptive adjustment strategy is designed.

[0055] Based on the local decision boundary adaptive adjustment strategy, the multi-modal heterogeneous data fusion model is learned to obtain a local model with local characteristics.

[0056] The local model is aggregated in the cloud by an average aggregation method.

[0057] The following is the specific implementation process of the application.

[0058] The application provides a multi-modal heterogeneous data fusion method based on multi-view domains in federated learning, comprising the following steps:

[0059] Step S1, a multi-modal heterogeneous data fusion model based on multi-view domains is designed in the cloud.

[0060] Step S2, a local decision boundary adaptive adjustment strategy is designed.

[0061] Step S3, the multi-modal heterogeneous data fusion model based on multi-view domains is initialized in the cloud, and the initialized global model is distributed to the edge nodes participating in training.

[0062] Step S4, the edge nodes participating in training utilize local data sets, and learn the multi-modal fusion model designed in step S1 according to the adaptive loss strategy designed in step S2 to obtain a local model with local characteristics.

[0063] Step S5, the local model is aggregated in the cloud by an average aggregation method.

[0064] Further, please refer to Figure 1The specific implementation process of step S1 is as follows:

[0065] A multi-modal heterogeneous data fusion model of a multi-view domain is constructed, which includes a feature extraction module a feature fusion module and a feature prediction module Since the present application focuses on solving the problems of domain drift and model divergence in the process of collaborative learning of multiple edge devices (which may be equipped with different types / quantities of sensors) using multi-modal data, the information on each heterogeneous node is fully utilized. Therefore, the design of the feature extraction module and the feature prediction module is not considered in the present patent.

[0066] Step S11, assuming that the outputs of M modal feature sub-networks and the fusion tensor are respectively and wherein The output of the multi-linear feature fusion module can be represented as:

[0067]

[0068] wherein the jth and element of the output can be represented as:

[0069]

[0070] Step S12, constructing a fusion module Φ(·) of a multi-view domain. We refer to any modality or combination of modalities as a view domain. In the constructed multi-view domain, the joint features of the ith modality and the jth modality are:

[0071]

[0072] wherein the core tensor is the factor matrix set is the dimension of each feature extraction is I, and the mapping dimension corresponding to the s th modality is r s Secondly, the R matrices and are used to represent as wherein * represents element-wise multiplication.

[0073] Therefore, the output of the multi-view domain fusion can be represented as:

[0074]

[0075] wherein contains inherent information and complementary information that is

[0076] Step S13, integrating information of different modal views by weighted sum of prediction results of multi-view domains, wherein P(·) is a feature prediction module:

[0077]

[0078] The present scheme adopts two data sets UTD-MHAD and FLASH to verify the present application. Since the modalities supported by UTD-MHAD and FLASH data sets are different, different sub-modules are adopted to constitute the feature extraction module. For UTD-MHAD data set, the feature extraction module includes a depth extraction sub-module (convolution layer 5x5x64, pooling layer 3x3, convolution layer 5x5x64, convolution layer 5x5x32, convolution layer 5x5x16, full connection layer 800x100), an inertial extraction sub-module (LSTM, full connection layer 100x100) and a skeleton extraction sub-module (convolution layer 5x5x2x64, convolution layer 5x5x2x64, convolution layer 5x5x1x32, convolution layer 5x5x1x16, full connection layer 1728x100); the feature prediction module is set to a connection layer 100x27. For FLASH data set, the feature extraction module includes a laser radar extraction sub-module (convolution layer 3x3x32, convolution layer 3x3x64, convolution layer 3x3x128, convolution layer 3x3x32, pooling layer 2x3, pooling layer 1x2, full connection layer 160x100), an image extraction sub-module (resnet18 network) and a GPS sub-module (full connection layer 2x100); the feature prediction module is set to a connection layer 100x64. The rank R of the feature fusion module is set to 5.

[0079] Further, the specific implementation process of the step S2 is as follows:

[0080] Step S21, the softmax loss function based on cosine distance is:

[0081]

[0082] Wherein, C is the total number of categories, s is a scaling factor for controlling the scale of loss, θ i is the angle between the feature vector and the true value, θ j is the angle between the feature vector and the weight vector of the jth category in the classifier.

[0083] Step S22, based on the above loss, a class-aware adjustment function M(·) is added to increase the decision boundary between positive samples and negative samples, increase the inter-class distance and enhance the intra-class compactness. At this time, the local adaptive loss function is:

[0084]

[0085] Wherein,

[0086]

[0087] where n i and n j are the number of samples of class i and class j on the node respectively, and α and ε are hyperparameters to prevent the denominator from being zero and control the strength of decision boundary adjustment respectively. In this scheme, s = 1, ε = 1e-7, and α = 0.5.

[0088] Further, in the step S3, after the cloud receives the connection request of the edge node, the global model of the above step 1 is initialized and sent to the edge node.

[0089] Further, in the step S4, after the global model is received by the heterogeneous edge nodes participating in training (which may be equipped with different types / quantities of sensors), the local model with local data characteristics is obtained by learning the global model using the local training set according to the adaptive loss strategy designed in step 2. At this time, we simulate the data heterogeneous scene in multi-modal federated learning from the aspects of class and modality. Since the FLASH dataset is a natural class-heterogeneous dataset, this scheme only simulates the class-based data heterogeneity of the UTD-MAHD dataset. Specifically, this scheme generates N pieces of non-iid data using Dir(β) to simulate the local dataset on the edge device, where the smaller β indicates the greater difference in the distribution of the dataset on each device. In this scheme, β = 0.1. Secondly, we use Θ = {α m | m ∈ M} to control the proportion of devices with m modalities, where

[0090] Further, in the step S5, after the local model is trained, the edge node sends the local model to the cloud, and the cloud aggregates the local models by averaging aggregation.

[0091]

[0092] where, is the model on the kth node in the t+1th round of communication, n k corresponding to the local data on the kth node, and n is the global data. After the aggregation is completed, the updated global model is returned to the edge node that uploaded the local model for a new round of training.

[0093] The common multi-classification indexes accuracy, precision and F1-score are used to evaluate the performance of the application. Specifically, in order to evaluate the generalization ability of the application in any view domain (including the view domain appearing in the training set and the view domain not appearing in the training set), the average value of each index of the test in all view domains is measured, and is respectively denoted as mAcc, mPrec and mF1:

[0094]

[0095] Wherein, M is the total global modal category number, and Omega (M, m) is a set of m modalities sampled from M modalities. Therefore, these indexes can not only reflect the classification performance of the model, but also reflect the generalization ability and reliability of the model in all view domains (including the view domain appearing in the training set and the view domain not appearing in the training set).

[0096] The application utilizes the tensor decomposition theory, proposes a multi-view domain based multi-modal heterogeneous data fusion method in federated learning, and realizes full utilization of the multi-modal complementary information collected by the heterogeneous devices. Secondly, in view of the model divergence caused by more complex domain drift in the multi-modal federated learning scene, an effective global decision boundary alignment scheme is proposed. Finally, the application proposes a local decision boundary adaptive adjustment strategy, which adjusts the decision boundary of the local category based on the global decision boundary according to the local data distribution, reduces the suppression effect of the head category on the tail category, and at the same time, relieves the model divergence caused by the long-tail distribution among the heterogeneous devices. This method effectively improves the performance of the model, and at the same time, improves the generalization ability of the model in any view domain (including the view domain appearing in the training set and the view domain not appearing in the training set).

[0097] The application also provides a multi-view domain based multi-modal heterogeneous data fusion system in federated learning, which comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor, and when the processor executes the computer program instructions, the method steps as described above can be realized.

[0098] The application also provides a computer readable storage medium, which stores computer program instructions capable of being executed by a processor, and when the processor executes the computer program instructions, the method steps as described above can be realized.

[0099] Those skilled in the art will appreciate that embodiments of the application can be readily used as software, hardware, or a combination of software and hardware. In one

[0100] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 an apparatus to perform functions specified in the flowchart block or blocks.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 an apparatus to perform functions specified in the flowchart block or blocks.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 an apparatus to perform functions specified in the flowchart block or blocks.

[0103] The above descriptions are only preferred embodiments of the present application, and are not intended to limit the present application to other forms. Any person skilled in the art can make modifications or alterations to the above-described embodiments without departing from the technical scope of the present application. Any simple modifications, equivalent changes and alterations made to the above-described embodiments based on the technical essence of the present application shall fall within the scope of protection of the present application.

Claims

1. A multi-modal heterogeneous data fusion method based on multi-view domain in federated learning, characterized in that, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

2. The method of claim 1, wherein the method is based on multi-view domain in federated learning. The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

3. The method of claim 2, wherein the method is based on multi-view domain in federated learning. In step S1, the multi-modal heterogeneous data fusion model of the multi-view domain includes a feature extraction module a feature fusion module and a feature prediction module 4. The method of claim 3, wherein the method is based on multi-view domain in federated learning. The method comprises the following steps: Step S11, assuming the outputs of the M modal feature sub-networks and the fusion tensor are respectively and wherein the output of the feature fusion module is represented as: The method comprises the following steps: The method comprises the following steps: Wherein, the core tensor is The factor matrix set is The dimension of each feature extraction is I, and the mapping dimension corresponding to the s th mode is r s ; Utilizing R matrices and Let be represented as where * denotes element-wise multiplication; The method comprises the following steps: wherein, contains intrinsic information and complementary information i.e. Step S13, integrating information of different modal views by weighted sum of prediction results of multi-view domains, wherein is a feature prediction module:

5. The method of claim 2, wherein the method is based on multi-view domain in federated learning. The method comprises the following steps: The method comprises the following steps: where C is the total number of classes, s is a scaling factor used to control the scale of the loss, and θ i is the angle between the feature vector and the true value, and θ j is the angle between the feature vector and the weight vector of the jth class in the classifier. The method comprises the following steps: The method comprises the following steps: where n i and n j are the number of samples of class i and class j on the edge node, respectively, and a and e are hyperparameters that prevent the denominator from being zero and control the strength of decision boundary adjustment, respectively.

6. The method of claim 2, wherein the method is based on multi-view domain in federated learning. The method comprises the following steps:

7. The method of claim 2, wherein the method is based on multi-view domain in federated learning. The method comprises the following steps:

8. The method of claim 2, wherein the method is based on multi-view domain in federated learning. The method comprises the following steps: The method comprises the following steps: wherein, is the local model on the kth edge node in the t+1th round of communication, n k corresponding to the local data on the kth edge node, K is the total number of edge nodes, n is the global data, and after the aggregation is completed, the updated global model is returned to the edge node that uploaded the local model for a new round of training.

9. A multi-modal heterogeneous data fusion system based on multi-view domains in federated learning, characterized in that, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the

Images