A model training method and task execution method based on feature distribution

By adopting a model training method based on feature distribution in federated learning, updating client model parameters, the model applicability problem caused by the differences in data distribution of different clients is solved, and the generalization ability and accuracy of the model are improved.

CN118691933BActive Publication Date: 2024-11-15ZHEJIANG LAB
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411168955.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-11-15
Estimated Expiration
2044-08-23

AI Technical Summary

Technical Problem

During the federated learning process, the data types of different clients are distributed differently, which makes the model unable to be accurately applied to local data, affecting the accuracy of the model.

Method used

Using a model training method based on feature distribution, the client obtains local sample images, determines data characteristics and classification results, sends individual data characteristics to the server, the server calculates global data characteristics, and returns them to the client, and updates the local classification model according to the feature distribution deviation. parameters to improve the generalization ability of the model.

Benefits of technology

By aligning the client data feature distribution, the model's feature expression ability and generalization ability can be improved, and the model's adaptability and accuracy can be enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118691933B_ABST
    Figure CN118691933B_ABST
Patent Text Reader

Abstract

This specification discloses a model training method and a task execution method based on feature distribution. The model training method includes: the client obtains each local sample image and determines the label information corresponding to each sample image; for each sample image, the sample image is input into the local classification model to be trained, the data feature corresponding to the sample image is determined and the classification result is determined; according to the label information of each sample image, the data distribution of the data feature corresponding to each sample image is determined, and the individual data feature is determined according to the data distribution, and the individual data feature is sent to the server, and the server determines the global data feature according to the individual data features sent by each client, and returns the global data feature to each client; according to the deviation between the classification result and the label information, and the deviation between the individual data feature and the global data feature, the loss value is determined; according to the loss value, the model parameters of the local classification model are updated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a model training method and a task execution method based on feature distribution. Background Art

[0002] With the rapid development of modern chip manufacturing, the computing power of computers has been greatly improved, thus providing a basis for the training of large neural network models. However, the training of a complex neural network model usually requires a large amount of data. Due to data privacy protection and other reasons, a lot of data is only allowed to be stored and used locally on the client side of different enterprises or institutions. In order to use this data to jointly train the model without transmitting the data to a server, federated learning came into being.

[0003] However, in the existing federated learning method, the client uploads the training data of local training to the server, and then the server processes the training data sent by all clients in a unified manner, obtains unified model data and sends it to each client. However, in actual application scenarios, the distribution of data types corresponding to data under different clients is often different. If the models in all clients use the same model data, the models in different clients will not be applicable to the distribution of local data, making it difficult to accurately identify the data.

[0004] Therefore, how to improve the accuracy of models under different clients in the process of federated learning is an urgent problem to be solved. Summary of the invention

[0005] This specification provides a model training method and a task execution method based on feature distribution to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a model training method based on feature distribution, which is applied to a client and includes:

[0008] Obtain local sample images and determine the label information corresponding to each sample image;

[0009] For each sample image, the sample image is input into a local classification model to be trained, so as to determine data features corresponding to the sample image through the local classification model, and determine a classification result of the sample image according to the data features;

[0010] Determine the data distribution of the data features corresponding to each sample image according to the label information of each sample image, determine the individual data features according to the data distribution, and send the individual data features to the server, so that the server determines the global data features according to the individual data features sent by each client, and returns the global data features to each client, wherein the individual data features are used to characterize the overall feature representation of the sample images under different label information used by the local classification model;

[0011] Determining a loss value of the local classification model according to a deviation between the classification result and the label information, and a deviation between the individual data feature and the global data feature;

[0012] According to the loss value, the model parameters of the local classification model are updated to obtain a trained local classification model.

[0013] Optionally, determining the loss value of the classification model according to the deviation between the classification result and the label information, and the deviation between the individual data feature and the global data feature, specifically includes:

[0014] Determining a classification loss value of the classification model according to a deviation between the classification result and the label information, and determining a correction value of the classification model according to a deviation between the individual data feature and the global data feature;

[0015] The classification loss value is corrected according to the correction value to determine the loss value of the local classification model.

[0016] Optionally, the model parameters of the local classification model include: feature extraction layer parameters;

[0017] According to the loss value, the common model parameters of the classification model are updated, specifically including:

[0018] According to the loss value, the feature extraction layer parameters are updated.

[0019] Optionally, the model parameters of the local classification model include: classification layer parameters;

[0020] According to the loss value, the model parameters of the local classification model are updated, specifically including:

[0021] Determine a change value corresponding to each classification layer parameter when the classification layer parameter is updated according to the classification loss value;

[0022] According to the classification loss value and the change value, the classification layer parameters are updated to obtain updated classification layer parameters and send them to the server, so that the server fuses the updated classification layer parameters sent by each client, obtains the fused classification layer parameters and returns them to each client.

[0023] Optionally, updating the classification layer parameters according to the loss value and the change value specifically includes:

[0024] According to the change value, the classification layer parameters are divided into personalized classification layer parameters and joint classification layer parameters;

[0025] The personalized classification layer parameters are updated according to the classification loss value, and the updated classification layer parameters are determined according to the joint classification layer parameters and the updated personalized classification layer parameters.

[0026] Optionally, according to the change value, the classification layer parameters are divided into personalized classification layer parameters and joint classification layer parameters, specifically including:

[0027] According to the change value corresponding to each updated classification layer parameter, sort the classification layer parameters in descending order;

[0028] The classification layer parameters ranked before the specified position are used as the personalized classification layer parameters, and the classification layer parameters ranked after the specified position are used as the joint classification layer parameters.

[0029] This specification provides a task execution method, including:

[0030] Obtain a target image to be classified;

[0031] Inputting the target image into a pre-trained classification model to determine a classification result corresponding to the target image through the classification model, wherein the classification model is trained by the above-mentioned model training method;

[0032] According to the classification result, the task is executed.

[0033] This specification provides a model training device based on feature distribution, including:

[0034] The acquisition module is used to acquire each local sample image and determine the label information corresponding to each sample image;

[0035] An input module, for each sample image, inputting the sample image into a local classification model to be trained, so as to determine data features corresponding to the sample image through the local classification model, and determine a classification result of the sample image according to the data features;

[0036] A first determination module is used to determine the data distribution of the data features corresponding to each sample image according to the label information of each sample image, and determine the individual data features according to the data distribution, and send the individual data features to the server, so that the server determines the global data features according to the individual data features sent by each client, and returns the global data features to each client, wherein the individual data features are used to characterize the overall feature representation of the sample images under different label information used by the local classification model;

[0037] A second determination module is used to determine the loss value of the local classification model according to the deviation between the classification result and the label information, and the deviation between the individual data feature and the global data feature;

[0038] An updating module is used to update the model parameters of the local classification model according to the loss value to obtain a trained local classification model.

[0039] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned model training method and task execution method based on feature distribution.

[0040] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned feature distribution-based model training method and task execution method when executing the program.

[0041] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0042] In the model training method based on feature distribution provided in the present specification, the client obtains each local sample image and determines the label information corresponding to each sample image; for each sample image, the sample image is input into the local classification model to be trained, the data feature corresponding to the sample image is determined and the classification result is determined; according to the label information of each sample image, the data distribution of the data feature corresponding to each sample image is determined, and the individual data feature is determined according to the data distribution, and the individual data feature is sent to the server. The server determines the global data feature based on the individual data features sent by each client, and returns the global data feature to each client; according to the deviation between the classification result and the label information, and the deviation between the individual data feature and the global data feature, the loss value is determined; according to the loss value, the model parameters of the local classification model are updated.

[0043] It can be seen from the above method that in the process of training the model, this scheme will determine the loss value based on the deviation between the classification results and the label information, as well as the deviation between the individual data features and the global data features. Compared with the federated learning process in which the models in different clients simply share model parameters, this scheme can align the distribution of data features of different clients as a whole, making the sample representation space of all customers more compact, which is conducive to improving the expression ability of features. Even if the feature offsets of each customer are large, their representation centers can still be made as consistent as possible by learning specific feature extraction capabilities, thereby improving the generalization ability of the model and further improving the performance of the model in different client individuals. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0045] Figure 1 A flowchart of a model training method based on feature distribution provided in this specification;

[0046] Figure 2 A schematic diagram of a classification layer parameter update process provided in this specification;

[0047] Figure 3 A flowchart of a task execution method provided in this specification;

[0048] Figure 4 A schematic diagram of a model training device based on feature distribution provided in this specification;

[0049] Figure 5 A schematic diagram of a task execution device provided in this specification;

[0050] Figure 6 A method corresponding to the Figure 1 or Figure 3 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0052] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0053] Figure 1 A flow chart of a model training method based on feature distribution provided in this specification includes the following steps:

[0054] S101: Acquire local sample images and determine label information corresponding to each sample image;

[0055] S102: For each sample image, input the sample image into a local classification model to be trained, so as to determine data features corresponding to the sample image through the local classification model, and determine a classification result of the sample image according to the data features.

[0056] Federated learning is a distributed training framework that includes a server (central server) and multiple clients participating in the training. The server mainly processes the model parameters sent by the client; the client mainly uses local data to train the model. Most importantly, the server and the client cannot transmit original data. However, they can transmit model parameters or gradients to train the model. This can well protect the data privacy of each customer.

[0057] The main process of the classic federated learning algorithm is: each client trains a neural network model based on local data and sends the model parameters to the server; the server sums and averages the model parameters of all clients participating in the training, and retransmits the averaged model to each client. This iteration continues until the model converges.

[0058] However, the prerequisite for the model parameter mean in federated learning to be effective is that the data type distribution of each client is the same, so that all clients can benefit from participating in the training together.

[0059] However, in real application scenarios, the data distribution between different customers is not the same. In some extreme cases, it is even worse than training the model locally. Therefore, federated learning becomes meaningless. For example, for hospitals with different user groups (such as children's hospitals, adult hospitals, men's hospitals, gynecological hospitals, etc.), due to the differences in age and gender of the user groups of these hospitals, the data type distribution of sample images in different hospitals is also quite different.

[0060] Based on this, this specification provides a model training method based on feature distribution, which allows each client to learn a model suitable for its local data through the methods of representation center alignment and model pruning.

[0061] Among them, for each client in the federated learning task, the client can obtain a local sample data set, which contains several sample images. The client can determine the label information corresponding to each sample image, and the label information can refer to the specific classification of the image itself or the specific classification of the instance in the image.

[0062] After acquiring each sample image, for each sample image, the client may input the sample image into a local classification model to be trained, so as to determine the data features corresponding to the sample image through the local classification model, and determine the classification result of the sample image according to the data features.

[0063] In this specification, the local classification model may include a feature extraction layer and a classification layer. Generally speaking, for a given convolutional neural network, the last layer of the neural network can be defined as a classification layer, and all previous network structures can be defined as feature extraction layers.

[0064] After the client inputs the sample image into the local classification model, it can extract features of the sample image through the feature extraction layer, thereby obtaining data features corresponding to each sub-data contained in the sample image as the data features corresponding to the sample image, and then input the extracted data features into the classification layer to obtain the classification results.

[0065] S103: Determine the data distribution of the data features corresponding to each sample image based on the label information of each sample image, and determine the individual data features based on the data distribution, and send the individual data features to the server, so that the server determines the global data features based on the individual data features received from each client, and returns the global data features to each client, wherein the individual data features are used to characterize the overall feature representation of the sample images under different label information used by the local classification model.

[0066] After determining the data features corresponding to each sub-data, the client can, based on the data features corresponding to each sub-data and the data distribution of the data features corresponding to each sample image, then use the distribution center of the data distribution as the individual data feature and send the individual data feature to the server.

[0067] Among them, for each client, the above-mentioned individual data features (distribution center) are used to characterize the overall feature representation of the sample images under different label information used by the client to train the local classification model. The individual data features can be the mean, median or mode of the corresponding data features of the sample images under each label information.

[0068] For Clients The corresponding sample image , the distribution of its data type can be expressed as , then the sample image Corresponding individual data characteristics It can be expressed as:

[0069]

[0070] The client can calculate individual data features through numerical approximation. The process can be expressed as:

[0071]

[0072] in, Indicates the amount of data corresponding to the sample image, represents the feature extraction layer parameters, Represents the input sub-data, Represents sub-data The corresponding label information.

[0073] The server can receive individual data features sent by each client in the federated learning task, and then determine the global data features based on the individual data features sent by each client. The global feature data is used to characterize the overall feature representation of sample images under different label information used by all clients in the federated learning task when training local classification models.

[0074] The server can then send the global data feature to each client. If the distribution of the data types corresponding to the sample images of all clients is expressed as , then the sample image Corresponding individual data characteristics It can be expressed as:

[0075]

[0076] In this specification, the server can determine the global data feature by calculating the average value of the individual data features sent by all clients. This process can be expressed as:

[0077]

[0078] in, Represents the total number of individual data features received by the server, which is equal to the total number of clients in the federated learning task.

[0079] S104: Determine the loss value of the local classification model according to the deviation between the classification result and the label information, and the deviation between the individual data feature and the global data feature.

[0080] The client can determine the loss value of the local classification model based on the deviation between the classification result and the label information, and the deviation between the individual data features and the global data features.

[0081] Specifically, the client can determine the classification loss value of the local classification model based on the deviation between the classification result and the label information. .

[0082] In actual applications, when the data distribution of the clients is the same, the clients share the feature extraction layer and they automatically have the same distribution center. However, when the data distribution between clients is different, even if different clients share the same feature extraction layer, the expected values ​​will be different due to the difference in data distribution, that is, the distribution center of the sample image under each client is also different.

[0083] That is to say, when there is a deviation between the individual data features corresponding to different clients, this deviation will reduce the ability of the feature extraction layer. Therefore, this effect can be reduced or even eliminated by reducing the distance between the individual data features corresponding to different clients.

[0084] Based on this, the client can add a center alignment regularization term to the local loss function, and then correct the classification loss value through the correction value determined by the center alignment regularization term, so as to align the distribution centers of the corresponding data features of the sample images under different clients.

[0085] The client determines the correction value of the local classification model based on the deviation between the individual data features and the global data features, and then adjusts the classification loss value based on the correction value. Make corrections to determine the loss of the local classification model Value, the loss value function can be expressed as:

[0086]

[0087] in, is the center alignment regularization term, which is used to determine the correction value of the local classification model.

[0088] It should be pointed out that the above center alignment regularization term (correction value) is mainly used for the training of the feature extraction layer and does not participate in the training of the classification layer.

[0089] S105: According to the loss value, the model parameters of the local classification model are updated to obtain a trained local classification model.

[0090] In this specification, the model parameters corresponding to the local classification model may include feature extraction layer parameters and classification layer parameters, which are determined by the server based on the updated model parameters sent by each client after the previous round of training.

[0091] Specifically, the client may update the feature extraction layer parameters corresponding to the feature extraction layer with minimizing the above loss value as the optimization goal, and update the classification layer parameters corresponding to the classification layer with minimizing the classification loss value as the optimization goal.

[0092] For the classification layer, the two mainstream training methods currently available cannot achieve satisfactory results. One is that all clients jointly train the classification layer by weighted summing up the classification layer parameters of each client in each training round. This solution is not conducive to the classification layer fitting the local data distribution, because the overall data distribution is not the same as the local data distribution. The other solution is to train the classification layer using local data alone. Moreover, the parameters of the classification layer are retained locally and not passed to the server. Although this method is conducive to the classification layer fitting the local data distribution, it is easy to produce overfitting due to the relatively small size of local data.

[0093] To solve the above problems, the client can first evaluate the correlation between each parameter of the classification layer and the local data distribution, and divide the classification layer parameters with a high correlation with the local data distribution into personalized classification layer parameters, and divide the remaining parameters into joint classification layer parameters. Personalized classification layer parameters only participate in local training, and joint classification layer parameters use all clients to participate in training. This division of classification layer parameters can effectively balance personalization and cooperation, so that the learned local classification model can not only fit the local data, but also improve the generalization ability of the model due to the help of other clients.

[0094] Specifically, the client can determine the change value corresponding to each public classification layer parameter when the public classification layer parameter is updated according to the above classification loss value. It can be determined by the gradient of the parameter and the value of the parameter itself:

[0095]

[0096] in, represents the set of classification layer parameters, Represents the updated classification layer parameters based on the loss value.

[0097] In this way, the client can determine the change value of each classification layer parameter after the update, and then divide the classification layer parameters into personalized classification layer parameters and joint classification layer parameters according to the change value corresponding to each classification layer parameter.

[0098] The client can sort the updated classification layer parameters in descending order, use the updated classification layer parameters before the specified position as the personalized classification layer parameters, and use the public classification layer parameters after the specified position as the joint classification layer parameters. The above-mentioned specified position can be set according to actual conditions, and this specification does not make specific restrictions on this.

[0099] In an embodiment provided in this specification, the number of personalized classification layer parameters and the number of joint classification layer parameters may be equal.

[0100] Afterwards, the client can update the personalized classification layer parameters with the minimization of the classification loss value as the optimization target, and determine the updated classification layer parameters according to the joint classification layer parameters and the updated personalized classification layer parameters, and then send the updated classification layer parameters to the server.

[0101] In this way, the classification layer of the client's local classification model can retain parameters that are more closely related to its own data (i.e., updated personalized classification layer parameters) and parameters that are less closely related to its own data (unupdated joint classification layer parameters).

[0102] In one embodiment provided in this specification, the client can define a mask matrix , whose number of rows and columns are equal to the number of rows and columns of the classification layer weights respectively. For the personalized classification layer parameters, The value of the corresponding position element in is 1. For the joint classification layer parameters, The value of the corresponding position element in is 0. At the same time, the client can define a Matrices of the same size , The component of each element in is 1. For ease of understanding, this specification provides a schematic diagram of the update process of the classification layer parameters, such as Figure 2 shown.

[0103] Figure 2 A schematic diagram of a classification layer parameter update process provided in this specification.

[0104] exist Figure 2 In , the local classification layer parameters represent the classification layer parameters updated locally on the client by the classification loss value, and the public classification layer parameters represent the unified parameters sent by the server to each client. The position with a value of 1 in the middle just represents the position of the joint classification layer parameter, correspondingly, The position with a value of 1 in represents the position of the personalized classification layer parameter. The client can update the personalized classification layer parameters through the above two matrices, and then determine the updated classification layer parameters according to the joint classification layer parameters and the updated personalized classification layer parameters. It can be expressed as:

[0105]

[0106] in, represents the common classification layer parameters (i.e., the model parameters obtained by averaging the updated classification layer parameters sent by all clients), Represents the local classification layer parameters updated based on the classification loss value (that is, the model parameters updated locally on the client).

[0107]

[0108] In actual applications, the client can set a hyperparameter , to determine the proportion of personalized parameters in all classification layer parameters, and then determine the above-mentioned "specified ranking". When the personalized classification layer parameters are equal to the joint classification layer parameters, ,certainly, It can also be set to other values ​​between 0 and 1.

[0109] It should be noted that in each round of training of the local classification model, for any client, the client can send the updated feature extraction layer parameters, updated classification layer parameters and individual data features obtained during this round of training to the server. After receiving the above information, the server will determine the feature extraction layer parameters corresponding to each client in the next round of training based on the updated feature extraction layer parameters sent by each client, determine the classification layer parameters corresponding to each client in the next round of training based on the updated classification layer parameters sent by each client, and determine the global data features based on the individual data features sent by each client.

[0110] Among them, the feature extraction layer parameters, classification layer parameters and data features corresponding to each client in the next round of training can be determined based on the mean of the updated feature extraction layer parameters sent by each client, the mean of the updated classification layer parameters sent by each client, and the mean of the individual data features sent by each client.

[0111] Of course, for the feature extraction layer, in each round of training, the feature extraction layer may not send the updated feature extraction layer parameters to the server, but only send individual data features to the server, and based on the deviation between the individual data features and the global data features sent by the server, as well as the deviation between the classification results and the label information, only update the parameters of the feature extraction layer locally.

[0112] In this specification, the above-mentioned model training method can be applied to various business scenarios such as medical treatment, unmanned driving, anomaly detection, etc., and this specification does not make specific limitations on this.

[0113] For example, in the field of big data medicine, each medical institution will save the patient's electronic medical records, CT images, etc. These data involve individual privacy and cannot be shared. At this time, the local classification model under each hospital can be federated through the model training method described above. On the one hand, it can ensure that the patient's individual information is retained in the local hospital. On the other hand, due to differences in age, exercise frequency, living habits, etc., the data of each hospital is often not independent and identically distributed. Therefore, after training with a personalized federated learning algorithm, each hospital can obtain a trained personalized neural network model that can better fit the local data distribution.

[0114] In addition, this solution can also be applied to scenarios with heterogeneous models. Due to the limitations of server computing power and storage space of each hospital, hospitals with abundant resources can deploy larger convolutional network models, while hospitals with scarce resources can only deploy simple convolutional network models. In order to ensure that the models obtained by each hospital participating in the training have a higher accuracy rate, the models deployed by each hospital have the same classifier but different feature extraction layers. In this case, this solution only needs slight adjustments to adapt to the scenario where model heterogeneity and data heterogeneity coexist, that is, the feature extraction layer of the model is not uploaded, only the classification layer parameters and individual data features of the model are uploaded. The server side only needs to update the classification layer parameters and global data features.

[0115] For ease of understanding, the following will describe an image classification method provided in this specification and applied to the above classification model from the perspective of practical application. Figure 3 shown.

[0116] Figure 3 The following is a flow chart of an image classification method provided in this specification, comprising the following steps:

[0117] S301: Acquire a target image to be classified;

[0118] S302: Inputting the target image into a pre-trained classification model to determine a classification result corresponding to the target image through the classification model, wherein the classification model is trained by the above-mentioned model training method;

[0119] S303: Execute a task according to the classification result.

[0120] For example, in an unmanned driving scenario, the target image can be image data collected by the unmanned driving device through sensors. Based on the classification results of the target image, obstacles around the unmanned driving device can be accurately identified, and then the path navigation of the unmanned driving device can be performed.

[0121] In the anomaly detection scenario for factory equipment, the target image can be the equipment image collected by the sensor. Based on the classification result of the target image, it can be detected whether the equipment operation is abnormal, thereby completing the abnormality monitoring task of the factory equipment.

[0122] In medical diagnosis, the target image can be an image of a patient acquired through a medical imaging device. Based on the classification result of the target image, the diseased part of the patient can be accurately detected, and the diagnostic task can be performed based on the detection result.

[0123] From the above method, we can see that during the training process, this solution can align the distribution center of each client data feature distribution by expressing the center regularization term. In this way, the average generalization error of the local classification model can be reduced. Compared with the ordinary method of simply sharing the parameters of the feature extraction layer, this solution can improve the generalization ability of the feature extraction layer, making the sample representation space of all customers more compact, which is conducive to improving the expressiveness of the features.

[0124] The classification layer parameters are further divided by the change value of the classification layer parameters, and the parameters that have a greater impact on the model's loss function are divided into personalized parameters, which is conducive to fitting the local data distribution. At the same time, relatively unimportant parameters are divided into cooperative parameters, which is conducive to obtaining information about the overall data and improving the generalization ability of the classification layer without affecting the fitting of personalized parameters to local data.

[0125] In particular, this solution can also be applied to more complex real-world scenarios, that is, scenarios where data heterogeneity and model heterogeneity coexist. In this scenario, different clients can have different feature extraction layers, but have the same classification layer, that is, the last linear connection layer. This training process is consistent with the above. However, due to the different feature extraction layers, there is no need to download and upload the parameters of the feature extraction layer. In other words, the feature extraction layer is only trained locally, and the rest of the process is the same as the above method.

[0126] It is also important to emphasize that this solution has good robustness for various data heterogeneity scenarios and a simpler network design architecture, which is conducive to the deployment and training of models in federated learning scenarios. Specifically, this solution has good robustness for the problem of unbalanced label distribution, and does not require the design of additional auxiliary networks to input the label distribution information of each client. This makes the model architecture in this solution more concise and easy to deploy.

[0127] At the same time, for the problem of feature distribution shift, this solution uses representation center alignment. Even if the feature shifts of each client are large, we can still make their representation centers as consistent as possible by learning a specific feature extraction layer. At this time, the methods designed for the problem of unbalanced label distribution lose their effectiveness in this scenario, because the label distribution of different clients is consistent, only the feature distribution shifts, so there will be no difference in label distribution.

[0128] The above is one or more implementations of the model training method and task execution method based on feature distribution in this specification. Based on the same idea, this specification also provides corresponding devices, such as Figure 4 So Figure 5 shown.

[0129] Figure 4 A schematic diagram of a model training device based on feature distribution provided in this specification includes:

[0130] The acquisition module 401 is used to acquire each local sample image and determine the label information corresponding to each sample image;

[0131] An input module 402 is used to input each sample image into a local classification model to be trained, so as to determine data features corresponding to the sample image through the local classification model, and determine a classification result of the sample image according to the data features;

[0132] A first determination module 403 is used to determine the data distribution of the data features corresponding to each sample image according to the label information of each sample image, and determine the individual data features according to the data distribution, and send the individual data features to the server, so that the server determines the global data features according to the individual data features sent by each client, and returns the global data features to each client, wherein the individual data features are used to characterize the overall feature representation of the sample images under different label information used by the local classification model;

[0133] A second determination module 404 is used to determine the loss value of the local classification model according to the deviation between the classification result and the label information, and the deviation between the individual data feature and the global data feature;

[0134] The updating module 405 is used to update the model parameters of the local classification model according to the loss value to obtain a trained local classification model.

[0135] Optionally, the second determination module 404 is specifically used to determine the classification loss value of the classification model based on the deviation between the classification result and the label information, and to determine the correction value of the classification model based on the deviation between the individual data features and the global data features; based on the correction value, the classification loss value is corrected to determine the loss value of the local classification model.

[0136] Optionally, the model parameters of the local classification model include: feature extraction layer parameters;

[0137] The second determination module 404 is specifically configured to update the feature extraction layer parameters according to the loss value.

[0138] Optionally, the model parameters of the local classification model include: classification layer parameters;

[0139] Optionally, the second determination module 404 is specifically used to determine a change value corresponding to each classification layer parameter when the classification layer parameter is updated according to the classification loss value;

[0140] According to the classification loss value and the change value, the classification layer parameters are updated to obtain updated classification layer parameters and send them to the server, so that the server fuses the updated classification layer parameters sent by each client, obtains the fused classification layer parameters and returns them to each client.

[0141] Optionally, the update module 405 is specifically used to divide the classification layer parameters into personalized classification layer parameters and joint classification layer parameters according to the change value; update the personalized classification layer parameters according to the classification loss value, and determine the updated classification layer parameters based on the joint classification layer parameters and the updated personalized classification layer parameters.

[0142] Optionally, the update module 405 is specifically used to sort the classification layer parameters in descending order according to the change value corresponding to each updated classification layer parameter; the classification layer parameters sorted before the specified position are used as the personalized classification layer parameters, and the classification layer parameters sorted after the specified position are used as the joint classification layer parameters.

[0143] Figure 5 A schematic diagram of a task execution device provided for this specification includes:

[0144] An acquisition module 501 is used to acquire a target image to be classified;

[0145] The classification module 502 is used to input the target image into a pre-trained classification model to determine the classification result corresponding to the target image through the classification model;

[0146] The execution module 503 is used to execute the task according to the classification result.

[0147] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 and Figure 3 A model training method and task execution method based on feature distribution are provided.

[0148] This manual also provides Figure 6 The one shown corresponds to Figure 1 or Figure 3 A schematic diagram of the electronic device. Figure 6 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The model training method based on feature distribution and Figure 3 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0149] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0150] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0151] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0152] For the convenience of description, the above device is described by dividing it into various units according to its functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0153] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0158] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0159] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0160] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0161] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0163] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0164] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A model training method based on feature distribution, characterized in that: The method is applied to a client and includes: Obtain local sample images and determine the label information corresponding to each sample image; For each sample image, the sample image is input into a local classification model to be trained, so as to determine data features corresponding to the sample image through the local classification model, and determine a classification result of the sample image according to the data features; Determine the data distribution of the data features corresponding to each sample image according to the label information of each sample image, determine the individual data features according to the data distribution, and send the individual data features to the server, so that the server determines the global data features according to the individual data features sent by each client, and returns the global data features to each client, wherein the individual data features are used to characterize the overall feature representation of the sample images under different label information used by the local classification model; According to the deviation between the classification result and the label information, and the deviation between the individual data feature and the global data feature, the loss value of the local classification model is determined, wherein the classification loss value of the classification model is determined according to the deviation between the classification result and the label information, and the correction value of the classification model is determined according to the deviation between the individual data feature and the global data feature, and the classification loss value is corrected according to the correction value to determine the loss value of the local classification model; According to the loss value, the model parameters of the local classification model are updated to obtain a trained local classification model.

2. The method according to claim 1, characterized in that The model parameters of the local classification model include: feature extraction layer parameters; According to the loss value, the common model parameters of the classification model are updated, specifically including: According to the loss value, the feature extraction layer parameters are updated.

3. The method according to claim 1, characterized in that The model parameters of the local classification model include: classification layer parameters; According to the loss value, the model parameters of the local classification model are updated, specifically including: Determine a change value corresponding to each classification layer parameter when the classification layer parameter is updated according to the classification loss value; According to the classification loss value and the change value, the classification layer parameters are updated to obtain updated classification layer parameters and send them to the server, so that the server fuses the updated classification layer parameters sent by each client, obtains the fused classification layer parameters and returns them to each client.

4. The method according to claim 3, characterized in that According to the loss value and the change value, the classification layer parameters are updated, specifically including: According to the change value, the classification layer parameters are divided into personalized classification layer parameters and joint classification layer parameters; The personalized classification layer parameters are updated according to the classification loss value, and the updated classification layer parameters are determined according to the joint classification layer parameters and the updated personalized classification layer parameters.

5. The method according to claim 4, characterized in that According to the change value, the classification layer parameters are divided into personalized classification layer parameters and joint classification layer parameters, specifically including: According to the change value corresponding to each updated classification layer parameter, sort the classification layer parameters in descending order; The classification layer parameters ranked before the specified position are used as the personalized classification layer parameters, and the classification layer parameters ranked after the specified position are used as the joint classification layer parameters.

6. A task execution method, characterized in that: include: Obtain a target image to be classified; Inputting the target image into a pre-trained classification model to determine a classification result corresponding to the target image through the classification model, wherein the classification model is trained by the method according to any one of claims 1 to 5 above; According to the classification result, the task is executed.

7. A model training device based on feature distribution, characterized in that: include: The acquisition module is used to acquire each local sample image and determine the label information corresponding to each sample image; An input module, for each sample image, inputting the sample image into a local classification model to be trained, so as to determine data features corresponding to the sample image through the local classification model, and determine a classification result of the sample image according to the data features; A first determination module is used to determine the data distribution of the data features corresponding to each sample image according to the label information of each sample image, and determine the individual data features according to the data distribution, and send the individual data features to the server, so that the server determines the global data features according to the individual data features sent by each client, and returns the global data features to each client, wherein the individual data features are used to characterize the overall feature representation of the sample images under different label information used by the local classification model; A second determination module is used to determine the loss value of the local classification model according to the deviation between the classification result and the label information, and the deviation between the individual data feature and the global data feature, wherein the classification loss value of the classification model is determined according to the deviation between the classification result and the label information, and the correction value of the classification model is determined according to the deviation between the individual data feature and the global data feature, and the classification loss value is corrected according to the correction value to determine the loss value of the local classification model; An updating module is used to update the model parameters of the local classification model according to the loss value to obtain a trained local classification model.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 6 is implemented.