Image classification model training method and device, medium and electronic equipment
By using knowledge distillation technology in federated learning, the central node adjusts the image classification model parameters of edge nodes, solving the problem of model inability to run or store caused by the architecture differences between different edge nodes, and realizing model training that adapts to the resources of each edge node is achieved.
Patent Information
- Application Number
- CN202411822578.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-11
- Publication Date
- 2025-02-18
AI Technical Summary
In federated learning, due to the differences in computing resources and sample data of different edge nodes, the architecture of the image classification model is different. The model trained by the central node cannot run or store at the edge node, resulting in the failure of the training task.
The central node obtains the local coding end parameters of each edge node and performs knowledge distillation to adjust the global coding end parameters. For each edge node, it adjusts its local coding end parameters according to the adjusted global coding end and local coding end, and sends adjustments. The parameters afterwards are to the edge node.
When each edge node shares data, each edge node obtains an image classification model adapted to its own computing resource configuration, which improves the encoding-end feature representation ability of the image classification model, and ensures the successful progress of the training task.
Smart Images
Figure CN119478549A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence, and in particular to a training method, device, medium and electronic device for an image classification model. Background Art
[0002] With the development of science and technology, privacy data has received increasing attention. With the development of science and technology, artificial intelligence has developed rapidly, and image classification models have been widely used in many fields.
[0003] At present, Federated Learning (FL) technology can use the central node to protect the privacy of local sample data of each edge node, so that each edge node can jointly train the image classification model, thereby improving the performance of the trained image classification model. In actual scenarios, since the computing resources and sample data corresponding to different edge nodes are quite different, the architectures of the image classification models deployed in different edge nodes are different. When training the image classification model, the central node can use the knowledge distillation technology to train the image classification model deployed in the central node, i.e., the student model, based on the image classification model trained by each edge node, i.e., the teacher model, to obtain the trained image classification model. However, as mentioned above, if the architectures of the image classification models deployed in different edge nodes are different, then when the central node sends the trained image classification model to each edge node, the edge node may be unable to run or store the trained image classification model, which may result in the inability to perform the task of training the image classification model.
[0004] Based on this, this application specification provides a training method for an image classification model. Summary of the invention
[0005] This specification provides a training method and device for an image classification model, a storage medium, and an electronic device to at least partially solve the above-mentioned problems existing in the prior art.
[0006] This manual adopts the following technical solutions:
[0007] This specification provides a training method for an image classification model, the method is applied to a central node, and the method is used to train an encoding end in the image classification model; the method includes:
[0008] Obtain the parameters of the local encoding end of the image classification model deployed in each edge node;
[0009] According to the parameters of each local encoder, the global encoder of the image classification model deployed in the central node is distilled to obtain the parameters of the global encoder after distillation;
[0010] For each edge node, according to the parameters of the global encoding end after the distillation and the preset training target, the parameters of the local encoding end corresponding to the edge node are adjusted to obtain the adjusted parameters of the local encoding end corresponding to the edge node;
[0011] The obtained adjusted parameters of the local encoding end corresponding to the edge node are sent to the edge node, so that the edge node obtains the trained image classification model corresponding to the edge node according to the encoding end constructed based on the adjusted parameters of the local encoding end.
[0012] Optionally, the image classification model deployed in each edge node includes a first online encoding end and a first target encoding end, and the first online encoding end and the first target encoding end are asymmetric twin networks;
[0013] For each edge node, the edge node determines the parameters of the local encoding end of the image classification model deployed in the edge node, specifically including:
[0014] The edge node determines, for each first sample image pre-stored in the edge node, a first enhanced image of the first sample image;
[0015] The edge node inputs the first sample image into the first online encoding end to obtain a first feature, and inputs the first enhanced image into the first target encoding end to obtain a second feature;
[0016] The edge node trains the first online encoder with the goal of maximizing the similarity between the first feature and the second feature to obtain a trained first online encoder, and uses the trained first online encoder as a local encoder;
[0017] According to the local encoding end, the edge node obtains the parameters of the local encoding end corresponding to the edge node.
[0018] Optionally, the image classification model deployed in each edge node further includes a prediction end, and the prediction end is connected to the first online encoding end;
[0019] The edge node inputs the first sample image into the first online encoding end to obtain a first feature, specifically including:
[0020] The edge node inputs the first sample image into the first online encoding end to obtain a first feature, and inputs the first feature into the prediction end to obtain a third feature;
[0021] Training the first online encoder includes:
[0022] The edge node trains the first online encoding end with the goal of maximizing the similarity between the third feature and the second feature.
[0023] Optionally, the method further comprises:
[0024] According to the first online encoder, the first target encoder is trained using an exponential moving average algorithm.
[0025] Optionally, adjusting a parameter of a local encoding end corresponding to the edge node to obtain an adjusted parameter of the local encoding end corresponding to the edge node specifically includes:
[0026] Using an encoder constructed based on the parameters of the global encoder after distillation as a second target encoder, and using an encoder constructed based on the parameters of the local encoder corresponding to the edge node as a second online encoder; and, for each second sample image pre-stored in the central node, determining a second enhanced image corresponding to the second sample image;
[0027] Inputting the second enhanced image to the second online encoding end to obtain a fourth feature, and inputting the second sample image to the second target encoding end to obtain a fifth feature;
[0028] With the goal of maximizing the similarity between the fourth feature and the fifth feature, adjusting the parameters of the second online encoding end to obtain adjusted parameters of the second online encoding end;
[0029] According to the adjusted parameters of the second online encoding end, the adjusted parameters of the local encoding end corresponding to the edge node are obtained.
[0030] Optionally, the global encoder of the image classification model deployed in the central node is distilled to obtain parameters of the global encoder after distillation, specifically including:
[0031] The encoder constructed based on the parameters of each local encoder is used as the teacher model, and the global encoder is used as the student model;
[0032] Determining a distillation loss based on each teacher model and the student model;
[0033] According to the distillation loss, adjusting the parameters of the student model to obtain a student model with adjusted parameters;
[0034] According to the student model after parameter adjustment, the parameters of the global encoding end after distillation are obtained.
[0035] Optionally, the distillation loss is determined according to each teacher model and the student model, specifically including:
[0036] For each second sample image pre-stored in the central node, the second sample image is input into each teacher model to obtain each sixth feature; and the second sample image is input into the student model to obtain the seventh feature, and each other sample image except the second sample image is input into the student model to obtain each eighth feature;
[0037] According to the sixth features and the seventh feature, using an attention mechanism, determining a weighted aggregate feature corresponding to the second sample image;
[0038] A distillation loss is determined based on the seventh feature, the weighted aggregate feature, and the eighth features.
[0039] Optionally, adjusting the parameters of the student model specifically includes:
[0040] Receiving a local loss sent by each edge node, where the local loss is a loss of a local encoding end of training of the edge node;
[0041] The parameters of the student model are adjusted according to the local losses and the distillation loss.
[0042] This specification provides a pre-training device for a language model, the device is located at a central node, and the device is used to train an encoding end in an image classification model; the device specifically includes:
[0043] An acquisition module, used to acquire parameters of the local encoding end of the image classification model deployed in each edge node;
[0044] A distillation module is used to distill the global encoder of the image classification model deployed in the central node according to the parameters of each local encoder to obtain the parameters of the global encoder after distillation;
[0045] An adjustment module is used to adjust the parameters of the local encoding end corresponding to each edge node according to the parameters of the global encoding end after distillation and the preset training target, so as to obtain the adjusted parameters of the local encoding end corresponding to the edge node;
[0046] The sending module is used to send the adjusted parameters of the local encoding end corresponding to the edge node to the edge node, so that the edge node obtains the trained image classification model corresponding to the edge node according to the encoding end constructed based on the adjusted parameters of the local encoding end.
[0047] Optionally, the image classification model deployed in each edge node includes a first online encoding end and a first target encoding end, and the first online encoding end and the first target encoding end are asymmetric twin networks;
[0048] The acquisition module is specifically used for: the edge node determines a first enhanced image of the first sample image for each first sample image pre-stored in the edge node; the edge node inputs the first sample image into the first online encoding end to obtain a first feature, and inputs the first enhanced image into the first target encoding end to obtain a second feature; the edge node trains the first online encoding end with the goal of maximizing the similarity between the first feature and the second feature to obtain the trained first online encoding end, and uses the trained first online encoding end as a local encoding end; according to the local encoding end, the edge node obtains the parameters of the local encoding end corresponding to the edge node.
[0049] Optionally, the image classification model deployed in each edge node further includes a prediction end, and the prediction end is connected to the first online encoding end;
[0050] The acquisition module is specifically configured to, the edge node inputs the first sample image into the first online encoding end to obtain a first feature, and inputs the first feature into the prediction end to obtain a third feature;
[0051] The acquisition module is specifically used for the edge node to train the first online encoding end with the goal of maximizing the similarity between the third feature and the second feature.
[0052] Optionally, the acquisition module is further used to train the first target encoding end by adopting an exponential moving average algorithm according to the first online encoding end.
[0053] Optionally, the adjustment module is specifically used to use the encoding end constructed based on the parameters of the global encoding end after the distillation as the second target encoding end, and the encoding end constructed based on the parameters of the local encoding end corresponding to the edge node as the second online encoding end; and, for each second sample image pre-stored in the central node, determine the second enhanced image corresponding to the second sample image; input the second enhanced image into the second online encoding end to obtain a fourth feature, and input the second sample image into the second target encoding end to obtain a fifth feature; with the goal of maximizing the similarity between the fourth feature and the fifth feature, adjust the parameters of the second online encoding end to obtain the adjusted parameters of the second online encoding end; according to the adjusted parameters of the second online encoding end, obtain the adjusted parameters of the local encoding end corresponding to the edge node.
[0054] Optionally, the distillation module is specifically used to use the encoding end constructed based on the parameters of each local encoding end as a teacher model, and the global encoding end as a student model; determine the distillation loss according to each teacher model and the student model; adjust the parameters of the student model according to the distillation loss to obtain the student model after parameter adjustment; obtain the parameters of the global encoding end after distillation according to the student model after parameter adjustment.
[0055] Optionally, the distillation module is specifically used to, for each second sample image pre-stored in the central node, input the second sample image into each teacher model respectively to obtain each sixth feature; and input the second sample image into the student model to obtain the seventh feature, and input each other sample image except the second sample image into the student model respectively to obtain each eighth feature; according to the sixth features and the seventh features, using the attention mechanism, determine the weighted aggregation feature corresponding to the second sample image; according to the seventh feature, the weighted aggregation feature and the eighth features, determine the distillation loss.
[0056] Optionally, the distillation module is specifically used to receive local losses sent by each edge node, where the local losses are losses of a local encoding end of edge node training; and adjust parameters of the student model according to each local loss and the distillation loss.
[0057] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the training method of the above-mentioned image classification model.
[0058] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned image classification model training method when executing the program.
[0059] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:
[0060] It can be seen from the training method of the image classification model provided in this specification that when each edge node trains an image classification model suitable for itself, the central node can use knowledge distillation technology to adjust the parameters of the global encoding end in the central node according to the parameters of the local encoding end of the image classification model of each edge node obtained, and obtain the adjusted parameters of the global encoding end. Then, for each edge node, the central node adjusts the parameters of the local encoding end of the edge node according to the adjusted global encoding end and the local encoding end of the edge node, and sends the adjusted parameters of the local encoding end to the edge node. In this method, the central node does not directly send the adjusted parameters of the global encoding end to the edge nodes, but adjusts the local encoding end based on the adjusted global encoding end, thereby sending the adjusted parameters of the local encoding end to the edge nodes. This improves the feature representation capability of the encoding end of the image classification model trained by each edge node, and enables each edge node to obtain an image classification model that is adapted to its own computing resource configuration when each edge node shares data. That is, when each edge node shares data, even if the architecture of the image classification model of each edge node is different, each edge node can still obtain an image classification model that is adapted to its own model architecture. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:
[0062] Figure 1 A flowchart of a training method for an image classification model in this specification;
[0063] Figure 2 A schematic diagram of the structure of an image classification model deployed in an edge node provided in this specification;
[0064] Figure 3 A schematic diagram of the structure of an image classification model deployed in an edge node provided in this specification;
[0065] Figure 4 A schematic diagram of the training process of an image classification model provided in this manual;
[0066] Figure 5 A schematic diagram of a training device for an image classification model provided in this specification;
[0067] Figure 6 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION
[0068] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this application will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in the specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0069] For image classification models, due to differences in sample data and computing resources of nodes where image classification models are deployed, the architecture of image classification models is different. For example, on a server in a hospital, an image classification model is deployed, which is used to classify medical images. In order to ensure the accuracy of classification, the architecture of the image classification model deployed on the server can be relatively complex. An image classification model is deployed on a user's mobile phone, which is used to recognize the user's facial image. Due to the limited computing resources on the mobile phone, a lightweight image classification model can be deployed. In this way, if the architecture of the model used by each edge node is different, and the image classification model is trained using federated learning technology, in the process of parameter transmission between the central node and the edge node, if the central node directly sends the obtained model parameters to each edge node, the edge node will be unable to run or store the trained image classification model, resulting in the inability to perform the task of training the image classification model.
[0070] Based on this, this specification provides a training method for an image classification model.
[0071] In this specification, a node can be any electronic device with computing capabilities, such as a server, a terminal, etc., and this specification does not impose any restrictions.
[0072] In addition, in this specification, each edge node is pre-deployed with an image classification model, and the image classification model of each edge node is adapted to its own computing resources and downstream tasks, etc. The specific architecture of the image classification model deployed in each edge node is not specifically limited in this specification, and each edge node can be pre-set. An image classification model is also pre-deployed in the central node. The image classification model in the central node is used to learn the feature representation of the image from the image classification model of each edge node to aggregate the feature representation capabilities of each edge node, thereby improving the generalization and accuracy of the image classification model in the central node.
[0073] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0074] Figure 1This is a flow chart of a training method for an image classification model provided in this specification. The method is applied to a central node. The method is used to train an encoding end in an image classification model. Specifically, the following steps may be included:
[0075] S100: Obtain parameters of the local encoding end of the image classification model deployed in each edge node.
[0076] In this specification, the central node can obtain the parameters of the local encoding end of the image classification model deployed in each edge node. Specifically, each edge node can send the parameters of the local encoding end of the image classification model deployed with it to the central node, and the central node can receive the parameters of the local encoding end sent by each edge node.
[0077] It should be noted that in the present specification, as mentioned above, an image classification model is deployed in each edge node, and the image classification model may be an untrained initial image classification model. The encoding end of the untrained initial image classification model deployed in each edge node is used as the initial encoding end. For each edge node, the edge node may train the initial encoding end corresponding to the edge node based on the local data of the edge node to obtain the trained image classification model, and then the encoding end in the trained image classification model may be used as a local encoding end, so that the parameters of the local encoding end may be sent to the central node.
[0078] like Figure 2 As shown, Figure 2 A structural diagram of an image classification model deployed in an edge node provided in this specification, in which the image classification model deployed in each edge node includes a first online encoding end and a first target encoding end, and the first online encoding end and the first target encoding end are asymmetric twin networks. For each edge node, when the edge node determines the parameters of its corresponding local encoding end, the edge node can determine the first enhanced image of the first sample image for each first sample image pre-stored in the edge node, and input the first sample image into the first online encoding end to obtain the first feature, and input the first enhanced image into the first target encoding end to obtain the second feature. Then, the edge node can train the first online encoding end with the goal of maximizing the similarity between the first feature and the second feature, obtain the trained first online encoding end, and use the trained first online encoding end as the local encoding end. Then, according to the local encoding end, the edge node obtains the parameters of the local encoding end corresponding to the edge node.
[0079] It should be noted that the first sample image can also be input into the first target encoding end to obtain the first feature, and the first enhanced image can be input into the first online encoding end to obtain the second feature. In this specification, the first sample image and the first enhanced image are positive samples, so the first online encoding end can be trained with the goal of maximizing the similarity between the feature representations of the positive samples by the first online encoding end and the first target encoding end. It can also be based on other goals in comparative learning, such as encoding different samples with the first online encoding end and the first target encoding end, and then the first online encoding end can be trained with the goal of minimizing the similarity between the encoding results of the two, which is not limited in this specification.
[0080] like Figure 3 As shown, Figure 3 This is a structural diagram of an image classification model deployed in an edge node provided in this specification. It can be seen that the image classification model deployed in the edge node also includes a prediction end, which is connected to the first online encoding end. The edge node inputs the first sample image into the first online encoding end to obtain the first feature, and inputs the first feature into the prediction end to obtain the third feature. Then the edge node can train the first online encoding end with the goal of maximizing the similarity between the third feature and the second feature. The trained first online encoding end is used as a local encoding end, so that the edge node can obtain the parameters of the local encoding end corresponding to the edge node and send them to the central node.
[0081] In addition, when the edge node trains the first online encoding end, it can also train the first target encoding end. The edge node can train the first target encoding end according to the first online encoding end using an exponential moving average algorithm. How to train the first target encoding end using an exponential moving average algorithm according to the first online encoding end is a relatively mature technology in the current asymmetric twin network training, and this specification will not be repeated.
[0082] Of course, the edge node can also directly use the parameters of the initial encoding end as the parameters of the local encoding end, thereby sending the parameters of the local encoding end to the central node. In addition, the image classification model deployed in each edge node can also be a trained image classification model, and then the parameters of the encoding end of the trained image classification model are used as the parameters of the local encoding end, thereby sending the parameters of the local encoding end to the central node. There are many ways for edge nodes to obtain the parameters of the local encoding end, and this specification does not make specific restrictions. As long as each edge node can send the parameters of the encoding end in its corresponding image classification model to the central node, the parameters of the encoding end sent to the central node are the local encoding end parameters.
[0083] S102: According to the parameters of each local encoding end, the global encoding end of the image classification model deployed in the central node is distilled to obtain the parameters of the global encoding end after distillation.
[0084] Therefore, the central node can distill the global encoding end of the image classification model deployed in the central node according to the parameters of each local encoding end to obtain the parameters of the global encoding end after distillation. In this specification, the central node can construct each encoding end based on the parameters of each local encoding end, and can use each encoding end constructed based on the parameters of each local encoding end as each teacher model, and use the global encoding end of the image classification model deployed in the central node as a student model. Therefore, the distillation loss can be determined according to each teacher model and the student model, and the parameters of the student model can be adjusted according to the distillation loss to obtain the adjusted student model. Therefore, according to the student model after parameter adjustment, the parameters of the global encoding end after distillation can be obtained.
[0085] The knowledge distillation technology is applied to federated learning, so that the global encoding end corresponding to the central node can aggregate and learn the feature representation capabilities of each local encoding end corresponding to each edge node, thereby training the image classification model while realizing the sharing of sample data of each edge node.
[0086] The above method determines the distillation loss based on each teacher model and the student model, and adjusts the parameters of the student model according to the distillation loss to obtain the adjusted student model. This is a common knowledge distillation method and will not be described in detail in this specification.
[0087] S104: For each edge node, according to the parameters of the global encoding end after distillation and the preset training target, the parameters of the local encoding end corresponding to the edge node are adjusted to obtain the adjusted parameters of the local encoding end corresponding to the edge node.
[0088] S106: Send the obtained adjusted parameters of the local encoding end corresponding to the edge node to the edge node, so that the edge node obtains the trained image classification model according to the encoding end constructed based on the adjusted parameters of the local encoding end.
[0089] Based on the above steps S100 to S102, the central node obtains the parameters of the local encoding end of each edge node, and obtains the parameters of the global encoding end after distillation based on the parameters of each local encoding end. The central node can adjust the parameters of the local encoding end corresponding to each edge node according to the parameters of the global encoding end after distillation and the preset training target, and obtain the adjusted parameters of the local encoding end corresponding to the edge node.
[0090] Specifically, the central node can use the encoder constructed based on the parameters of the global encoder after distillation as the second target encoder, and use the encoder constructed based on the parameters of the local encoder corresponding to the edge node as the second online encoder, then the second target encoder and the second online encoder are also asymmetric Siamese networks. The central node can also determine the second enhanced image corresponding to each second sample image pre-stored in the central node, input the second enhanced image into the second online encoder to obtain the fourth feature, and input the second sample image into the second target encoder to obtain the fifth feature. Then, the parameters of the second online encoder can be adjusted with the goal of maximizing the similarity between the fourth feature and the fifth feature to obtain the adjusted parameters of the second online encoder, and the parameters of the adjusted local encoder corresponding to the edge node can be obtained according to the adjusted parameters of the second online encoder. That is, the contrast learning method can be used to maximize the similarity between the feature representations of the second online encoder and the second target encoder for the same sample image, and to minimize the similarity between the feature representations of the second online encoder and the second target encoder for different sample images, and adjust the parameters of the second online encoder.
[0091] Of course, when the parameters of the local encoding end corresponding to the edge node are adjusted according to the parameters of the global encoding end after distillation and the preset training targets, the preset training targets can be determined according to the specific application scenarios and business needs, and this manual does not make specific restrictions.
[0092] Finally, the central node may send the adjusted parameters of the local encoding end corresponding to the edge node to the edge node, so that the edge node obtains the trained image classification model corresponding to the edge node according to the encoding end constructed based on the adjusted parameters of the local encoding end.
[0093] It should be noted that the central node and each edge node in this specification can constitute a federated learning framework. When performing the task of training an image classification model, it is intended to enable each edge node with different architectures of the deployed image classification model to obtain the trained image classification model corresponding to each edge node under the condition of data sharing.
[0094] Then for each edge node, when the edge node receives the adjusted parameters of the local encoding end corresponding to the edge node sent by the central node, the edge node can construct the encoding end according to the parameters of the adjusted local encoding end corresponding to the edge node, so as to obtain the trained image classification model corresponding to the edge node. Of course, training the encoding end of the image classification model is also an iterative training process, so the edge node or the central node can determine whether the end requirement is met when the encoding end constructed based on the adjusted parameters of the local encoding end is obtained. The end requirement includes that the number of training iterations reaches a preset number threshold or the loss when the adjusted parameters of the local encoding end are obtained is less than a preset loss threshold. If so, the central node sends the parameters of the adjusted local encoding end corresponding to the edge node to the edge node, and the training is terminated. The edge node constructs the encoding end based on the parameters of the adjusted local encoding end corresponding to the edge node to obtain the trained image classification model. If not, the central node sends the parameters of the adjusted local coding end corresponding to the edge node to the edge node, and the edge node constructs the coding end based on the parameters of the adjusted local coding end corresponding to the edge node, and continues to train the coding end constructed based on the parameters of the adjusted local coding end corresponding to the edge node based on the training data in the edge node. The trained coding end is used as the local coding end again, and the parameters of the local coding end are sent to the central node. S100 to S106 are repeated until the end requirements are met. The training is terminated, and the central node can send the parameters of the adjusted local coding end corresponding to the edge node determined for the last time to the edge node. The edge node can construct the coding end based on the parameters of the adjusted local coding end corresponding to the edge node determined for the last time, and obtain a trained image classification model.
[0095] Further, the encoder constructed by the edge node based on the parameters of the adjusted local encoder is a trained encoder. Each edge node can also train other network layers in the image classification model according to its specific target business, so as to obtain an image classification model suitable for the target business, which is not described in detail in this specification.
[0096] like Figure 4 As shown, it is a schematic diagram of the training process of an image classification model provided in this specification. It can be seen that when each edge node trains an image classification model suitable for itself, the central node can use knowledge distillation technology to adjust the parameters of the global encoding end in the central node according to the parameters of the local encoding end of the image classification model of each edge node obtained, and obtain the adjusted parameters of the global encoding end. Then, for each edge node, the central node adjusts the parameters of the local encoding end of the edge node according to the adjusted global encoding end and the local encoding end of the edge node, and sends the adjusted parameters of the local encoding end to the edge node.
[0097] based on Figure 1 In the training method of the image classification model shown in the figure, the central node does not directly send the adjusted parameters of the global encoding end to the edge node, but adjusts the local encoding end based on the adjusted global encoding end, thereby sending the adjusted parameters of the local encoding end to the edge node. While improving the feature representation capability of the encoding end of the image classification model trained by each edge node, it is achieved that when each edge node shares data, each edge node obtains an image classification model that adapts to its own computing resource configuration. That is, when each edge node can share data, even if the architecture of the image classification model of each edge node is different, each edge node can still obtain an image classification model that adapts to its own model architecture.
[0098] In addition, in order to further improve the performance of the global encoder trained by the central node, thereby improving the performance of the local encoder trained by each edge node, in this specification, when the central node determines the distillation loss according to each teacher model and the student model, the central node can input the second sample image into each teacher model for each second sample image pre-stored in the central node, and obtain each sixth feature. And the second sample image can be input into the student model to obtain the seventh feature, and the central node can input each other sample image except the second sample image into the student model to obtain each eighth feature. Then, according to each sixth feature and the seventh feature, the attention mechanism can be used to determine the weighted aggregation feature corresponding to the second sample image, so that the distillation loss can be determined according to the seventh feature, the weighted aggregation feature and each eighth feature.
[0099] The distillation loss after using the attention mechanism can be expressed as follows:
[0100]
[0101]
[0102] Among them, r s For the student model, r n is the teacher model, τ is the temperature parameter that controls entropy, sim() is the function that determines similarity, and Attn() is the attention network. i is the second sample image, r N,i is the feature representation obtained after inputting the second sample image into a teacher model, which is also the sixth feature. s,i It is the feature representation obtained after the second sample image is input into the student model, that is, the seventh feature. It indicates that after the attention mechanism is used, the weighted aggregate feature corresponding to the second sample image is obtained, that is, the feature representation of the second sample image i is obtained by using the attention mechanism based on the sixth feature output by each teacher model and the seventh feature output by the student model. j indicates other sample images except the second sample image.
[0103] It can be seen that when the distillation loss is determined according to the seventh feature, the weighted aggregate feature and the eighth features, and the parameters of the student model are adjusted according to the distillation loss, the parameters of the student model are adjusted with the goal of maximizing the similarity between the seventh feature and the weighted aggregate feature, and with the goal of minimizing the similarity between the seventh feature and the eighth features.
[0104] Of course, when determining the distillation loss, the attention mechanism can be omitted and only the current knowledge distillation technology can be used. The attention mechanism enables the central node to weight the feature representation capabilities of each edge node, thereby further improving the feature representation capabilities of the global encoder trained by the central node, and further improving the feature representation capabilities of the local encoder trained by each edge node.
[0105] Furthermore, when adjusting the parameters of the student model, the central node can also receive the local loss sent by each edge node, and adjust the parameters of the student model according to the local loss and the distillation loss. Among them, the local loss is the loss of the local encoding end obtained by the edge node. As mentioned above, the local encoding end is obtained by the edge node based on the local sample data of the edge node, and the initial encoding end is trained. Therefore, when each edge node obtains the local encoding end, there is a local loss of training the initial encoding end to the local encoding end.
[0106] Based on the training method of the image classification model described above, the embodiment of this specification also provides a schematic diagram of a training device for an image classification model, such as Figure 5 shown.
[0107] Figure 5 A pre-training device for a language model provided in an embodiment of this specification is located at a central node and is used to train an encoding end in an image classification model; the device specifically includes:
[0108] An acquisition module 500 is used to acquire parameters of a local encoding end of an image classification model deployed in each edge node;
[0109] A distillation module 502 is used to distill the global encoding end of the image classification model deployed in the central node according to the parameters of each local encoding end to obtain the parameters of the global encoding end after distillation;
[0110] An adjustment module 504 is used to adjust the parameters of the local encoding end corresponding to each edge node according to the parameters of the global encoding end after distillation and the preset training target, so as to obtain the adjusted parameters of the local encoding end corresponding to the edge node;
[0111] The sending module 506 is used to send the adjusted parameters of the local encoding end corresponding to the edge node to the edge node, so that the edge node obtains the trained image classification model corresponding to the edge node according to the encoding end constructed based on the adjusted parameters of the local encoding end.
[0112] Optionally, the image classification model deployed in each edge node includes a first online encoding end and a first target encoding end, and the first online encoding end and the first target encoding end are asymmetric twin networks;
[0113] The acquisition module 500 is specifically used for: the edge node determines a first enhanced image of the first sample image for each first sample image pre-stored in the edge node; the edge node inputs the first sample image into the first online encoding end to obtain a first feature, and inputs the first enhanced image into the first target encoding end to obtain a second feature; the edge node trains the first online encoding end with the goal of maximizing the similarity between the first feature and the second feature to obtain the trained first online encoding end, and uses the trained first online encoding end as a local encoding end; based on the local encoding end, the edge node obtains the parameters of the local encoding end corresponding to the edge node.
[0114] Optionally, the image classification model deployed in each edge node further includes a prediction end, and the prediction end is connected to the first online encoding end;
[0115] The acquisition module 500 is specifically configured to: the edge node inputs the first sample image into the first online encoding end to obtain a first feature, and inputs the first feature into the prediction end to obtain a third feature;
[0116] The acquisition module 500 is specifically configured to enable the edge node to train the first online encoding end with the goal of maximizing the similarity between the third feature and the second feature.
[0117] Optionally, the acquisition module 500 is further configured to train the first target encoding end by adopting an exponential moving average algorithm according to the first online encoding end.
[0118] Optionally, the adjustment module 504 is specifically used to use the encoding end constructed based on the parameters of the global encoding end after the distillation as the second target encoding end, and the encoding end constructed based on the parameters of the local encoding end corresponding to the edge node as the second online encoding end; and, for each second sample image pre-stored in the central node, determine the second enhanced image corresponding to the second sample image; input the second enhanced image into the second online encoding end to obtain a fourth feature, and input the second sample image into the second target encoding end to obtain a fifth feature; with the goal of maximizing the similarity between the fourth feature and the fifth feature, adjust the parameters of the second online encoding end to obtain the adjusted parameters of the second online encoding end; according to the adjusted parameters of the second online encoding end, obtain the adjusted parameters of the local encoding end corresponding to the edge node.
[0119] Optionally, the distillation module 502 is specifically used to use the encoding end constructed based on the parameters of each local encoding end as a teacher model and the global encoding end as a student model; determine the distillation loss according to each teacher model and the student model; adjust the parameters of the student model according to the distillation loss to obtain the student model after the parameters are adjusted; and obtain the parameters of the global encoding end after distillation according to the student model after the parameters are adjusted.
[0120] Optionally, the distillation module 502 is specifically used to, for each second sample image pre-stored in the central node, input the second sample image into each teacher model respectively to obtain each sixth feature; and input the second sample image into the student model to obtain the seventh feature, and input each other sample image except the second sample image into the student model respectively to obtain each eighth feature; according to the sixth features and the seventh features, using the attention mechanism, determine the weighted aggregation feature corresponding to the second sample image; according to the seventh feature, the weighted aggregation feature and the eighth features, determine the distillation loss.
[0121] Optionally, the distillation module 502 is specifically used to receive local losses sent by each edge node, where the local losses are losses of a local encoding end of edge node training; and adjust parameters of the student model according to each local loss and the distillation loss.
[0122] The embodiments of this specification also provide a computer-readable storage medium, which stores a computer program. The computer program can be used to execute the training method of the image classification model described above.
[0123] Based on the training method of the image classification model described above, this specification also proposes Figure 6 The schematic structure diagram of the electronic device shown in FIG. Figure 6At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the training method of the image classification model described above.
[0124] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0125] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.
[0126] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.
[0127] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0128] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0129] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0130] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0131] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0133] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0134] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0135] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0136] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0137] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0138] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0139] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0140] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.
Claims
1. A training method for an image classification model, the method being applied to a central node and used to train an encoding end in the image classification model; the method comprising: Obtain the parameters of the local encoding end of the image classification model deployed in each edge node; According to the parameters of each local encoder, the global encoder of the image classification model deployed in the central node is distilled to obtain the parameters of the global encoder after distillation; For each edge node, according to the parameters of the global encoding end after distillation and the preset training target, the parameters of the local encoding end corresponding to the edge node are adjusted to obtain the adjusted parameters of the local encoding end corresponding to the edge node; The obtained adjusted parameters of the local encoding end corresponding to the edge node are sent to the edge node, so that the edge node obtains the trained image classification model corresponding to the edge node according to the encoding end constructed based on the adjusted parameters of the local encoding end.
2. The method according to claim 1, wherein the image classification model deployed in each edge node comprises a first online encoding end and a first target encoding end, and the first online encoding end and the first target encoding end are asymmetric twin networks; For each edge node, the edge node determines the parameters of the local encoding end of the image classification model deployed in the edge node, specifically including: The edge node determines, for each first sample image pre-stored in the edge node, a first enhanced image of the first sample image; The edge node inputs the first sample image into the first online encoding end to obtain a first feature, and inputs the first enhanced image into the first target encoding end to obtain a second feature; The edge node trains the first online encoder with the goal of maximizing the similarity between the first feature and the second feature to obtain a trained first online encoder, and uses the trained first online encoder as a local encoder; According to the local encoding end, the edge node obtains the parameters of the local encoding end corresponding to the edge node.
3. The method according to claim 2, wherein the image classification model deployed in each edge node further comprises a prediction end, and the prediction end is connected to the first online encoding end; The edge node inputs the first sample image into the first online encoding end to obtain a first feature, specifically including: The edge node inputs the first sample image into the first online encoding end to obtain a first feature, and inputs the first feature into the prediction end to obtain a third feature; Training the first online encoder includes: The edge node trains the first online encoding end with the goal of maximizing the similarity between the third feature and the second feature.
4. The method of claim 2, further comprising: According to the first online encoder, the first target encoder is trained using an exponential moving average algorithm.
5. The method according to claim 1, wherein the parameters of the local encoding end corresponding to the edge node are adjusted to obtain the adjusted parameters of the local encoding end corresponding to the edge node, specifically comprising: Using an encoder constructed based on the parameters of the global encoder after distillation as a second target encoder, and using an encoder constructed based on the parameters of the local encoder corresponding to the edge node as a second online encoder; and, for each second sample image pre-stored in the central node, determining a second enhanced image corresponding to the second sample image; Inputting the second enhanced image to the second online encoding end to obtain a fourth feature, and inputting the second sample image to the second target encoding end to obtain a fifth feature; With the goal of maximizing the similarity between the fourth feature and the fifth feature, adjusting the parameters of the second online encoding end to obtain adjusted parameters of the second online encoding end; According to the adjusted parameters of the second online encoding end, the adjusted parameters of the local encoding end corresponding to the edge node are obtained.
6. The method according to claim 1, wherein the global encoder of the image classification model deployed in the central node is distilled to obtain parameters of the global encoder after distillation, specifically comprising: The encoding ends constructed based on the parameters of the local encoding ends are used as teacher models, and the global encoding end is used as the student model; Determining a distillation loss based on each teacher model and the student model; According to the distillation loss, adjusting the parameters of the student model to obtain a student model with adjusted parameters; According to the student model after parameter adjustment, the parameters of the global encoding end after distillation are obtained.
7. The method according to claim 6, determining the distillation loss according to each teacher model and the student model, specifically comprising: For each second sample image pre-stored in the central node, the second sample image is input into each teacher model to obtain each sixth feature; and inputting the second sample image into the student model to obtain the seventh feature, and inputting each other sample image except the second sample image into the student model to obtain each eighth feature; According to the sixth features and the seventh feature, using an attention mechanism, determining a weighted aggregate feature corresponding to the second sample image; A distillation loss is determined based on the seventh feature, the weighted aggregate feature, and the eighth features.
8. The method of claim 6, wherein adjusting the parameters of the student model comprises: Receiving a local loss sent by each edge node, where the local loss is a loss of a local encoding end of training of the edge node; The parameters of the student model are adjusted according to the local losses and the distillation loss.
9. A training device for an image classification model, the device being located at a central node and used for training an encoding end in an image classification model; the device specifically comprises: An acquisition module, used to acquire parameters of the local encoding end of the image classification model deployed in each edge node; A distillation module is used to distill the global encoder of the image classification model deployed in the central node according to the parameters of each local encoder to obtain the parameters of the global encoder after distillation; An adjustment module is used to adjust the parameters of the local encoding end corresponding to each edge node according to the parameters of the global encoding end after distillation and the preset training target for each edge node, so as to obtain the adjusted parameters of the local encoding end corresponding to the edge node; The sending module is used to send the obtained adjusted local encoding end parameters corresponding to the edge node to the edge node, so that the edge node obtains the trained image classification model according to the encoding end constructed based on the adjusted local encoding end parameters.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 8.
11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the program.