Edge - side Model Update Method, Device, Equipment and Medium Based on Federated Learning

Abstract

The present invention discloses an edge model update method, apparatus, device and medium based on federated learning, comprising: performing knowledge distillation on the edge model according to an initial data set and a cloud model to obtain an initial edge model; calculating the inference confidence and inference results of the initial edge model on each verification sample, and obtaining an inference error probability model based on Gaussian mixture model fitting; each edge node performs inference prediction on multiple edge samples according to the initial edge model to obtain sample confidence; determining the inference error probability of each edge sample, screening out difficult samples, obtaining feature maps corresponding to the difficult samples and uploading them to the cloud; aggregating the feature maps to obtain a feature map set, and performing knowledge distillation on the initial edge model according to the feature map set and the cloud model to obtain a target edge model. The present invention reduces the computing cost of edge devices and enhances the data security of edge devices, and can be applied to the field of artificial intelligence technology.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an edge model updating method, apparatus, device, and medium based on federated learning. Background Art

[0002] In today's intelligent era, machine learning technology has become a key force driving progress in numerous fields. In various applications, model performance often directly depends on the quality, quantity, and diversity of the data used. Traditional machine learning paradigms rely on aggregating massive amounts of data onto a central server to train models. The hope is that by learning from this vast amount of rich data, the model will be able to capture complex patterns and regularities within the data, thereby achieving accurate predictions. However, data distribution in the real world is highly fragmented. Strict data privacy regulations and the strong demands of data owners for data security and privacy protection make centralized data collection and processing difficult. This fragmented and isolated data has become a major bottleneck hindering the further improvement of machine learning model performance and its widespread application. An innovative solution is urgently needed to overcome this impasse, fully tapping and leveraging the value of distributed data while protecting data privacy, thereby promoting the in-depth development and effective application of machine learning technology across various industries.

[0003] Federated learning has emerged as a distributed machine learning method to address the problem of data silos. It aims to allow various data holders to collaboratively train a shared global model without directly exchanging original data, so as to fully utilize data resources while protecting data privacy. For example, patent CN118381600B "Federated Learning Privacy Protection Method and System" uses federated learning, homomorphic encryption and blockchain technology to transmit the client's model update gradient back to the central server in an encrypted manner to solve the gradient leakage problem that may exist in federated learning. Patent CN118428491B "A Federated Learning Method for Distributed Data Resources under Privacy Protection Constraints" calculates the Euclidean distance of each client data set to calculate the weights during federated learning weighted aggregation to better solve the problem of client data heterogeneity. In essence, these patented technologies all use federated learning to resolve the contradiction between AI technology's demand for data diversity and data privacy.

[0004] Hospitals and banks are typical examples of scenarios where federated learning is suitable. On the one hand, their data is highly private and difficult to fully share across different entities, requiring federated learning to fully utilize this data. On the other hand, these institutions also have ample high-quality data (including annotations) and sufficient computing resources for local training, which meets the requirements of federated learning. However, while federated learning has shown great potential, it currently has some significant drawbacks, especially when the client is a much smaller entity than a hospital or bank.

[0005] On the one hand, in many practical application scenarios, clients often struggle to generate high-quality data annotations for model training. For example, in the Internet of Things (IoT) environment, numerous edge devices, such as smart home sensors and monitoring equipment on industrial production lines, can collect large amounts of raw data. However, due to their limited computing power and storage resources, these devices struggle to run complex data annotation algorithms. Furthermore, they lack the specialized knowledge and manpower to accurately annotate data. As a result, during federated learning, the training data may be inaccurately or incompletely labeled, which in turn affects the performance and convergence speed of the global model, and may even make local training impossible.

[0006] Another key issue is the limited computing power of edge devices. Unlike data centers with powerful computing resources, edge devices typically utilize low-power, low-cost hardware designs, and their computing power is far inferior to that of traditional servers. When training models for federated learning, complex model calculations and frequent parameter updates place a heavy computational burden on edge devices, resulting in slow training and even possible interruptions due to device resource exhaustion. This severely hinders the widespread deployment and efficient operation of federated learning in real-world scenarios.

[0007] The above problems need to be solved urgently. Summary of the Invention

[0008] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0009] To this end, one purpose of an embodiment of the present invention is to provide an edge model update method based on federated learning, which can protect data privacy while making full use of edge data to improve edge model performance. Compared with traditional federated learning methods, there is no need to deploy labeled data at the edge, and there is no need for edge hardware to support backpropagation training, which reduces the computing cost of edge devices and enhances the data security of edge devices.

[0010] Another object of an embodiment of the present invention is to provide an edge model updating device based on federated learning.

[0011] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0012] On the one hand, an embodiment of the present invention provides an edge model update method based on federated learning, comprising the following steps:

[0013] Training a cloud model based on an initial data set, initializing an edge model based on the cloud model, and performing knowledge distillation on the edge model based on the initial data set and the cloud model to obtain an initial edge model;

[0014] Counting the inference confidence and inference results of the initial edge model on each verification sample in a preset verification set, fitting the inference confidence and the inference results based on a Gaussian mixture model to obtain an inference error probability model;

[0015] Sending the initial edge model to each edge node, so that each edge node performs inference and prediction on multiple edge samples according to the initial edge model to obtain a sample confidence of each edge sample;

[0016] Determining an inference error probability of each edge sample according to the inference error probability model and the sample confidence, screening out difficult samples from the edge samples according to the inference error probability, and then obtaining a feature map corresponding to the difficult sample, and uploading the feature map to the cloud;

[0017] Aggregating the feature graphs uploaded by each edge node to obtain a feature graph set, and performing knowledge distillation on the initial edge model based on the feature graph set and the cloud model to obtain a target edge model;

[0018] The inference result is correct or incorrect, and the inference error probability model is used to characterize the probability of inference error under different confidence levels.

[0019] Furthermore, in one embodiment of the present invention, the cloud model is trained based on the initial data set, the edge model is initialized based on the cloud model, and knowledge distillation is performed on the edge model based on the initial data set and the cloud model to obtain the initial edge model, which specifically includes:

[0020] Acquire the initial data set, where the initial data set includes a plurality of initial samples and corresponding hard labels;

[0021] Training an initial cloud model according to the initial data set to obtain the cloud model;

[0022] Obtain target model parameters of a first feature extraction layer of the cloud model, and initialize the edge model according to the target model parameters to obtain a first feature extraction layer identical to that of the cloud model;

[0023] Determine a soft label for each initial sample in the initial data set according to the cloud model;

[0024] Inputting the initial sample into the edge model to obtain an initial result;

[0025] determining a first loss value based on the initial result, the hard label, and the soft label;

[0026] Keeping parameters of the first feature extraction layer of the edge model unchanged, updating other parameters of the edge model according to the first loss value, and obtaining the initial edge model.

[0027] Furthermore, in one embodiment of the present invention, the inference confidence and inference results of the initial edge model on each verification sample of a preset verification set are statistically analyzed, and the inference confidence and the inference results are fitted based on a Gaussian mixture model to obtain an inference error probability model, which specifically includes:

[0028] Inputting the verification sample into the initial edge model to obtain a verification result and the inference confidence;

[0029] Determining the inference result according to the verification result and the label of the verification sample, and constructing a binary array according to the inference confidence and the inference result;

[0030] Initialize a Gaussian mixture model, and iteratively calculate parameters of the Gaussian mixture model according to the binary array to obtain the inference error probability model.

[0031] Furthermore, in one embodiment of the present invention, the inference prediction of multiple edge samples based on the initial edge model to obtain the sample confidence of each edge sample specifically includes:

[0032] Acquire the edge samples stored locally on the edge node, where the edge samples are unlabeled data;

[0033] The edge samples are input into the initial edge model to obtain the corresponding sample confidence.

[0034] Furthermore, in one embodiment of the present invention, determining the inference error probability of each edge sample based on the inference error probability model and the sample confidence, screening out difficult samples from the edge samples based on the inference error probability, and then obtaining feature maps corresponding to the difficult samples, specifically includes:

[0035] Inputting the sample confidence into the inference error probability model to calculate the inference error probability corresponding to the edge sample;

[0036] When the inference error probability is greater than a preset first threshold, determining the corresponding edge sample as the difficult sample;

[0037] The feature map obtained after the difficult sample passes through the first feature extraction layer of the initial edge model is obtained.

[0038] Furthermore, in one embodiment of the present invention, the feature graphs uploaded by each edge node are aggregated to obtain a feature graph set, and knowledge distillation is performed on the initial edge model based on the feature graph set and the cloud model to obtain a target edge model, which specifically includes:

[0039] When a preset interval length is reached or the number of feature maps reaches a preset number, the feature maps are aggregated to obtain a feature map set;

[0040] Determining a feature soft label for each feature graph in the feature graph set according to the cloud model;

[0041] Inputting the feature map into the initial edge model to obtain a feature inference result;

[0042] Determine a second loss value according to the feature inference result and the feature soft label;

[0043] Keeping the parameters of the first feature extraction layer of the initial edge model unchanged, updating other parameters of the initial edge model according to the second loss value, and obtaining the target edge model.

[0044] On the other hand, an embodiment of the present invention provides an edge model updating device based on federated learning, including:

[0045] An initial model training module is used to train a cloud model based on an initial data set, initialize an edge model based on the cloud model, and perform knowledge distillation on the edge model based on the initial data set and the cloud model to obtain an initial edge model;

[0046] A probability model fitting module is used to count the inference confidence and inference results of the initial edge model on each verification sample in a preset verification set, and fit the inference confidence and the inference results based on a Gaussian mixture model to obtain an inference error probability model;

[0047] An edge reasoning module is used to send the initial edge model to each edge node, so that each edge node performs inference and prediction on multiple edge samples according to the initial edge model to obtain a sample confidence of each edge sample;

[0048] A difficult sample screening module is used to determine the inference error probability of each edge sample based on the inference error probability model and the sample confidence, screen out difficult samples from the edge samples based on the inference error probability, obtain feature maps corresponding to the difficult samples, and upload the feature maps to the cloud;

[0049] A cloud-side update module is configured to aggregate the feature graphs uploaded by each edge node to obtain a feature graph set, and perform knowledge distillation on the initial edge model based on the feature graph set and the cloud-side model to obtain a target edge model;

[0050] The inference result is correct or incorrect, and the inference error probability model is used to characterize the probability of inference error under different confidence levels.

[0051] On the other hand, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, a program stored on the memory and runnable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the edge model update method based on federated learning as described above is implemented.

[0052] On the other hand, an embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the edge model update method based on federated learning as described above.

[0053] On the other hand, an embodiment of the present invention further provides a washing machine, which includes the edge model updating device or electronic device based on federated learning as described above.

[0054] The advantages and benefits of the present invention will be described in part in the following description and will become apparent from the following description or learned through practice of the present invention:

[0055] In an embodiment of the present invention, a cloud model is trained based on an initial data set, an edge model is initialized based on the cloud model, and knowledge distillation is performed on the edge model based on the initial data set and the cloud model to obtain an initial edge model. The inference confidence and inference results of the initial edge model on each verification sample in a preset verification set are statistically analyzed. The inference confidence and inference results are fitted based on a Gaussian mixture model to obtain an inference error probability model. The initial edge model is distributed to each edge node, so that each edge node performs inference and prediction on multiple edge samples based on the initial edge model to obtain a sample confidence of each edge sample. The inference error probability of each edge sample is determined based on the inference error probability model and the sample confidence. Difficult samples are screened from the edge samples based on the inference error probability, and feature maps corresponding to the difficult samples are obtained and uploaded to the cloud. The feature maps uploaded by each edge node are aggregated to obtain a feature map set. Knowledge distillation is performed on the initial edge model based on the feature map set and the cloud model to obtain a target edge model. The inference result is either a correct inference or an inference error, and the inference error probability model is used to characterize the probability of inference error at different confidence levels. The embodiments of the present invention can protect data privacy while fully utilizing edge data to improve edge model performance. Compared with traditional federated learning methods, there is no need to deploy labeled data at the edge, and no edge hardware is required to support backpropagation training, which reduces the computing cost of edge devices and enhances the data security of edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0057] Figure 1 A flowchart of a method for updating an edge model based on federated learning provided by an embodiment of the present invention;

[0058] Figure 2 A data interaction diagram of the edge model update method based on federated learning provided by an embodiment of the present invention;

[0059] Figure 3 A flowchart of step S101 provided in an embodiment of the present invention;

[0060] Figure 4 A flowchart of step S102 provided in an embodiment of the present invention;

[0061] Figure 5A flowchart of step S103 provided in an embodiment of the present invention;

[0062] Figure 6 A flowchart of step S104 provided in an embodiment of the present invention;

[0063] Figure 7 A flowchart of step S105 provided in an embodiment of the present invention;

[0064] Figure 8 A schematic diagram of the structure of an edge model updating device based on federated learning provided by an embodiment of the present invention;

[0065] Figure 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention;

[0066] Figure 10 A schematic diagram of the structure of a storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limitations on the present application. It should be noted that, although the functional modules are divided in the system schematic and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the system schematic or the order in the flow chart. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and no limitation is placed on the order between the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0068] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of "first" or "second", it is only used to distinguish technical features and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in this document are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0069] The edge model update method based on federated learning provided in the embodiments of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running on the terminal or the server side. In some embodiments, the terminal can be a smart phone, tablet computer, laptop computer, desktop computer, set-top box, etc.; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the edge model update method based on federated learning, etc., but is not limited to the above forms.

[0070] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0071] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0072] An AI washing machine is an intelligent washing machine equipped with AI capabilities. It can select the appropriate wash method based on the user's laundry input, including detergent type and dosage, water volume, water temperature, and wash cycle. The core function is to use a camera to identify the user's laundry, including type, color, material, type and degree of soiling. AI washing machines achieve this function through AI models. The performance of AI models is often directly dependent on the quality, quantity, and diversity of the data used. Generally speaking, machine learning relies on the aggregation of massive datasets onto a central server to train the model. By learning from this rich data, the model can capture complex patterns and regularities in the data, thereby achieving accurate predictions. For highly personalized items like clothing, it is difficult to collect sufficient and diverse data all at once. Therefore, it is often necessary for the AI model to be able to iterate over time and accumulate data.

[0073] However, real-world data distribution is highly fragmented. Strict data privacy regulations and the strong desire of data owners to protect their own data security and privacy make centralized data collection and processing difficult. This data challenge is particularly acute for AI-powered washing machines, which process user clothing.

[0074] Patent CN115287869A, "Washing Machine and Detergent Dispensing Method," acquires data by directly uploading user clothing images to a server via a terminal. This approach of directly collecting data from users can address data diversity issues to a certain extent, but it neglects user data privacy.

[0075] Patent CN116356524A "A washing machine control method, system and smart washing machine capable of preventing color mixing" uploads clothing pictures through an app, and the user uploads the pictures through the app. Although it prevents the user's private data from being collected without their knowledge to a certain extent, on the one hand, the user may not be fully aware that their private data will be uploaded to the cloud. On the other hand, it also increases the user's extra operations, bringing a negative product experience.

[0076] Patent CN119312400A, "A Data Privacy Protection Method and Medical Service System in a Federated Learning Framework," uses a federated learning framework to protect user privacy and data security. However, federated learning requires AI model training on each terminal and aggregating the gradients generated by each terminal's training to update the model. This requires hardware computing power and data annotation at the terminal for training, which are difficult to meet in the AI washing machine scenario.

[0077] The purpose of this invention is to propose an edge model update method based on federated learning, which protects data privacy while fully utilizing edge client data to improve the performance of edge client AI models. Compared with traditional federated learning methods, the method proposed in this invention does not require labeled data on the client, nor does it require client hardware to support backpropagation training, which is more suitable for the use scenario of AI washing machines.

[0078] like Figure 1 The figure shows a flow chart of the steps of the edge model update method based on federated learning provided by an embodiment of the present invention, referring to Figure 1 , an embodiment of the present invention provides an edge model updating method based on federated learning, which specifically includes the following steps:

[0079] S101. Train a cloud model based on the initial data set, initialize an edge model based on the cloud model, and perform knowledge distillation on the edge model based on the initial data set and the cloud model to obtain an initial edge model.

[0080] S102: Calculate the inference confidence and inference results of the initial edge model on each verification sample in the preset verification set, and fit the inference confidence and inference results based on a Gaussian mixture model to obtain an inference error probability model;

[0081] S103: Send the initial edge model to each edge node, so that each edge node performs inference and prediction on multiple edge samples based on the initial edge model to obtain the sample confidence of each edge sample;

[0082] S104: Determine the inference error probability of each edge sample based on the inference error probability model and the sample confidence, filter out difficult samples from the edge samples based on the inference error probability, obtain feature maps corresponding to the difficult samples, and upload the feature maps to the cloud;

[0083] S105: Aggregate the feature graphs uploaded by each edge node to obtain a feature graph set, and perform knowledge distillation on the initial edge model based on the feature graph set and the cloud model to obtain a target edge model;

[0084] The inference result is correct or incorrect, and the inference error probability model is used to characterize the probability of inference error under different confidence levels.

[0085] Specifically, if Figure 2 The data interaction diagram of the edge model update method based on federated learning provided by an embodiment of the present invention is shown in FIG. Figure 2 , a complete process of an embodiment of the present invention is as follows:

[0086] 1) Use the initial dataset S0 to train the cloud model M in the cloudC , based on cloud model M C Initialize the edge model M E , where M E The number and size of parameters of the first feature extraction layer l0 are set to and M C same;

[0087] 2) Using the initial dataset S0 and the cloud model M C For edge models M with smaller parameters E Perform knowledge distillation to obtain the initial edge model M E 0 , M during distillation E The parameters of the first feature extraction layer l0 are fixed;

[0088] 3) Count the initial edge model M on the validation set E 0 The confidence of the inference result and the corresponding label result are modeled using a Gaussian mixture model (GMM);

[0089] 4) Initial edge model M E 0 Send it to the edge nodes of each scenario and perform inference and prediction;

[0090] 5) During a period of time, each edge end E i For each inference result confidence, the probability of inference error is calculated using the mixed Gaussian model in step 3, and the initial input of the inference event with an error probability greater than the threshold γ is recorded after passing through the initial edge model M E 0 The set S of feature maps obtained by the first feature extraction layer l0 E0 i , and upload it to the cloud;

[0091] 6) The cloud aggregates the feature graph set S from each edge terminal E0 i Get the total set S E0 , and use it as the cloud model M C The first feature extraction layer l0 is the input of the next layer l1, and uses M C The output of the soft label distillation initial edge model M E 0 Get M E 1 , where M E 1 The l0 remains unchanged;

[0092] 7) Repeat steps 4-6 several times until the preset convergence condition is reached and the edge model M is sent down. En To the edge node.

[0093] It can be recognized that the update of the edge model in the embodiment of the present invention is completed in the cloud, and the edge node only needs to screen out difficult samples with a high probability of inference errors and return the feature maps corresponding to the difficult samples to the cloud. It can protect data privacy under the premise of fully utilizing edge data to improve the performance of the edge model. Compared with traditional federated learning methods, there is no need to deploy labeled data at the edge, and there is no need for edge hardware to support backpropagation training, which reduces the computing cost of edge devices and enhances the data security of edge devices.

[0094] The specific implementation process of the embodiment of the present invention is further described below with reference to the accompanying drawings.

[0095] like Figure 3 FIG. 1 is a flowchart of step S101 provided in an embodiment of the present invention, referring to FIG. Figure 3 As an optional implementation, a cloud model is trained based on the initial dataset, an edge model is initialized based on the cloud model, and knowledge distillation is performed on the edge model based on the initial dataset and the cloud model to obtain an initial edge model, which specifically includes:

[0096] S1011. Obtain an initial data set, where the initial data set includes multiple initial samples and corresponding hard labels;

[0097] S1012. Train the initial cloud model according to the initial data set to obtain a cloud model;

[0098] S1013. Obtain target model parameters of the first feature extraction layer of the cloud model, and initialize the edge model with the same first feature extraction layer as the cloud model according to the target model parameters;

[0099] S1014. Determine the soft label of each initial sample in the initial data set according to the cloud model;

[0100] S1015: Input the initial sample into the edge model to obtain an initial result;

[0101] S1016. Determine a first loss value based on the initial result, the hard label, and the soft label;

[0102] S1017. Keep the parameters of the first feature extraction layer of the edge model unchanged, update other parameters of the edge model according to the first loss value, and obtain an initial edge model.

[0103] Specifically, randomly initialize the cloud model M with a large number of parameters C , use the pre-prepared initial dataset S0 to train M CAfter the training is completed, use M C The parameter weights of the first feature extraction layer l0 are initialized to obtain the first feature extraction layer and M C The same edge model M E , and using datasets S0 and M C To M E Perform knowledge distillation and distill out the frozen M E Except for the parameter weight of l0, the process and parameter selection principles of general knowledge distillation are consistent.

[0104] It should be noted that the cloud model M C The output of contains not only the final prediction result (hard label), but also rich probability distribution information (soft label), combined with soft label loss (such as KL divergence) and hard label cross entropy loss, to balance the cloud model M C The teacher knowledge and real label supervision, edge model M E Training is performed by minimizing the following objective:

[0105] Loss1= α 1×L hard + β 1×L soft1

[0106] Among them, L hard is the cross entropy loss of the true label (hard label), L soft1 is the KL divergence loss of the soft label, α 1 and β 1 is the preset weight parameter.

[0107] like Figure 4 FIG. 1 is a flowchart of step S102 provided in an embodiment of the present invention, referring to FIG. Figure 4 As an optional implementation, the inference confidence and inference results of the initial edge model on each verification sample in the preset verification set are statistically analyzed, and the inference confidence and inference results are fitted based on a Gaussian mixture model to obtain an inference error probability model, which specifically includes:

[0108] S1021. Input the verification sample into the initial edge model to obtain the verification result and reasoning confidence;

[0109] S1022. Determine an inference result based on the verification result and the label of the verification sample, and construct a binary array based on the inference confidence and the inference result;

[0110] S1023. Initialize the Gaussian mixture model, and iteratively calculate the parameters of the Gaussian mixture model according to the binary array to obtain an inference error probability model.

[0111] Specifically, using distilled ME Perform inference on the validation dataset (with labels) and count <Conf i ,Y i >, where Conf i It's M E The confidence of inference on sample i, Y i Indicates whether the inference is correct. When the confidence distribution patterns corresponding to different categories are quite different, they can be counted separately. Finally, one or more Gaussian mixture models (GMM) are used to calculate the statistical<Conf, Y> The model is constructed by combining the two sets of data to obtain a probability model of the inference confidence distribution and the correctness of the inference result. This model will be used for difficult sample mining at the edge. The parameter calculation process of the Gaussian mixture model is not described here.

[0112] like Figure 5 FIG. 1 is a flowchart of step S103 provided in an embodiment of the present invention, referring to FIG. Figure 5 As an optional implementation, inference prediction is performed on multiple edge samples based on the initial edge model to obtain the sample confidence of each edge sample, which specifically includes:

[0113] S1031. Obtain edge samples stored locally at the edge node, where the edge samples are unlabeled data.

[0114] S1032: Input the edge samples into the initial edge model to obtain corresponding sample confidences.

[0115] like Figure 6 FIG. 1 is a flowchart of step S104 provided in an embodiment of the present invention, referring to FIG. Figure 6 As an optional implementation, the inference error probability of each edge sample is determined based on the inference error probability model and the sample confidence, and difficult samples are screened out from the edge samples based on the inference error probability, thereby obtaining feature maps corresponding to the difficult samples, which specifically includes:

[0116] S1041. Input the sample confidence into the inference error probability model to calculate the inference error probability corresponding to the edge sample;

[0117] S1042: When the inference error probability is greater than a preset first threshold, determine that the corresponding edge sample is a difficult sample;

[0118] S1043. Obtain a feature map of the difficult sample after it passes through the first feature extraction layer of the initial edge model.

[0119] Specifically, the threshold γ can be a fixed value or a relative value (for example, taking the N inferences with the highest error probability within a certain period of time), depending on the needs. For edge samples with an error probability greater than γ, the feature map obtained by passing it through the first feature extraction layer l0 of the model is recorded.

[0120] like Figure 7 FIG. 1 is a flowchart of step S105 provided in an embodiment of the present invention, referring to FIG. Figure 7 As an optional implementation, the feature graphs uploaded by each edge node are aggregated to obtain a feature graph set, and the initial edge model is subjected to knowledge distillation based on the feature graph set and the cloud model to obtain a target edge model, which specifically includes:

[0121] S1051. When a preset interval length is reached or a preset number of feature maps is reached, the feature maps are aggregated to obtain a feature map set;

[0122] S1052. Determine a feature soft label for each feature graph in the feature graph set based on the cloud model;

[0123] S1053: Input the feature map into the initial edge model to obtain a feature inference result;

[0124] S1054. Determine a second loss value based on the feature inference result and the feature soft label;

[0125] S1055: Keep the parameters of the first feature extraction layer of the initial edge model unchanged, update other parameters of the initial edge model according to the second loss value, and obtain the target edge model.

[0126] Specifically, the conditions for triggering a round of aggregation are flexible and diverse. They can be triggered by time, by the number of feature maps collected for potential erroneous inference, or by other specific conditions. After the aggregation is triggered, the cloud will use both the initial dataset S0 and the cloud model M C In the feature set S E0 The corresponding soft label on the original edge model M E Distill to get a new edge model M E 1 , due to S E0 There is no corresponding true label. When the input comes from S E0 When , the distillation loss function will only include the loss corresponding to the soft label, and during the distillation process, the l0 of the edge model remains unchanged. The distillation loss function for calculating the second loss value can be expressed as follows:

[0127] Loss2= α 2×L hard + β 2×L soft2

[0128] Among them, L soft2 is the KL divergence loss of the feature soft label, L hard is the cross entropy loss of the true label (hard label), α 2 and β 2 is the preset weight parameter, and α 2 is less than α 1, β 2 is greater than β 1, thus paying more attention to the loss corresponding to the feature soft labels when performing knowledge distillation on the initial edge model based on the feature map set and the cloud model.

[0129] It should be noted that since the feature graphs in the feature graph set are extracted by the first feature extraction layer of the initial edge model of the edge node, the edge node does not need to upload the original sample data of the edge to the cloud, and the edge does not need to update the edge model locally; for the reasoning of the feature graph, it is used as the cloud model M C The input of the next layer l1 of the first feature extraction layer l0 can obtain the corresponding feature soft label, which is also used as the initial edge model M in the process of knowledge distillation. E 0 The input of the next layer l2 of the first feature extraction layer l0 is used to obtain the feature inference result and calculate the second loss value in combination with the feature soft label. The other parameters of the initial edge model are updated according to the second loss value to obtain the target edge model.

[0130] The above describes the method steps of an embodiment of the present invention. It can be appreciated that, compared to traditional federated learning frameworks, the present invention does not require edge model training to generate parameter update gradients. The present invention uses a mixed Gaussian model to model model inference probability, constructing an indicator that can select interesting (potentially erroneous) samples at the edge, thereby implementing an online difficult sample mining method and significantly reducing the amount of data to be processed on the cloud. The present invention avoids large amounts of data labeling through knowledge distillation. By constructing cloud and edge models that share the first feature extraction layer, the present invention enables distillation of edge models using the cloud model without the need for raw data.

[0131] The present invention provides a method for collecting data from multiple edge devices and improving AI model performance while protecting user privacy. Generally, federated learning is used to address the aforementioned data privacy issues. However, federated learning requires model training on each client and uploading the updated gradients to the cloud for aggregation to obtain the final updated gradients. This requires the client to have both the computing power to support AI training and the data and labels required for AI training. This significantly limits the scope of application of federated learning.

[0132] The advantage of the present invention is that, compared to traditional federated learning methods, it eliminates the need for AI model training at the edge, significantly expanding the scope of federated learning. It is particularly suitable for edge computing scenarios with limited client resources (such as those in the Internet of Things). Unlike typical federated learning scenarios (e.g., between different entities like banks and hospitals), edge computing often relies on hardware capable of AI inference but unable to support AI training (both in terms of resources and architecture). Furthermore, many scenarios are unmanned, making it difficult to provide labeled data. Previously, such edge scenarios were limited to one of three options: forego online updates to edge models and endure delays in timely model iteration; sacrifice data privacy and upload raw edge data to the cloud in exchange for online model performance optimization; or allocate sufficient computing power and annotation resources at the edge, increasing costs in exchange for model updates and data privacy. The method proposed by the present invention balances model update timeliness and data privacy without increasing edge costs.

[0133] like Figure 8 The diagram shows the structure of the edge model updating device based on federated learning provided by an embodiment of the present invention, with reference to Figure 8 , an embodiment of the present invention provides an edge model updating device based on federated learning, comprising:

[0134] The initial model training module is used to train the cloud model based on the initial dataset, initialize the edge model based on the cloud model, and perform knowledge distillation on the edge model based on the initial dataset and the cloud model to obtain the initial edge model;

[0135] The probability model fitting module is used to calculate the inference confidence and inference results of the initial edge model on each verification sample in the preset verification set, and fit the inference confidence and inference results based on the Gaussian mixture model to obtain the inference error probability model;

[0136] The edge inference module is used to send the initial edge model to each edge node, so that each edge node can perform inference and prediction on multiple edge samples based on the initial edge model to obtain the sample confidence of each edge sample;

[0137] The difficult sample screening module is used to determine the inference error probability of each edge sample based on the inference error probability model and sample confidence, screen out difficult samples from the edge samples based on the inference error probability, and then obtain the feature map corresponding to the difficult samples and upload the feature map to the cloud;

[0138] The cloud update module is used to aggregate the feature maps uploaded by each edge node to obtain a feature map set, and perform knowledge distillation on the initial edge model based on the feature map set and the cloud model to obtain the target edge model;

[0139] The inference result is correct or incorrect, and the inference error probability model is used to characterize the probability of inference error under different confidence levels.

[0140] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0141] An embodiment of the present invention further provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the aforementioned method for updating edge models based on federated learning is implemented. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0142] like Figure 9 FIG2 is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention, referring to FIG2 Figure 9 , an embodiment of the present invention provides an electronic device, including:

[0143] The processor 901 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.

[0144] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called by the processor 901 to execute the edge model update method based on federated learning in the embodiments of the present invention.

[0145] Input / output interface 903, used to implement information input and output;

[0146] Communication interface 904, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0147] Bus 905 , which transmits information between various components of the device (e.g., processor 901 , memory 902 , input / output interface 903 , and communication interface 904 );

[0148] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0149] like Figure 10 FIG2 is a schematic diagram of the structure of the storage medium provided by the embodiment of the present invention, referring to FIG2 Figure 10 An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs 1001, and the one or more programs 1001 can be executed by one or more processors to implement the above-mentioned edge model update method based on federated learning.

[0150] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0151] An embodiment of the present invention also provides a washing machine, which includes the above-mentioned edge model updating device based on federated learning or the electric drive assembly of the electronic device.

[0152] Specifically, the washing machine may be a drum washing machine or a pulsator washing machine, which is deployed with an AI model for detecting and identifying clothes put in by the user.

[0153] This invention provides a washing machine that can collect data from an AI washing machine and improve AI model performance while protecting user privacy, achieving the following goals:

[0154] 1) Enhanced Privacy Protection: Through the federated learning framework, each washing machine only uploads feature maps processed by the first feature extraction layer, rather than raw images or sensor data. Feature maps cannot be restored to the original input, fundamentally mitigating the risk of privacy leaks.

[0155] 2) Zero training load on the washing machine: Model training tasks are completely eliminated on the washing machine side, requiring only inference and feature extraction. Using knowledge distillation technology, the cloud generates soft labels from aggregated feature maps, directly optimizing model parameters at the edge. This adapts to low-computing hardware and eliminates the need for additional data labeling. This reduces both hardware and data labeling costs for the AI washing machine.

[0156] 3) Dynamic Difficult Sample Mining: Based on real-time analysis of the inference confidence distribution using a Gaussian Mixture Model (GMM), low-confidence samples are automatically screened, and weak links in the model are optimized. This improves the AI model performance of the AI washing machine.

[0157] The embodiment of the present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The method shown.

[0158] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0159] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present invention set forth in the claims using ordinary skills without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0160] If the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the above methods in various embodiments of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0161] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0162] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable media on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other media, and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.

[0163] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0164] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0165] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0166] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for updating edge models based on federated learning, characterized in that: The following steps are involved: Training a cloud model based on an initial data set, initializing an edge model based on the cloud model, and performing knowledge distillation on the edge model based on the initial data set and the cloud model to obtain an initial edge model; Counting the inference confidence and inference results of the initial edge model on each verification sample in a preset verification set, fitting the inference confidence and the inference results based on a Gaussian mixture model to obtain an inference error probability model; Sending the initial edge model to each edge node, so that each edge node performs inference and prediction on multiple edge samples according to the initial edge model to obtain a sample confidence of each edge sample; Determining an inference error probability of each edge sample according to the inference error probability model and the sample confidence, screening out difficult samples from the edge samples according to the inference error probability, obtaining feature maps corresponding to the difficult samples, and uploading the feature maps to the cloud; Aggregating the feature graphs uploaded by each edge node to obtain a feature graph set, and performing knowledge distillation on the initial edge model based on the feature graph set and the cloud model to obtain a target edge model; Among them, the inference result is correct inference or incorrect inference, the inference error probability model is used to characterize the probability of inference error under different confidence levels, the edge node is an AI washing machine, the initial data set is a preset clothing image data set, the verification sample is a preset clothing image verification sample, the edge sample is a clothing image sample collected by the edge node, and the target edge model is an AI model for detecting and identifying clothing images.

2. The method for updating edge models based on federated learning according to claim 1, characterized in that: The method of training a cloud model based on an initial data set, initializing an edge model based on the cloud model, and performing knowledge distillation on the edge model based on the initial data set and the cloud model to obtain an initial edge model specifically includes: Acquire the initial data set, where the initial data set includes a plurality of initial samples and corresponding hard labels; Training an initial cloud model according to the initial data set to obtain the cloud model; Obtain target model parameters of a first feature extraction layer of the cloud model, and initialize the edge model according to the target model parameters to obtain a first feature extraction layer identical to that of the cloud model; Determining a soft label for each initial sample in the initial data set according to the cloud model; Inputting the initial sample into the edge model to obtain an initial result; determining a first loss value based on the initial result, the hard label, and the soft label; Keeping parameters of the first feature extraction layer of the edge model unchanged, updating other parameters of the edge model according to the first loss value, and obtaining the initial edge model.

3. The method for updating edge models based on federated learning according to claim 1, characterized in that: The statistical inference confidence and inference results of the initial edge model on each verification sample of the preset verification set are obtained, and the inference confidence and the inference results are fitted based on a Gaussian mixture model to obtain an inference error probability model, which specifically includes: Inputting the verification sample into the initial edge model to obtain a verification result and the inference confidence; Determining the inference result according to the verification result and the label of the verification sample, and constructing a binary array according to the inference confidence and the inference result; Initialize a Gaussian mixture model, and iteratively calculate parameters of the Gaussian mixture model according to the binary array to obtain the inference error probability model.

4. The method for updating edge models based on federated learning according to claim 1, characterized in that: The inference prediction of multiple edge samples based on the initial edge model to obtain the sample confidence of each edge sample specifically includes: Acquire the edge samples stored locally on the edge node, where the edge samples are unlabeled data; The edge samples are input into the initial edge model to obtain the corresponding sample confidence.

5. The method for updating edge models based on federated learning according to claim 1, characterized in that: The determining of the inference error probability of each edge sample according to the inference error probability model and the sample confidence, screening out difficult samples from the edge samples according to the inference error probability, and then obtaining feature maps corresponding to the difficult samples specifically includes: Inputting the sample confidence into the inference error probability model to calculate the inference error probability corresponding to the edge sample; When the inference error probability is greater than a preset first threshold, determining the corresponding edge sample as the difficult sample; The feature map obtained after the difficult sample passes through the first feature extraction layer of the initial edge model is obtained.

6. The method for updating an edge model based on federated learning according to any one of claims 1 to 5, characterized in that: The aggregating the feature graphs uploaded by each edge node to obtain a feature graph set, and performing knowledge distillation on the initial edge model based on the feature graph set and the cloud model to obtain a target edge model, which specifically includes: When a preset interval length is reached or the number of feature maps reaches a preset number, the feature maps are aggregated to obtain a feature map set; Determining a feature soft label for each feature graph in the feature graph set according to the cloud model; Inputting the feature map into the initial edge model to obtain a feature inference result; Determine a second loss value according to the feature inference result and the feature soft label; Keeping the parameters of the first feature extraction layer of the initial edge model unchanged, updating other parameters of the initial edge model according to the second loss value, and obtaining the target edge model.

7. An edge model updating device based on federated learning, characterized in that: include: An initial model training module is used to train a cloud model based on an initial data set, initialize an edge model based on the cloud model, and perform knowledge distillation on the edge model based on the initial data set and the cloud model to obtain an initial edge model; A probability model fitting module is used to count the inference confidence and inference results of the initial edge model on each verification sample in a preset verification set, and fit the inference confidence and the inference results based on a Gaussian mixture model to obtain an inference error probability model; An edge reasoning module is configured to send the initial edge model to each edge node, so that each edge node performs inference and prediction on multiple edge samples according to the initial edge model to obtain a sample confidence of each edge sample; A difficult sample screening module is used to determine the inference error probability of each edge sample based on the inference error probability model and the sample confidence, screen out difficult samples from the edge samples based on the inference error probability, obtain feature maps corresponding to the difficult samples, and upload the feature maps to the cloud; A cloud-side update module is configured to aggregate the feature graphs uploaded by each edge node to obtain a feature graph set, and perform knowledge distillation on the initial edge model based on the feature graph set and the cloud-side model to obtain a target edge model; Among them, the inference result is correct inference or incorrect inference, the inference error probability model is used to characterize the probability of inference error under different confidence levels, the edge node is an AI washing machine, the initial data set is a preset clothing image data set, the verification sample is a preset clothing image verification sample, the edge sample is a clothing image sample collected by the edge node, and the target edge model is an AI model for detecting and identifying clothing images.

8. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the edge model update method based on federated learning are realized as described in any one of claims 1 to 6.

9. A storage medium, which is a computer-readable storage medium and is used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the edge model updating method based on federated learning as described in any one of claims 1 to 6.

10. A washing machine, characterized in that: The washing machine includes the edge model updating device based on federated learning as described in claim 7 or the electronic device as described in claim 8.

Images