Federal learning method and device, client device and storage medium

Abstract

The invention discloses a federated learning method and device, client equipment and a storage medium, and the method comprises the steps: training a federated learning model based on a first training set, taking the trained federated learning model as a first model, obtaining a first test result of the first model on a test set, obtaining a first recognition accuracy rate corresponding to the abnormal category in the first test result; marking the abnormal categories of which the first recognition accuracy is lower than a threshold value as rare categories, and forming a second training set by a plurality of rare categories and a plurality of rare samples; inputting rare samples of the second training set into the first model; the first model is trained, when the average loss value is smaller than a preset loss value, training of the first model is stopped, and the trained first model serves as a second model; and when the second recognition accuracy corresponding to each rare category is greater than the threshold value and the set is an empty set, storing the second model. According to the invention, the recognition accuracy of the client device on the rare category can be improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a federated learning method, apparatus, client device and storage medium. Background Art

[0002] Federated learning builds a logical framework that protects data privacy and promotes intelligent collaboration. In the environment of federated learning, client devices play a vital role. Client devices usually have certain computing resources and storage capabilities. Through federated learning, these resources can be fully utilized.

[0003] However, when predicting or classifying, the recognition accuracy of the client device for rare categories is low. Therefore, how to improve the recognition accuracy of the client device for rare categories is an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide a federated learning method, apparatus, client device, and storage medium to solve the above-mentioned technical problem of how to improve the recognition accuracy of rare categories by client devices.

[0005] In a first aspect, an embodiment of the present application provides a federated learning method, which is applied to a client device, wherein the client device stores a federated learning model, and the federated learning method includes: Based on a preset first training set, the federated learning model is trained, the trained federated learning model is used as a first model, a first test result of the first model on the test set is obtained, and a first recognition accuracy rate corresponding to the abnormal category is obtained in the first test result; Marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, obtaining rare samples corresponding to the rare category, and combining a plurality of the rare categories and a plurality of the rare samples into a second training set; Inputting the rare samples of the second training set into the first model, obtaining a category probability distribution output by the first model based on the rare samples, and obtaining a predicted probability when a preset image of the rare samples belongs to the rare category in the category probability distribution; Obtain the total number of rare samples, obtain the number of rare samples corresponding to the rare category, the first proportional coefficient, and the second proportional coefficient, and obtain a preset value when the preset image of the rare sample belongs to the rare category, wherein the total number of rare samples is the sum of the number of rare samples corresponding to each of the rare categories; Generate an average loss value corresponding to the second training set according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value, and a preset average loss model; Based on the average loss value, the first model is trained; when the average loss value is less than a preset loss value, the training of the first model is stopped, and the trained first model is used as the second model; When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, the second model is saved.

[0006] Exemplarily, the average loss model is: ; Wherein, F is the average loss value; N is the total number of rare samples; i is the serial number of the rare sample; K is the total number of rare samples; c is the serial number of the rare category; is the number of rare samples of the cth abnormal category; is the first proportionality coefficient; is the second proportionality coefficient; is the predicted probability when the preset image of the i-th rare sample belongs to the c-th rare category; is the preset value when the preset image of the i-th rare sample belongs to the c-th rare category.

[0007] In a possible implementation manner of the first aspect, the federated learning model is trained based on a preset first training set, the trained federated learning model is used as a first model, a first test result of the first model on a test set is obtained, and before a first recognition accuracy rate corresponding to an abnormal category is obtained in the first test result, the federated learning method includes: Obtaining a first model parameter sent by a previous device, aggregating the first model parameter with a local model parameter to generate a second model parameter; Obtain the federated learning model using the second model parameters.

[0008] In a possible implementation manner of the first aspect, the method of training the federated learning model based on a preset first training set, using the trained federated learning model as a first model, obtaining a first test result of the first model on a test set, and obtaining a first recognition accuracy rate corresponding to an abnormal category in the first test result includes: Based on the preset first training set, the federated learning model is trained, and when the number of training rounds is greater than the preset number of rounds, the training of the federated learning model is stopped, and the trained federated learning model is used as the first model; Obtain the first test result of the first model on the test set, and obtain the first recognition accuracy rate corresponding to the abnormal category in the first test result.

[0009] In a possible implementation manner of the first aspect, the method of training the federated learning model based on a preset first training set, using the trained federated learning model as a first model, obtaining a first test result of the first model on a test set, and obtaining a first recognition accuracy rate corresponding to an abnormal category in the first test result includes: Based on the preset first training set, the federated learning model is trained, and when the loss function converges, the training of the federated learning model is stopped, and the trained federated learning model is used as the first model; Obtain the first test result of the first model on the test set, and obtain the first recognition accuracy rate corresponding to the abnormal category in the first test result.

[0010] In a possible implementation manner of the first aspect, marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, obtaining rare samples corresponding to the rare category, and forming a second training set with multiple rare categories and multiple rare samples, includes: Marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, and obtaining the preset abnormal sample corresponding to the abnormal category whose recognition accuracy is lower than the threshold; The preset abnormal samples are marked as rare samples corresponding to the rare categories, and a plurality of the rare categories and a plurality of the rare samples are combined into the second training set.

[0011] In a possible implementation manner of the first aspect, when the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, saving the second model includes: Obtaining a second test result of the second model on the test set; Obtaining a second recognition accuracy rate corresponding to each of the rare categories in the second test result, and excluding the rare categories whose second recognition accuracy rate is greater than the threshold in the set; When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, the second model is saved.

[0012] In a possible implementation manner of the first aspect, when the second recognition accuracy rates corresponding to each of the rare categories are greater than the threshold and the set is an empty set, after saving the second model, the federated learning method includes: Acquire a current image, and input the current image into the second model; The second model is used to identify the current image and the recognition result of the second model output is obtained. In a second aspect, an embodiment of the present application provides a federated learning device, which is applied to a client device, and the client device stores a federated learning model, including: A first acquisition module, configured to train the federated learning model based on a preset first training set, use the trained federated learning model as a first model, obtain a first test result of the first model on a test set, and obtain a first recognition accuracy rate corresponding to an abnormal category in the first test result; A second acquisition module is used to mark the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, add the rare category to a set, obtain rare samples corresponding to the rare category, and form a second training set with multiple rare categories and multiple rare samples; A third acquisition module is used to input the rare sample of the second training set into the first model, obtain the category probability distribution output by the first model based on the rare sample, and obtain the predicted probability when the preset image of the rare sample belongs to the rare category in the category probability distribution; A fourth acquisition module is used to acquire the total number of rare samples, acquire the number of rare samples corresponding to the rare category, the first proportional coefficient, and the second proportional coefficient, and acquire a preset value when the preset image of the rare sample belongs to the rare category, wherein the total number of rare samples is the sum of the number of rare samples corresponding to each of the rare categories; A generating module, configured to generate an average loss value corresponding to the second training set according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value, and a preset average loss model; A training module, used for training the first model based on the average loss value, and when the average loss value is less than a preset loss value, stopping the training of the first model and using the trained first model as the second model; A saving module is used to save the second model when the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set.

[0013] In a third aspect, an embodiment of the present application provides a client device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the federated learning method of the first aspect when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the federated learning method of the first aspect is implemented.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a client device, enables the client device to execute the federated learning method of the first aspect.

[0016] The beneficial effects of the embodiments of the present application lie in two aspects. On the one hand, the average loss value corresponding to the second training set is generated according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value and the preset average loss model. Since the first proportional coefficient and the second proportional coefficient can increase the proportion of rare categories in the average loss value, the first model can pay more attention to rare samples of these rare categories during the training process. On the other hand, the trained first model is used as the second model. When the second recognition accuracy corresponding to each of the rare categories is greater than the threshold and the set is an empty set, the second model is saved. Since the second recognition accuracy of the saved second model on each of the rare categories is greater than the threshold, the second model is used to help improve the recognition accuracy of the client device for the rare categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 An application scenario diagram of the federated learning method provided in the embodiment of the present application; Figure 2 is a flowchart of the federated learning method provided in the embodiment of the present application; Figure 3 A flowchart of saving a second model provided in an embodiment of the present application; Figure 4 A schematic block diagram of a federated learning device provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a client device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0020] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0021] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0022] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0023] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0024] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0026] In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0027] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0028] The federated learning method provided in the embodiments of the present application can be applied to client devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiments of the present application do not impose any restrictions on the specific types of client devices.

[0029] For example, the client device may be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a TV set top box (STB), a customer premises equipment (CPE) and / or other devices for communicating on a wireless system and a next generation communication system, such as a mobile terminal in a 5G network or a mobile terminal in a future evolved Public Land Mobile Network (PLMN) network, etc.

[0030] See also Figure 1 , Figure 1 The application scenario diagram of the federated learning method provided in the embodiment of the present application is described in detail as follows: M client devices are connected to form a ring network structure.

[0031] M is set by the user or by the system default and is not restricted here.

[0032] For the sake of illustration, an example is given below: For example, M is 6, and there are 6 client devices participating in federated learning, namely client device 1, client device 2, client device 3, client device 4, client device 5, and client device 6; The data of client device 1 is represented as D1. The initialized federated learning model is trained and updated using data D1. The obtained model parameter is V1, which is transmitted to client device 2. Client device 2 uses V1 as the initial model parameter of the device, and the data of client device 2 is represented by D2. V1 is trained and updated once using data D2, and the obtained model parameter is V2, which is transmitted to client device 3; Client device 3 uses V2 as the initial model parameter of the device. The data of client device 3 is represented by D3. V2 is trained and updated once using data D3. The obtained model parameter is V3, which is transmitted to client device 3. Client device 4 uses V3 as the initial model parameter of the device, and the data of client device 4 is represented by D4. V3 is trained and updated once using data D4, and the obtained model parameter is V4, which is transmitted to client device 4; Client device 5 uses V4 as the initial model parameter of the device, and the data of client device 5 is represented by D5. V4 is trained and updated once using data D5, and the obtained model parameter is V5, which is transmitted to client device 5; Client device 6 uses V5 as the initial model parameter of the device. The data of client device 6 is represented by D6. V5 is trained and updated once using data D6. The obtained model parameter is V6, which is transmitted to client device 6.

[0033] Among them, the initial model parameters of each client device are the model parameters after training of the previous client device. After traversing all client devices, the training order of all client devices is randomly sorted, and the above parameter transfer process is executed again in a loop until the federated learning model converges.

[0034] In the embodiment of the present application, due to the use of a ring network structure, the data is always retained on the client device and is not centrally stored in a certain central node, which is conducive to further protecting data privacy.

[0035] See also Figure 2 , Figure 2 It is a flow chart of the federated learning method provided in an embodiment of the present application. The method can be applied to a client device, and the client device stores a federated learning model.

[0036] like Figure 2 As shown, the federated learning method provided in the embodiment of the present application includes the following steps, which are described in detail as follows: S201, training the federated learning model based on a preset first training set, using the trained federated learning model as a first model, obtaining a first test result of the first model on a test set, and obtaining a first recognition accuracy rate corresponding to an abnormal category in the first test result; Wherein, before training the federated learning model based on the preset first training set, using the trained federated learning model as the first model, obtaining a first test result of the first model on the test set, and obtaining a first recognition accuracy rate corresponding to the abnormal category in the first test result, the federated learning method includes: Obtaining a first model parameter sent by a previous device, aggregating the first model parameter with a local model parameter to generate a second model parameter; Obtain the federated learning model using the second model parameters.

[0037] The method of training the federated learning model based on a preset first training set, using the trained federated learning model as a first model, obtaining a first test result of the first model on a test set, and obtaining a first recognition accuracy rate corresponding to an abnormal category in the first test result includes: Based on the preset first training set, the federated learning model is trained, and when the number of training rounds is greater than the preset number of rounds, the training of the federated learning model is stopped, and the trained federated learning model is used as the first model; Obtain the first test result of the first model on the test set, and obtain the first recognition accuracy rate corresponding to the abnormal category in the first test result.

[0038] The method of training the federated learning model based on a preset first training set, using the trained federated learning model as a first model, obtaining a first test result of the first model on a test set, and obtaining a first recognition accuracy rate corresponding to an abnormal category in the first test result includes: Based on the preset first training set, the federated learning model is trained, and when the loss function converges, the training of the federated learning model is stopped, and the trained federated learning model is used as the first model; Obtain the first test result of the first model on the test set, and obtain the first recognition accuracy rate corresponding to the abnormal category in the first test result.

[0039] S202, marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, obtaining rare samples corresponding to the rare category, and combining a plurality of the rare categories and a plurality of the rare samples into a second training set; Among them, the abnormal categories with the first recognition accuracy lower than the threshold are difficult-to-identify abnormal categories, and the difficult-to-identify abnormal categories are often closely related to the insufficient number of samples or the imbalance of data distribution. Therefore, marking the abnormal categories with the first recognition accuracy lower than the threshold as rare categories can better handle these rare categories.

[0040] The threshold is set by the user or by the system default and is not restricted here.

[0041] The step of marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, obtaining rare samples corresponding to the rare category, and forming a second training set with multiple rare categories and multiple rare samples includes: Marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, and obtaining the preset abnormal sample corresponding to the abnormal category whose recognition accuracy is lower than the threshold; The preset abnormal samples are marked as rare samples corresponding to the rare categories, and a plurality of the rare categories and a plurality of the rare samples are combined into the second training set.

[0042] Abnormal samples and their corresponding abnormal categories are widely used in many fields, especially in medical treatment, security monitoring, industrial detection and other fields.

[0043] For the sake of illustration, here are some examples of abnormal samples and their corresponding abnormal categories, detailed as follows: For example, the abnormal category is structural abnormality, and the abnormal samples are medical samples or industrial detection samples: Medical samples: In X-ray or MRI samples, the bone morphology at the fracture site is obviously broken or dislocated.

[0044] Industrial test samples: products with non-compliant structural features such as cracks, holes or deformation on the surface.

[0045] For example, the anomaly category is texture anomaly, and the anomaly sample is a monitoring sample.

[0046] Surveillance samples: In surveillance samples, unusual textures on the ground may indicate that items have been left behind or hidden.

[0047] For example, the anomaly category is color anomaly and the anomaly sample is an agricultural monitoring sample.

[0048] Agricultural monitoring samples: In agricultural monitoring samples, abnormal crop color in a certain area may indicate pests or diseases or malnutrition.

[0049] S203, inputting the rare sample of the second training set into the first model, obtaining a category probability distribution output by the first model based on the rare sample, and obtaining a predicted probability when a preset image of the rare sample belongs to the rare category in the category probability distribution; S204, obtaining the total number of rare samples, obtaining the number of rare samples corresponding to the rare category, the first proportional coefficient, and the second proportional coefficient, and obtaining a preset value when the preset image of the rare sample belongs to the rare category, wherein the total number of rare samples is the sum of the number of rare samples corresponding to each of the rare categories; S205, generating an average loss value corresponding to the second training set according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value, and a preset average loss model; Exemplarily, the average loss model is: ; Wherein, F is the average loss value; N is the total number of rare samples; i is the serial number of the rare sample; K is the total number of rare samples; c is the serial number of the rare category; is the number of rare samples of the cth abnormal category; is the first proportionality coefficient; is the second proportionality coefficient; is the predicted probability when the preset image of the i-th rare sample belongs to the c-th rare category; is the preset value when the preset image of the i-th rare sample belongs to the c-th rare category.

[0050] Each rare category is represented as a binary vector whose length is equal to the total number of categories, where one element is set to 1 to represent the current category and the rest are all set to 0.

[0051] For the sake of illustration, an example is given below: For example, there are four rare categories, named rare category 1, rare category 2, rare category 3 and rare category 4.

[0052] Each of the four rare categories is converted into a binary vector of length 4.

[0053] Rare class 1: [1, 0, 0, 0], Rare class 2: [0, 1, 0, 0], Rare class 3: [0, 0, 1, 0], rare class 4: [0, 0, 0, 1].

[0054] When c is 1, that is, the preset image of the i-th rare sample belongs to the first rare category, The default value of is [1, 0, 0, 0].

[0055] When c is 2, that is, the preset image of the i-th rare sample belongs to the second rare category, The default value of is [0, 1, 0, 0].

[0056] When c is 3, that is, the preset image of the i-th rare sample belongs to the third rare category, The default value of is [0, 0, 1, 0].

[0057] When c is 4, that is, the preset image of the i-th rare sample belongs to the fourth rare category, The default value of is [0, 0, 0, 1].

[0058] When it is less than 1, If it is greater than 1 and other values ​​remain unchanged, The weight of the c-th rare category may be increased to increase the contribution of the rare samples of the c-th rare category to the entire average loss value.

[0059] When it is less than 0, The smaller, The larger the value, the smaller the Can increase Contribution to the overall average loss value.

[0060] In the training process, a higher weight is given to rare categories, and the proportion of prediction errors of rare samples in rare categories in the total loss value will increase, which will prompt the first model to pay more attention to these rare samples during the training process. In this way, the first model can capture the characteristics of rare samples in rare categories, thereby improving the recognition accuracy of the first model on rare categories.

[0061] S206, training the first model based on the average loss value, and when the average loss value is less than a preset loss value, stopping the training of the first model, and using the trained first model as the second model; Among them, the preset loss value is set by the user or by the system default, and is not restricted here.

[0062] Based on the average loss value, the first model is trained. When the average loss value is not less than a preset loss value, the first model is continued to be trained through federated training.

[0063] S207: When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, save the second model.

[0064] Among them, when the second recognition accuracy rates corresponding to each of the rare categories are not greater than the threshold or the set is not an empty set, step 206 is re-executed until the second recognition accuracy rates corresponding to each of the rare categories are greater than the threshold and the set is an empty set, and the second model is saved.

[0065] Among them, when the second recognition accuracy corresponding to each of the rare categories is greater than the threshold and the set is an empty set, the second model is saved, which means that the second model can make accurate predictions for each rare category, which means that the second model has stronger adaptability and versatility and can be applied to more diverse scenarios and tasks.

[0066] For the sake of illustration, an example is given below: In the agricultural field, the agricultural platform can identify rare types of crops and take targeted prevention and control measures to protect the healthy growth of crops.

[0067] In the e-commerce field, e-commerce platforms can provide users with more personalized product recommendations by identifying rare categories of goods, thereby improving user experience and satisfaction.

[0068] Wherein, when the second recognition accuracy corresponding to each of the rare categories is greater than the threshold and the set is an empty set, after saving the second model, the federated learning method includes: Acquire a current image, and input the current image into the second model; Obtain a recognition result output by the second model based on the current image.

[0069] The feature vector of the current image is input into the second model, and a recognition result output by the second model based on the feature vector of the current image is obtained.

[0070] The beneficial effects of the embodiments of the present application lie in two aspects. On the one hand, the average loss value corresponding to the second training set is generated according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value and the preset average loss model. Since the first proportional coefficient and the second proportional coefficient can increase the proportion of rare categories in the average loss value, the first model can pay more attention to rare samples of these rare categories during the training process. On the other hand, the trained first model is used as the second model. When the second recognition accuracy corresponding to each of the rare categories is greater than the threshold and the set is an empty set, the second model is saved. Since the second recognition accuracy of the saved second model on each of the rare categories is greater than the threshold, the second model is used to help improve the recognition accuracy of the client device for the rare categories.

[0071] See also Figure 3 , Figure 3 The flowchart of saving the second model provided in the embodiment of the present application is described in detail as follows: S301, obtaining a second test result of the second model on the test set; S302, obtaining a second recognition accuracy rate corresponding to each of the rare categories in the second test result, and removing the rare categories whose second recognition accuracy rate is greater than the threshold in the set; S303: When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, save the second model.

[0072] In an embodiment of the present application, when the second recognition accuracy corresponding to each of the rare categories is greater than the threshold and the set is an empty set, the second model is saved. Since the threshold is based on a custom function, the second model can meet the user's personalized needs and improve the user's usage experience.

[0073] For the federated learning method described in the above embodiment, please refer to Figure 4 , Figure 4 A schematic block diagram of a federated learning device provided in an embodiment of the present application, Figure 4 The federated learning apparatus 400 shown can be applied to Figure 1 The client device in the application scenario diagram shown below takes the client device as an example. Figure 4 The federated learning device 400 shown is described in detail. The federated learning device 400 may include a first acquisition module 401, a second acquisition module 402, a third acquisition module 403, a fourth acquisition module 404, a generation module 405, a training module 406, and a storage module 407.

[0074] A first acquisition module 401 is used to train the federated learning model based on a preset first training set, use the trained federated learning model as a first model, obtain a first test result of the first model on a test set, and obtain a first recognition accuracy rate corresponding to an abnormal category in the first test result; A second acquisition module 402 is used to mark the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, add the rare category to a set, obtain rare samples corresponding to the rare category, and form a second training set with multiple rare categories and multiple rare samples; A third acquisition module 403 is used to input the rare sample of the second training set into the first model, obtain the category probability distribution output by the first model based on the rare sample, and obtain the predicted probability when the preset image of the rare sample belongs to the rare category in the category probability distribution; A fourth acquisition module 404 is used to acquire the total number of rare samples, acquire the number of rare samples corresponding to the rare category, the first proportional coefficient, and the second proportional coefficient, and acquire a preset value when the preset image of the rare sample belongs to the rare category, wherein the total number of rare samples is the sum of the number of rare samples corresponding to each of the rare categories; A generating module 405, configured to generate an average loss value corresponding to the second training set according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value, and a preset average loss model; A training module 406 is used to train the first model based on the average loss value, and when the average loss value is less than a preset loss value, stop training the first model and use the trained first model as the second model; The saving module 407 is used to save the second model when the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set.

[0075] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0076] The beneficial effects of the embodiments of the present application lie in two aspects. On the one hand, the average loss value corresponding to the second training set is generated according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value and the preset average loss model. Since the first proportional coefficient and the second proportional coefficient can increase the proportion of rare categories in the average loss value, the first model can pay more attention to rare samples of these rare categories during the training process. On the other hand, the trained first model is used as the second model. When the second recognition accuracy corresponding to each of the rare categories is greater than the threshold and the set is an empty set, the second model is saved. Since the second recognition accuracy of the saved second model on each of the rare categories is greater than the threshold, the second model is used to help improve the recognition accuracy of the client device for the rare categories.

[0077] See also Figure 5 , Figure 5 A schematic diagram of the structure of a client device provided in an embodiment of the present application.

[0078] like Figure 5 As shown, Figure 5The client device 2 includes: at least one processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the at least one processor 20, and when the processor 20 executes the computer program 22, the steps in any of the above-mentioned method embodiments are implemented.

[0079] The client device 2 may include, but is not limited to, a processor 20 and a memory 21. Those skilled in the art will appreciate that Figure 5 It is only an example of the client device 2 and does not constitute a limitation on the client device 2. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0080] The processor 20 is used to run the computer program 22 stored in the memory 21, and implement the following steps when executing the computer program 22: Based on a preset first training set, the federated learning model is trained, the trained federated learning model is used as a first model, a first test result of the first model on the test set is obtained, and a first recognition accuracy rate corresponding to the abnormal category is obtained in the first test result; Marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, obtaining rare samples corresponding to the rare category, and combining a plurality of the rare categories and a plurality of the rare samples into a second training set; Inputting the rare samples of the second training set into the first model, obtaining a category probability distribution output by the first model based on the rare samples, and obtaining a predicted probability when a preset image of the rare samples belongs to the rare category in the category probability distribution; Obtain the total number of rare samples, obtain the number of rare samples corresponding to the rare category, the first proportional coefficient, and the second proportional coefficient, and obtain a preset value when the preset image of the rare sample belongs to the rare category, wherein the total number of rare samples is the sum of the number of rare samples corresponding to each of the rare categories; According to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value and the preset average loss model, the average loss value corresponding to the second training set is generated; based on the average loss value, the first model is trained, and when the average loss value is less than the preset loss value, the training of the first model is stopped, and the trained first model is used as the second model; when the second recognition accuracy rates corresponding to each of the rare categories are greater than the threshold and the set is an empty set, the second model is saved.

[0081] In some embodiments, the processor 20 is configured to implement: Obtaining a first model parameter sent by a previous device, aggregating the first model parameter with a local model parameter to generate a second model parameter; Obtain the federated learning model using the second model parameters.

[0082] In some embodiments, the processor 20 is configured to implement: Based on the preset first training set, the federated learning model is trained, and when the number of training rounds is greater than the preset number of rounds, the training of the federated learning model is stopped, and the trained federated learning model is used as the first model; Obtain the first test result of the first model on the test set, and obtain the first recognition accuracy rate corresponding to the abnormal category in the first test result.

[0083] In some embodiments, the processor 20 is configured to implement: Based on the preset first training set, the federated learning model is trained, and when the loss function converges, the training of the federated learning model is stopped, and the trained federated learning model is used as the first model; Obtain the first test result of the first model on the test set, and obtain the first recognition accuracy rate corresponding to the abnormal category in the first test result.

[0084] In some embodiments, the processor 20 is configured to implement: Marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, and obtaining the preset abnormal sample corresponding to the abnormal category whose recognition accuracy is lower than the threshold; The preset abnormal samples are marked as rare samples corresponding to the rare categories, and a plurality of the rare categories and a plurality of the rare samples are combined into the second training set.

[0085] In some embodiments, the processor 20 is configured to implement: Obtaining a second test result of the second model on the test set; Obtaining a second recognition accuracy rate corresponding to each of the rare categories in the second test result, and excluding the rare categories whose second recognition accuracy rate is greater than the threshold in the set; When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, the second model is saved.

[0086] In some embodiments, the processor 20 is configured to implement: Acquire a current image, and input the current image into the second model; Obtain a recognition result output by the second model based on the current image.

[0087] The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0088] In some embodiments, the memory 21 may be an internal storage unit of the client device 2, such as a hard disk or memory of the client device 2. In other embodiments, the memory 21 may also be an external storage device of the client device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the client device 2. Further, the memory 21 may also include both an internal storage unit and an external storage device of the client device 2. The memory 21 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 21 may also be used to temporarily store data that has been output or is to be output.

[0089] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0090] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0091] The computer-readable storage medium stores program code, which can be called by a processor to execute the federated learning method described in the above method embodiment.

[0092] The computer-readable storage medium has a storage space for program codes.

[0093] The program code includes the code of any step in the federated learning method described in the above method embodiment.

[0094] For example, the program code is called by the processor and can execute the following steps: Based on a preset first training set, the federated learning model is trained, the trained federated learning model is used as a first model, a first test result of the first model on the test set is obtained, and a first recognition accuracy rate corresponding to the abnormal category is obtained in the first test result; Marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, obtaining rare samples corresponding to the rare category, and combining a plurality of the rare categories and a plurality of the rare samples into a second training set; Inputting the rare samples of the second training set into the first model, obtaining a category probability distribution output by the first model based on the rare samples, and obtaining a predicted probability when a preset image of the rare samples belongs to the rare category in the category probability distribution; Obtain the total number of rare samples, obtain the number of rare samples corresponding to the rare category, the first proportional coefficient, and the second proportional coefficient, and obtain a preset value when the preset image of the rare sample belongs to the rare category, wherein the total number of rare samples is the sum of the number of rare samples corresponding to each of the rare categories; Generate an average loss value corresponding to the second training set according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value, and a preset average loss model; Based on the average loss value, the first model is trained; when the average loss value is less than a preset loss value, the training of the first model is stopped, and the trained first model is used as the second model; When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, the second model is saved.

[0095] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0096] The computer-readable storage medium may also be an external storage device of the federated learning device or the client device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), a non-transitory computer-readable storage medium (non-transitory computer-readable storage medium), etc., equipped on the federated learning device or the client device.

[0097] Since the computer program stored in the computer-readable storage medium can execute any one of the federated learning methods provided in the embodiments of the present application, the computer-readable storage medium can achieve the beneficial effects that can be achieved by any one of the federated learning methods provided in the embodiments of the present application. For details, please refer to the previous embodiments and will not be repeated here.

[0098] An embodiment of the present application provides a computer program product. When the computer program product is run on a client device, the client device executes the above-mentioned federated learning method.

[0099] The computer program product is loaded by the client device and can perform the following steps: Based on a preset first training set, the federated learning model is trained, the trained federated learning model is used as a first model, a first test result of the first model on the test set is obtained, and a first recognition accuracy rate corresponding to the abnormal category is obtained in the first test result; Marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, obtaining rare samples corresponding to the rare category, and combining a plurality of the rare categories and a plurality of the rare samples into a second training set; Inputting the rare samples of the second training set into the first model, obtaining a category probability distribution output by the first model based on the rare samples, and obtaining a predicted probability when a preset image of the rare samples belongs to the rare category in the category probability distribution; Obtain the total number of rare samples, obtain the number of rare samples corresponding to the rare category, the first proportional coefficient, and the second proportional coefficient, and obtain a preset value when the preset image of the rare sample belongs to the rare category, wherein the total number of rare samples is the sum of the number of rare samples corresponding to each of the rare categories; Generate an average loss value corresponding to the second training set according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value, and a preset average loss model; Based on the average loss value, the first model is trained; when the average loss value is less than a preset loss value, the training of the first model is stopped, and the trained first model is used as the second model; When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, the second model is saved.

[0100] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0101] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0102] Based on such understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the client device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0103] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0104] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A federated learning method, characterized in that: Applied to a client device, the client device stores a federated learning model, and the federated learning method includes: Based on a preset first training set, the federated learning model is trained, the trained federated learning model is used as a first model, a first test result of the first model on the test set is obtained, and a first recognition accuracy rate corresponding to the abnormal category is obtained in the first test result; Marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, obtaining rare samples corresponding to the rare category, and combining a plurality of the rare categories and a plurality of the rare samples into a second training set; Inputting the rare samples of the second training set into the first model, obtaining a category probability distribution output by the first model based on the rare samples, and obtaining a predicted probability when a preset image of the rare samples belongs to the rare category in the category probability distribution; Obtain the total number of rare samples, obtain the number of rare samples corresponding to the rare category, the first proportional coefficient, and the second proportional coefficient, and obtain a preset value when the preset image of the rare sample belongs to the rare category, wherein the total number of rare samples is the sum of the number of rare samples corresponding to each of the rare categories; Generate an average loss value corresponding to the second training set according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value, and a preset average loss model; Based on the average loss value, the first model is trained; when the average loss value is less than a preset loss value, the training of the first model is stopped, and the trained first model is used as the second model; When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, the second model is saved.

2. The federated learning method according to claim 1, characterized in that: Before training the federated learning model based on the preset first training set, using the trained federated learning model as the first model, obtaining a first test result of the first model on the test set, and obtaining a first recognition accuracy rate corresponding to the abnormal category in the first test result, the federated learning method includes: Obtaining a first model parameter sent by a previous device, aggregating the first model parameter with a local model parameter to generate a second model parameter; Obtain the federated learning model using the second model parameters.

3. The federated learning method according to claim 1, characterized in that: The step of training the federated learning model based on a preset first training set, using the trained federated learning model as a first model, obtaining a first test result of the first model on a test set, and obtaining a first recognition accuracy rate corresponding to an abnormal category in the first test result includes: Based on the preset first training set, the federated learning model is trained, and when the number of training rounds is greater than the preset number of rounds, the training of the federated learning model is stopped, and the trained federated learning model is used as the first model; Obtain the first test result of the first model on the test set, and obtain the first recognition accuracy rate corresponding to the abnormal category in the first test result.

4. The federated learning method according to claim 1, characterized in that: The step of training the federated learning model based on a preset first training set, using the trained federated learning model as a first model, obtaining a first test result of the first model on a test set, and obtaining a first recognition accuracy rate corresponding to an abnormal category in the first test result includes: Based on the preset first training set, the federated learning model is trained, and when the loss function converges, the training of the federated learning model is stopped, and the trained federated learning model is used as the first model; Obtain the first test result of the first model on the test set, and obtain the first recognition accuracy rate corresponding to the abnormal category in the first test result.

5. The federated learning method according to claim 1, characterized in that: The step of marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, obtaining rare samples corresponding to the rare category, and forming a second training set with a plurality of the rare categories and a plurality of the rare samples includes: Marking the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, adding the rare category to a set, and obtaining the preset abnormal sample corresponding to the abnormal category whose recognition accuracy is lower than the threshold; The preset abnormal samples are marked as rare samples corresponding to the rare categories, and a plurality of the rare categories and a plurality of the rare samples are combined into the second training set.

6. The federated learning method according to claim 1, characterized in that: When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, saving the second model comprises: Obtaining a second test result of the second model on the test set; Obtaining a second recognition accuracy rate corresponding to each of the rare categories in the second test result, and excluding the rare categories whose second recognition accuracy rate is greater than the threshold in the set; When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, the second model is saved.

7. The federated learning method according to claim 1, characterized in that: When the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set, after saving the second model, the federated learning method includes: Acquire a current image, and input the current image into the second model; Obtain a recognition result output by the second model based on the current image.

8. A federated learning device, characterized in that: Applied to a client device, the client device stores a federated learning model, including: A first acquisition module is used to train the federated learning model based on a preset first training set, use the trained federated learning model as a first model, obtain a first test result of the first model on a test set, and obtain a first recognition accuracy rate corresponding to an abnormal category in the first test result; A second acquisition module is used to mark the abnormal category whose first recognition accuracy is lower than a threshold as a rare category, add the rare category to a set, obtain rare samples corresponding to the rare category, and form a second training set with multiple rare categories and multiple rare samples; A third acquisition module is used to input the rare sample of the second training set into the first model, obtain the category probability distribution output by the first model based on the rare sample, and obtain the predicted probability when the preset image of the rare sample belongs to the rare category in the category probability distribution; A fourth acquisition module is used to acquire the total number of rare samples, acquire the number of rare samples corresponding to the rare category, the first proportional coefficient, and the second proportional coefficient, and acquire a preset value when the preset image of the rare sample belongs to the rare category, wherein the total number of rare samples is the sum of the number of rare samples corresponding to each of the rare categories; A generating module, configured to generate an average loss value corresponding to the second training set according to the total number of rare samples, the number of rare samples, the first proportional coefficient, the second proportional coefficient, the predicted probability, the preset value, and a preset average loss model; A training module, used for training the first model based on the average loss value, and when the average loss value is less than a preset loss value, stopping the training of the first model and using the trained first model as the second model; A saving module is used to save the second model when the second recognition accuracy rates corresponding to the rare categories are all greater than the threshold and the set is an empty set.

9. A client device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the federated learning method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the federated learning method according to any one of claims 1 to 7 is implemented.

Images