Image data enhancement method and device based on horizontal federated learning, equipment and medium

Through the image data augmentation method based on horizontal federated learning, GAN is used to train model between each client and the central server, and the problem of large sample distribution gap after image data augmentation is solved, and high-quality data augmentation and accuracy are achieved. Suitable for scenarios with small data volume and privacy protection requirements.

CN119091251APending Publication Date: 2024-12-06JILIN UNIVERSITY
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411224959.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing image data enhancement methods lead to a large gap between the sample distribution after data enhancement and the original sample distribution, which affects the learning effect of deep neural networks. Especially when the data volume is small and privacy protection is required, the accuracy of model training is difficult to ensure.

Method used

Using a horizontal federated learning-based image data augmentation method, the model training is carried out between each client and the central server through a generation adversarial network (GAN). The generator is used to generate simulated images, and the discriminator is used to determine the authenticity and category of images. The FedAVG algorithm is used to aggregate local model parameters to reduce communication load and improve the authenticity and accuracy of data enhancement.

Benefits of technology

Effectively expand the amount of training data, improve the generator's ability to generate high-quality images, ensure the accuracy of data annotation, reduce the overall training cost and time, and is suitable for scenarios with small data volume and privacy protection, such as brain CT images of medical institutions Data enhancement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091251A_ABST
    Figure CN119091251A_ABST
Patent Text Reader

Abstract

The invention discloses an image data enhancement method and device based on horizontal federated learning, equipment and a medium, and relates to the technical field of data enhancement, and the method comprises the steps: training a local model and a global model of a central server side through employing a data set of each client side based on a horizontal federated learning framework, taking the trained global model as a data enhancement model; the local model and the global model are both generative adversarial networks, generators in the generative adversarial networks are used for generating simulation images, and discriminators in the generative adversarial networks are used for discriminating authenticity and categories of the simulation images; and carrying out image data enhancement by adopting the data enhancement model. The authenticity of image data enhancement is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data enhancement technology, and in particular to an image data enhancement method, device, equipment and medium based on horizontal federated learning. Background Art

[0002] With the widespread application of machine learning, especially deep learning, in image detection, the requirements for model detection accuracy are getting higher and higher. During model training, the amount of data in the training set can affect the accuracy of model training. However, in actual applications, some institutions have small data volumes and data that requires privacy protection, such as medical institutions. In order to avoid these situations, data augmentation methods can be used to expand the data scale.

[0003] At present, the common methods of data enhancement for image data, such as image superposition, rotation, scaling, etc., have a problem that the sample distribution after data enhancement is too different from the original sample distribution, which affects the detection accuracy of the model obtained after learning the deep neural network. Summary of the invention

[0004] The purpose of this application is to provide an image data enhancement method, device, equipment and medium based on horizontal federated learning, which improves the authenticity of image data enhancement.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides an image data enhancement method based on horizontal federated learning, comprising:

[0007] Based on the horizontal federated learning framework, the data sets of each client are used to train the local model and the global model of the central server, and the trained global model is used as the data enhancement model; the local model and the global model are both generative adversarial networks, the generator in the generative adversarial network is used to generate simulated images, and the discriminator in the generative adversarial network is used to discriminate the authenticity and category of the simulated images;

[0008] The data enhancement model is used to perform image data enhancement.

[0009] Optionally, based on the horizontal federated learning framework, the data sets of each client are used to train the local model and the global model of the central server, and the trained global model is used as the data enhancement model, specifically including:

[0010] Perform K rounds of global training. During each round of global training:

[0011] Randomly select N clients from a set number of clients, perform L iterative training of the local model in each client, and after the local model of each client is trained L times, upload the currently trained local model parameters to the central server; wherein K, N and L are all integers greater than 0;

[0012] At the central server, the local model parameters uploaded by each client are aggregated, the model parameters of the global model are updated using the aggregated model parameters, and the updated model parameters of the global model are sent to each client;

[0013] After receiving the model parameters of the global model, each client updates the model parameters of the local model.

[0014] Optionally, the local model parameters uploaded by each client are aggregated, specifically including: adopting a FedAVG algorithm to aggregate the local model parameters uploaded by each client.

[0015] Optionally, the objective function of the generative adversarial network is expressed as:

[0016]

[0017] Among them, V fed (D,G) is the objective function value, D is the discriminator, G is the generator, N is the number of clients, D n () represents the nth discriminator, G n () represents the nth generator, x is the sample image in the data set, z is the noise, It represents the expected value of the logarithm of the probability that the nth discriminator correctly judges a sample image as true in the real data distribution. It represents the expected value of the logarithm of the probability that the n-th discriminator judges the simulated image generated by the n-th generator as fake on the noise distribution.

[0018] Optionally, each client performs L iterative training of the local model, specifically including:

[0019] The formula is used in each iteration training Update parameters;

[0020] Among them, θ d,g represents the parameters of the local model before updating, θ d',g' represents the updated parameters of the local model, α represents the learning rate, β1 and β2 both represent hyperparameters, Represents the gradient of the local model, ε sets the value, v t is the first-order momentum at time t, s t is the second-order momentum at time t, v t-1is the first-order momentum at time t-1, s t-1 is the second-order momentum at time t-1.

[0021] Optionally, the discriminator includes five convolutional layers connected in sequence, and the discriminator also includes a Softmax layer and a Sigmoid function. The inputs of the Softmax layer and the Sigmoid function are both connected to the output of the last convolutional layer in the five convolutional layers connected in sequence. The Softmax layer is used to output the category of the simulated image, and the Sigmoid function is used to output the authenticity of the simulated image.

[0022] Optionally, the client is a medical institution, and the sample images in the data set are brain CT images; the data enhancement model is used to perform data enhancement on the brain CT images.

[0023] In a second aspect, the present application provides an image data enhancement device based on horizontal federated learning, comprising:

[0024] A data enhancement model generation module is used to train a local model and a global model of a central server using data sets of each client based on a horizontal federated learning framework, and use the trained global model as a data enhancement model; both the local model and the global model are generative adversarial networks, the generator in the generative adversarial network is used to generate simulated images, and the discriminator in the generative adversarial network is used to discriminate the authenticity and category of the simulated images;

[0025] The data enhancement model application module is used to perform image data enhancement using the data enhancement model.

[0026] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-mentioned image data enhancement methods based on horizontal federated learning.

[0027] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned methods for image data enhancement based on horizontal federated learning.

[0028] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0029] The present application provides an image data enhancement method, apparatus, device and medium based on horizontal federated learning, which uses horizontal federated learning and generative adversarial networks (GAN) training to obtain a data enhancement model, and uses sample data in the data set of each client for model training, thereby expanding the amount of training data, thereby improving the generator's ability to generate high-quality image data, thereby improving the authenticity of image data enhancement. In addition, the discriminator also has the function of generating categories, thereby ensuring the accuracy of the generated data annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 A flowchart of an image data enhancement method based on horizontal federated learning provided in one embodiment of the present application;

[0032] Figure 2 A schematic diagram of the horizontal federated learning principle provided for an embodiment of the present application;

[0033] Figure 3 A schematic diagram of a generative adversarial network structure provided in one embodiment of the present application;

[0034] Figure 4 A schematic diagram of functional modules of an image data enhancement device based on horizontal federated learning provided in another embodiment of the present application;

[0035] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0038] This application provides an image data enhancement method based on horizontal federated learning, such as Figure 1 As shown, the image data enhancement based on horizontal federated learning includes the following steps 101 and 102.

[0039] Step 101: Based on the horizontal federated learning framework, the data sets of each client are used to train the local model and the global model of the central server, and the trained global model is used as the data enhancement model; the local model and the global model are both generative adversarial networks, the generator in the generative adversarial network is used to generate simulated images, and the discriminator in the generative adversarial network is used to discriminate the authenticity and category of the simulated images.

[0040] Step 102: Use the data enhancement model to perform image data enhancement.

[0041] This application uses horizontal federated learning and generative adversarial network training to obtain a data enhancement model, and uses sample data from the dataset of each client to train the model, thereby expanding the amount of training data, thereby improving the generator's ability to generate high-quality image data, thereby improving the authenticity of image data enhancement. In addition, the discriminator also has the function of generating categories, thereby ensuring the accuracy of the generated data annotation.

[0042] The process of implementing federated learning in the global model of this application is as follows Figure 2 As shown in the figure, it includes N clients and 1 central server. The client can be a medical institution. Each client uses its own image dataset to participate in the horizontal federated learning process. With the help of the central server, a global data enhancement model is built to generate high-quality usable synthetic images to reduce local data pressure. By expanding the dataset, the training accuracy of the local diagnostic model can be improved.

[0043] The present application discloses an image data enhancement method based on horizontal federated learning, which uses image data enhancement in multiple fields. For example, the client is a medical institution, and the sample images in the data set are brain computed tomography (CT) images. The data enhancement model is used to perform data enhancement on brain CT images, and the categories include the presence of cerebral hemorrhage and the absence of cerebral hemorrhage, thereby providing high-quality image sample data for cerebral hemorrhage detection through brain CT images.

[0044] The problem of communication load is common in federated learning. Without considering hardware facilities and network protocols, the problem is solved by increasing the number of local iterations and reducing the total number of communication rounds. In the experiment, K represents the number of global communication rounds (global communication is the process of transmitting information back and forth on the central server), which includes two communications, one is the downlink (sending the global model parameters to the local training node, that is, the client), and the other is the aggregation (uploading the updates of the local training node back to the central server), and uploading and aggregation are performed after every L local training iterations. Therefore, if a global round K includes the two steps of downlink and aggregation, then in K global rounds, each local training node will receive K times of model parameters sent down, and upload K times of local training updates. Therefore, for K global rounds, the total number of communications for each local training node will be 2K times (each round includes one downlink and one upload), which is the average number of communication rounds per L iterations, so the communication consumption per round is On the contrary, if each round of local training is uploaded, there will be a lot of communication consumption, and one local update and one upload will consume 2L communications. Therefore, the method of this application reduces the amount of data transmitted over the network, can effectively improve communication efficiency, and thus reduce the overall training cost and time.

[0045] In order to meet the actual application scenarios, this application uses an active sampling method to select fixed clients that join the horizontal federated data augmentation model for parallel training instead of randomly selecting multiple clients each time. That is, a set number of fixed clients is first determined, and then N clients are randomly selected within the fixed client range.

[0046] Among them, step 101 specifically includes: performing K rounds of global training, in each round of global training: randomly selecting N clients from a set number of clients, performing L iterative training of the local model in each client, after the local model of each client is trained L times, uploading the currently trained local model parameters to the central server; wherein K, N and L are all integers greater than 0; at the central server, aggregating the local model parameters uploaded by each client, using the aggregated model parameters to update the model parameters of the global model, and sending the updated model parameters of the global model to each client; after each client receives the sent model parameters of the global model, it updates the model parameters of the local model.

[0047] In an exemplary embodiment, it is assumed that each client has its own data set, denoted as D n , which follows a distribution P n(d), where d is a real data sample. In order to improve the accuracy of the global model, the local model of the client is designed to use the same GAN model as the global model, including a generator and a discriminator. Each client trains a generator based on its dataset D n Generate data distribution P using latent space z n (g(z)), aims to simulate the real data distribution P n (d), we can finally get P n (g(z))≈P n (d) In a federated GAN, federated entities collaborate to learn to generate “real” fake images (simulated images) to fool all discriminators, which attempt to distinguish real data generated by the generator from fake images.

[0048] The objective function of the generative adversarial network is expressed as:

[0049]

[0050] Among them, V fed (D,G) is the objective function value, D is the discriminator, G is the generator, N is the number of clients, D n () represents the nth discriminator, G n () represents the nth generator, x is the sample image in the data set, z is the noise, It represents the expected value of the logarithm of the probability that the nth discriminator correctly judges a sample image as true in the real data distribution. It represents the expected value of the logarithm of the probability that the n-th discriminator judges the simulated image generated by the n-th generator as fake on the noise distribution.

[0051] In an exemplary embodiment, the process of training the local model of the client and the global model of the central server includes the central server training part and the client training part respectively.

[0052] 1) The central server training part includes: randomly selecting N clients and performing K rounds of global training. In each round of global training, after the selected clients perform local training, the local model parameter update is expressed as:

[0053] Among them, θ d' is the model parameter of the nth local model after the kth global training round update, θ g' is the model parameter of the global model after the kth global training round update, θ d Update the model parameters of the nth local model before the kth global training round, θ g Update the model parameters of the global model before the kth global training round.

[0054] After the current round of local model training is completed, the parameters of the local model are uploaded, and the central server aggregates and updates the model parameters of the global model, which is expressed as:

[0055] After the central server updates the model parameters of the global model, the model parameters of the global model (θ d' ,θ g' ) is distributed to each client.

[0056] 2) The client training part includes: the local model performs L local trainings, and in each local training, the noise P n (g(z)) has a sample size of m and real data P n (x) is the number of samples to be sampled, m.

[0057] The discriminator parameter update in the local model is expressed as:

[0058]

[0059] Among them, α represents the learning rate, D() represents the discriminator output of real data, G() represents the output of the generator, b represents the training batch size, and z j represents the latent space of the jth sample in the training batch, x represents the input data, y represents the label of the real data, The data output by the generator is input into the discriminator, and the discriminator calculates the gradient of it. It means that the real data set is input into the discriminator, and the discriminator calculates the gradient of the real data set.

[0060] The generator parameter update in the local model is expressed as: represents the gradient with respect to the generator parameters.

[0061] After each round of local training, the client model parameters (θ d ,θ g ) n Send server side.

[0062] This application aggregates the local model parameters uploaded by each client, specifically including: using the FedAVG algorithm to aggregate the local model parameters uploaded by each client. Using the FedAVG algorithm can reduce the error, so the weight update of the central server can be expressed as:

[0063] Among them, w n is the weight of the central server in the nth round of global training, m is the number of batches, is the central server weight for the jth batch during the nth round of global training.

[0064] After the local model training of each client in the current round is completed, the central server will update the aggregated model parameters on the central server, and the updated parameters will be sent to each client for a new round of training.

[0065] During the local execution of the client, each client performs L local iterations, and each local iteration performs parameter updates of the discriminator D and the generator G respectively, where y represents the label of the real data, ▽ D(G(zj)) and D(x) Represents the gradient of the discriminator parameters. ▽ G(D(G(z))) Represents the gradient of the generator parameters. After local training is completed, the client sends the updated parameters to the server. During the client network model training process, the client uses the client optimizer to perform multiple training stages, and the loss function is set to: Loss = -(y×log(P(x))+(1-y)×log(1-P(x))).

[0066] Where y is the true label and P(x) is the predicted probability of the local model. By measuring the difference between the local model output and the actual target, the local model's prediction is made closer to the true value, thereby minimizing the loss. Minimizing the loss function reflects the process of adjusting the parameters of the local model during the learning process to adapt to the characteristics of the training data, thereby improving the generalization ability to future data. When the local model performs well on the training data and the loss function tends to the minimum value, it is expected to make accurate predictions on new, unseen data.

[0067] In order to make the model converge better, this application introduces momentum in the parameter update of the local generative adversarial network model. The momentum optimizer can help the model converge to the minimum value of the loss function faster, especially in the presence of noise or complex data structures. By introducing momentum, the previous gradient information can be considered when updating the parameters, so that the model parameters can be adjusted more smoothly during training, and the efficiency of the optimization process of model update in client model training is improved. Such optimization methods help ensure that the model can achieve better performance within a limited training time. Therefore, L iterative training of the local model is performed in each client, specifically including:

[0068] The formula is used in each iteration training Perform parameter updates to dynamically adjust the parameters of each local client.

[0069] Among them, θ d,g represents the parameters of the local model before updating, θ d',g'represents the updated parameters of the local model, α represents the learning rate, β1 and β2 both represent hyperparameters, Represents the gradient of the local model, ε sets the value, and ε is a very small value to maintain numerical stability. v t is the first-order momentum at time t, s t is the second-order momentum at time t, v t-1 is the first-order momentum at time t-1, s t-1 is the second-order momentum at time t-1.

[0070] This application introduces momentum in the parameter update of the local model to achieve dynamic model adjustment, so that the model has the ability of self-regulation and can dynamically adjust the model parameters according to the data characteristics of each client, effectively solving the problem that the global model is difficult to converge when the data is not independent and identically distributed (non-IID), thereby significantly improving the global model performance and enhancing the robustness of the system.

[0071] like Figure 3 As shown, the generative adversarial network consists of two convolutional neural networks (CNNs), in which the generator is configured with 10 convolutional layers and the discriminator is configured with 5 convolutional layers. The parameter selection of these layers needs to be cleverly adjusted according to different data sets. This application sets the global training to 500 rounds, that is, K = 500, and each local model receives 20 rounds of training, that is, L = 20. In the optimization process, the optimization algorithm of the adaptive learning rate is unique in that it can dynamically adjust the learning rate according to the model training state. This flexible learning rate adjustment method can usually better adapt to different data sets and model structures, thereby improving the training effect and performance of the overall model. The 10 convolutional layers of the generator are respectively connected in sequence, namely, convolutional layer 1, convolutional layer 2, convolutional layer 3, ..., convolutional layer 10. The features output by convolutional layer 1 to convolutional layer 9 are successively subjected to batch normalization operations (BatchNorm) and ReLU functions, and the features output by convolutional layer 9 are used as the output of the generator after passing through the Tanh function.

[0072] The discriminator includes five convolutional layers connected in sequence. The five convolutional layers in the discriminator are convolutional layer 11, convolutional layer 12, convolutional layer 13, convolutional layer 14 and convolutional layer 15. The output of convolutional layer 11 is sequentially subjected to the LeakyReLU function and random dropout (DropOut) and then input into convolutional layer 12, the output of convolutional layer 12 is sequentially subjected to batch normalization operation and LeakyReLU function and then input into convolutional layer 13, the output of convolutional layer 13 is sequentially subjected to batch normalization operation, LeakyReLU function and random dropout and then input into convolutional layer 14, and the output of convolutional layer 14 is sequentially subjected to batch normalization operation, LeakyReLU function and random dropout and then input into convolutional layer 15.

[0073] The discriminator also includes a Softmax layer and a Sigmoid function, the inputs of the Softmax layer and the Sigmoid function are connected to the output of the last convolutional layer (convolutional layer 15) in the five sequentially connected convolutional layers, and the Softmax layer is used to output the category of the simulated image, that is, the classification prediction result. The Softmax layer outputs all categories of different data sets. The Sigmoid function is used to output the authenticity of the simulated image to further optimize the performance of the discriminator.

[0074] This application generates adversarial network (GAN) structures, which maximize the generator's ability to produce high-quality data while ensuring efficiency. This structure not only improves the authenticity of the generated data, but also ensures the accuracy of the generated data annotation, which is crucial to improving the diversity and richness of the training data set. Synthetic data generation is matched.

[0075] The data enhancement model obtained in this application can also provide corresponding accurate data annotations. This ability is crucial for training high-precision machine learning models, especially in scenarios that require a large amount of accurately labeled data.

[0076] In order to improve the image generation performance of the overall model, this application uses group normalization technology to ensure that the network can still achieve the best results under different batch sizes. By maximizing the quality of the generator, the performance of the model is further enhanced, making it more adaptable and robust. This series of optimization measures helps to obtain better image generation results in various scenarios.

[0077] This application can generate high-quality synthetic data, that is, it can generate synthetic data that is close to reality. These "fake" data are no different from real data in quality, but the source is more flexible and can be customized for specific training needs, greatly expanding the availability and applicability of the data. Based on the above advantages, this application not only improves the model convergence speed and training efficiency, but also shows all improvements in the generalization ability of the model and adaptability to different data distributions. In addition, the model design of this application focuses on flexibility and scalability, making it not only suitable for current research problems, but also can be easily extended to other fields and application scenarios, and has broad practical application potential.

[0078] Based on the same inventive concept, the embodiment of the present application also provides an image data enhancement device based on horizontal federated learning for implementing the above-mentioned image data enhancement method based on horizontal federated learning. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the image data enhancement device based on horizontal federated learning provided below can refer to the limitations of the image data enhancement method based on horizontal federated learning above, and will not be repeated here.

[0079] In an exemplary embodiment, Figure 4 As shown, an image data enhancement device based on horizontal federated learning is provided, comprising:

[0080] A data enhancement model generation module is used to train a local model and a global model of a central server using data sets of each client based on a horizontal federated learning framework, and use the trained global model as a data enhancement model; both the local model and the global model are generative adversarial networks, the generator in the generative adversarial network is used to generate simulated images, and the discriminator in the generative adversarial network is used to discriminate the authenticity and category of the simulated images;

[0081] The data enhancement model application module is used to perform image data enhancement using the data enhancement model.

[0082] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image data enhancement data based on horizontal federated learning. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for image data enhancement based on horizontal federated learning is implemented.

[0083] Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0084] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0085] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0087] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0088] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a data processing logic of a programmable logic device, etc., but is not limited thereto.

[0089] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An image data enhancement method based on horizontal federated learning, characterized in that: The image data enhancement based on horizontal federated learning includes: Based on the horizontal federated learning framework, the data sets of each client are used to train the local model and the global model of the central server, and the trained global model is used as the data enhancement model; the local model and the global model are both generative adversarial networks, the generator in the generative adversarial network is used to generate simulated images, and the discriminator in the generative adversarial network is used to discriminate the authenticity and category of the simulated images; The data enhancement model is used to perform image data enhancement.

2. The image data enhancement method based on horizontal federated learning according to claim 1, characterized in that: Based on the horizontal federated learning framework, the local model and the global model of the central server are trained using the data sets of each client, and the trained global model is used as the data enhancement model. Specifically, it includes: Perform K rounds of global training. During each round of global training: Randomly select N clients from a set number of clients, perform L iterative training of the local model in each client, and after the local model of each client is trained L times, upload the currently trained local model parameters to the central server; wherein K, N and L are all integers greater than 0; At the central server, the local model parameters uploaded by each client are aggregated, the model parameters of the global model are updated using the aggregated model parameters, and the updated model parameters of the global model are sent to each client; After receiving the model parameters of the global model, each client updates the model parameters of the local model.

3. The image data enhancement method based on horizontal federated learning according to claim 2, characterized in that: Aggregating the local model parameters uploaded by each client, specifically including: adopting the FedAVG algorithm to aggregate the local model parameters uploaded by each client.

4. The image data enhancement method based on horizontal federated learning according to claim 1, characterized in that: The objective function of the generative adversarial network is expressed as: Among them, V fed (D,G) is the objective function value, D is the discriminator, G is the generator, N is the number of clients, D n () represents the nth discriminator, G n () represents the nth generator, x is the sample image in the data set, z is the noise, It represents the expected value of the logarithm of the probability that the nth discriminator correctly judges a sample image as true in the real data distribution. It represents the expected value of the logarithm of the probability that the n-th discriminator judges the simulated image generated by the n-th generator as fake on the noise distribution.

5. The image data enhancement method based on horizontal federated learning according to claim 2, characterized in that: Each client performs L iterative training of the local model, including: The formula is used in each iteration training Update parameters; Among them, θ d,g represents the parameters of the local model before updating, θ d',g' represents the updated parameters of the local model, α represents the learning rate, β1 and β2 both represent hyperparameters, Represents the gradient of the local model, ε sets the value, v t is the first-order momentum at time t, s t is the second-order momentum at time t, v t-1 is the first-order momentum at time t-1, s t-1 is the second-order momentum at time t-1.

6. The image data enhancement method based on horizontal federated learning according to claim 1, characterized in that: The discriminator includes five convolutional layers connected in sequence, and the discriminator also includes a Softmax layer and a Sigmoid function. The inputs of the Softmax layer and the Sigmoid function are both connected to the output of the last convolutional layer in the five convolutional layers connected in sequence. The Softmax layer is used to output the category of the simulated image, and the Sigmoid function is used to output the authenticity of the simulated image.

7. The image data enhancement method based on horizontal federated learning according to claim 1, characterized in that: The client is a medical institution, the sample images in the data set are brain CT images, and the data enhancement model is used to perform data enhancement on the brain CT images.

8. An image data enhancement device based on horizontal federated learning, characterized in that: The image data enhancement device based on horizontal federated learning includes: A data enhancement model generation module is used to train a local model and a global model of a central server using data sets of each client based on a horizontal federated learning framework, and use the trained global model as a data enhancement model; both the local model and the global model are generative adversarial networks, the generator in the generative adversarial network is used to generate simulated images, and the discriminator in the generative adversarial network is used to discriminate the authenticity and category of the simulated images; The data enhancement model application module is used to perform image data enhancement using the data enhancement model.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image data enhancement method based on horizontal federated learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image data enhancement method based on horizontal federated learning described in any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Multi-layer vertical and horizontal federated generative adversarial network data enhancement method

    CN121562739A

  • A Data Augmentation Method for Multi-Layer Federation Generative Adversarial Networks

    CN121562739B