System and method for improving image recognition accuracy under the premise of privacy protection
Through the round training and parameter update mechanism between user nodes and central nodes, the twin noise image library and KL divergence entropy function are used to solve the privacy protection problem in the training of medical image data center, and the medical image recognition model is improved Identify accuracy and training level.
Patent Information
- Application Number
- CN202210472140.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-29
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Under the premise of protecting patient privacy, it is difficult to concentrate medical image training data from various hospitals, resulting in poor recognition effect of medical image recognition models in different data scenarios, and it is difficult to achieve high training level and high recognition accuracy.
By establishing a round training and parameter update mechanism between the user node module and the central node module, using the twin noise image library construction and KL divergence entropy function, centralized training and model update of medical image data in each hospital are realized, privacy is protected and improved Recognition accuracy.
It has achieved the ability to concentrate the advantageous data of each hospital under the premise of protecting privacy, improve the training level and recognition accuracy of medical image recognition models, and ensure that patient privacy data is not leaked.
Smart Images

Figure CN114864054B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a system and method for improving image recognition accuracy under the premise of privacy protection, and belongs to the field of information technology, in particular to the field of medical image recognition technology. Background Art
[0002] With the extensive and long-term development of deep learning technology, more and more computer tasks are being solved or partially solved by deep learning, including tasks related to the segmentation and recognition of large-scale medical images. People hope to use the current deep learning neural network technology to help hospitals achieve automatic segmentation and automatic recognition of medical images, thereby improving the processing speed of medical images and allowing patients to receive faster and more accurate diagnoses.
[0003] At present, the training of models based on deep learning requires a large amount of high-quality data to support its performance. Specifically, a large amount of data is needed to train the model. In order to realize the automatic recognition of medical images, a large amount of medical image training data is needed. At present, the image data of CT and MRI imaging in major hospitals require authoritative expert doctors to manually segment the regional boundaries between normal tissues and lesions according to the characteristics of the disease and imaging characteristics, so as to create high-quality training data. Moreover, we need to collect as much of this data as possible, and then input this data into the medical image recognition model for unified training, so that the final medical image recognition model can have a good image recognition accuracy rate.
[0004] Usually, different hospitals have different specialized departments. The best departments are staffed by the most experienced expert physicians, and the medical image training data they provide is of higher quality. Therefore, if we can centralize the medical image training data of the specialized departments of different hospitals for unified training, the resulting medical image recognition model can have a high recognition accuracy and good image segmentation results for a wide range of medical images.
[0005] However, due to the need to protect patient privacy, it is currently difficult to centralize the medical image training data of various hospitals and conduct unified training for medical image recognition models. As a result, if only the medical image training data of one hospital is used to train the medical image recognition model, then the resulting medical image recognition model will only perform well on data similar to that of the hospital's data set, and may perform poorly in other data scenarios (such as CT and MRI images of different parts of the body). Even when the hospital's data is insufficient, it is difficult for the medical image recognition model to achieve good recognition results for CT and MRI images of any part of the body.
[0006] To sum up, how to concentrate the advantageous medical image training data of various hospitals under the premise of protecting patient privacy, improve the training level of medical image recognition models, and improve the recognition accuracy of medical images has become a technical problem that urgently needs to be solved in the field of medical image recognition technology. Summary of the invention
[0007] In view of this, the purpose of the present invention is to invent a system and method that can not only concentrate the superior medical image training data of various hospitals, but also protect the privacy of patients and improve the training level of medical image recognition models.
[0008] In order to achieve the above object, the present invention proposes a system for improving image recognition accuracy under the premise of privacy protection, the system includes n user node modules and a central node module, the user node module and the central node module are interconnected; n is a natural number greater than 1; the modules are as follows:
[0009] User node module: The functions of this module are: (1) to use its own local medical image library to perform round-by-round training on the medical image recognition model; (2) to send the parameters of the medical image recognition model trained in each round to the central node module; (3) to use the medical image recognition model trained in this round to generate a twin noise image library of the local medical image library, and to send the twin noise image library to the central module; (4) to use the medical image recognition model trained in this round to calculate the output noiseLogits of each twin noise image in the twin noise image library, and to send the output noiseLogits to the central node module; (4) to receive the latest parameters of the medical image recognition model sent by the central node module and to update the medical image recognition model;
[0010] Central node module: The functions of this module are: (1) sending the medical image recognition model to all user node modules; (2) updating the parameters of the medical image recognition model based on the parameters of the medical image recognition model received from all user node modules in this round; (3) retraining the updated medical image recognition model based on the twin noise image library received from all user node modules in this round and the output noiseLogits of each twin noise image in the twin noise image library; (4) sending the parameters of the medical image recognition model trained in this round to the user node modules.
[0011] The user node module uses the medical image recognition model trained in this round to generate the twin noise image library of the local medical image library. The specific content is:
[0012] (1) For each class image in the local medical image library, the average output of the medical image recognition model is calculated according to the following formula:
[0013]
[0014] In the above formula, Represents the local medical image library of the i-th user node module, which has a total of images from c0 to c k-1 k categories; avgLogits c Represents the local medical image library for the i-th user node module The average output calculated by the medical image recognition model for the c-th image in ; represents the output obtained by inputting sample x into the medical image recognition model that has completed the tth round of training; y represents the category of sample x;
[0015] (2) A random noise image is constructed. The medical image recognition model that has completed the tth round of training is used to update the constructed noise image using the following loss function according to the gradient descent back propagation method:
[0016]
[0017] In the above formula, represents the KL divergence entropy function; Represents the softmax function; noiseLogits c Indicates the output obtained by inputting the noise image into the medical image recognition model that has completed the t-th round of training; after the noise image is updated a set number of times or the loss function value is less than the set threshold, the noise image obtained by the above iterative update is the local medical image library of the i-th user node module. A twin noise image corresponding to the c-th image in .
[0018] The central node module updates the parameters of the medical image recognition model based on the parameters of the medical image recognition model received from all user node modules in this round. The specific content is:
[0019] The parameters of the medical image recognition model are updated according to the following formula:
[0020]
[0021] In the above formula, SetC t Represents a collection of user nodes; is the parameter of the medical image recognition model sent by the i-th user node module at round t The calculated parameter change is in is the parameter of the medical image recognition model sent by the i-th user node module at round t-1; w t is the parameter of the medical image recognition model of the central node module at round t; γ is a weight parameter.
[0022] The central node module retrains the updated medical image recognition model based on the twin noise image library received from all user node modules in this round and the output noiseLogits of each twin noise image in the twin noise image library. The specific content is:
[0023] The data in the twin noise image library and the output noiseLogits of each twin noise image in the twin noise image library are divided into batches. For each batch of data b, the medical image recognition model is retrained according to the following loss function:
[0024]
[0025] In the above formula, represents the KL divergence entropy function; represents the softmax function; nd represents a received twin noise image, and nt represents the output noiseLogits of the medical image recognition model of the user node module that sends nd for nd; It indicates that the central node module inputs nd into the output obtained by the medical image recognition model that has completed the t+1th round of training; after a set number of rounds of training, the latest model parameters of the medical image recognition model are obtained.
[0026] The present invention also proposes a method for improving image recognition accuracy under the premise of privacy protection, and the method includes the following steps:
[0027] (1) System initialization: The central node module sends the medical image recognition model to all user node modules;
[0028] (2) The user node module receives the latest parameters of the medical image recognition model sent by the central node module and updates its local medical image recognition model;
[0029] (3) The user node module uses its own local medical image library to perform rounds of training on its local medical image recognition model;
[0030] (4) The user node module uses the medical image recognition model trained in this round to generate a twin noise image library of the local medical image library;
[0031] (5) The user node module uses the medical image recognition model trained in this round to calculate the output noiseLogits of each twin noise image in the twin noise image library;
[0032] (6) The user node module sends the parameters of the medical image recognition model trained in this round, the twin noise image library, and the output noiseLogits of each twin noise image in the twin noise image library to the central node module;
[0033] (7) The central node module updates the parameters of the medical image recognition model based on the parameters of the medical image recognition model received from all user node modules in this round;
[0034] (8) The central node module retrains the updated medical image recognition model based on the twin noise image library received from all user node modules in this round and the output noiseLogits of each twin noise image in the twin noise image library;
[0035] (9) The central node module sends the parameters of the medical image recognition model retrained in this round to all the user node modules;
[0036] (10) Repeat steps (2) to (9) until the medical image recognition model is fully trained.
[0037] The beneficial effects of the present invention are as follows: the system and method of the present invention, by constructing a twin noise image library of the local medical image library of each user node module, achieves the goal of protecting patient privacy and concentrating the superior medical image training data of each hospital, thereby obtaining a medical image recognition model with a high training level and improving the accuracy of automatic recognition of medical images. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 It is a module schematic diagram of a system for improving image recognition accuracy under the premise of privacy protection proposed by the present invention.
[0039] Figure 2 It is a flow chart of the method for improving image recognition accuracy under the premise of privacy protection proposed by the present invention. DETAILED DESCRIPTION
[0040] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings.
[0041] See also Figure 1, introduces a system for improving image recognition accuracy under the premise of privacy protection proposed by the present invention, the system includes n user node modules and one central node module, the user node modules and the central node module are interconnected; n is a natural number greater than 1; in the embodiment, n=5.
[0042] The modules are as follows:
[0043] User node module: The functions of this module are: (1) to use its own local medical image library to perform round-by-round training on the medical image recognition model; (2) to send the parameters of the medical image recognition model trained in each round to the central node module; (3) to use the medical image recognition model trained in this round to generate a twin noise image library of the local medical image library, and to send the twin noise image library to the central module; (4) to use the medical image recognition model trained in this round to calculate the output noiseLogits of each twin noise image in the twin noise image library, and to send the output noiseLogits to the central node module; (4) to receive the latest parameters of the medical image recognition model sent by the central node module and to update the medical image recognition model;
[0044] The user mentioned in the present invention refers to a hospital or a medical unit; the user node module is deployed in the hospital or the medical unit; each user has one or more medical fields in which he / she is specialized, and in the medical field, each user has established his / her own local medical image library;
[0045] In an embodiment, the medical image recognition model adopts the deep convolutional neural network model ResNet50.
[0046] Central node module: The functions of this module are: (1) sending the medical image recognition model to all user node modules; (2) updating the parameters of the medical image recognition model based on the parameters of the medical image recognition model received from all user node modules in this round; (3) retraining the updated medical image recognition model based on the twin noise image library received from all user node modules in this round and the output noiseLogits of each twin noise image in the twin noise image library; (4) sending the parameters of the medical image recognition model trained in this round to the user node modules.
[0047] The user node module uses the medical image recognition model trained in this round to generate the specific content of the twin noise image library of the local medical image library:
[0048] (1) For each class image in the local medical image library, the average output of the medical image recognition model is calculated according to the following formula:
[0049]
[0050] In the above formula, Represents the local medical image library of the i-th user node module, which has a total of images from c0 to c k-1 k categories; avgLogits c Represents the local medical image library for the i-th user node module The average output calculated by the medical image recognition model for the c-th image in ; represents the output obtained by the i-th user node module inputting sample x into the medical image recognition model that has completed the t-th round of training; y represents the category of sample x;
[0051] In the embodiment, k=3, that is, each user's local medical image library has 3 categories;
[0052] (3) A noise image is randomly constructed. The medical image recognition model that has completed the t-th round of training is used to update the constructed noise image using the following loss function according to the gradient descent back propagation method:
[0053]
[0054] In the above formula, represents the KL divergence entropy function; Represents the softmax function; noiseLogits c Indicates the output obtained by inputting the noise image into the medical image recognition model that has completed the t-th round of training; after the noise image is updated a set number of times or the loss function value is less than the set threshold, the noise image obtained by the above iterative update is the local medical image library of the i-th user node module. A twin noise image corresponding to the c-th image in .
[0055] The twin noise image generated by the above process is embedded in the local medical image library of the i-th user node module. The complete feature information of the cth type image in the can be used to replace the medical images of the patient in the local medical image library.
[0056] The inventors found through experiments that the twin noise image remains a noise image after visualization, without any image information recognizable by the human eye. The inventors counted the pixel values of the twin noise image pixels and checked its probability distribution, and found that its probability distribution is almost the same as the distribution of the randomly constructed noise image at the beginning, which proves that from the perspective of statistical distribution, the twin noise image does not leak the patient's privacy data.
[0057] In an embodiment, the corresponding twin noise images are generated according to the number of image categories and the number of images in each category of the local medical image library of the user node module. In order to obtain better experimental results, in an embodiment, more twin noise images can also be generated for each category.
[0058] The central node module updates the parameters of the medical image recognition model based on the parameters of the medical image recognition model received from all user node modules in this round. The specific content is:
[0059] The parameters of the medical image recognition model are updated according to the following formula:
[0060]
[0061] In the above formula, SetC t Represents a collection of user nodes; is the parameter of the medical image recognition model sent by the i-th user node module at round t The calculated parameter change is in is the parameter of the medical image recognition model sent by the i-th user node module at round t-1; w t is a parameter of the medical image recognition model of the central node module at round t; γ is a weight parameter, and in the embodiment, γ=0.1.
[0062] The central node module retrains the updated medical image recognition model based on the twin noise image library received from all user node modules in this round and the output noiseLogits of each twin noise image in the twin noise image library. The specific content is:
[0063] The data in the twin noise image library and the output noiseLogits of each twin noise image in the twin noise image library are divided into batches. For each batch of data b, the medical image recognition model is retrained according to the following loss function:
[0064]
[0065] In the above formula, represents the KL divergence entropy function; represents the softmax function; nd represents a received twin noise image, and nt represents the output noiseLogits of the medical image recognition model of the user node module that sends nd for nd; It indicates that the central node module inputs nd into the output obtained by the medical image recognition model that has completed the t+1th round of training; after a set number of rounds of training (the number is 100 in the embodiment), the latest model parameters of the medical image recognition model are obtained.
[0066] See also Figure 2 , introduces a method for improving image recognition accuracy under the premise of privacy protection proposed by the present invention, the method comprising the following steps:
[0067] (1) System initialization: The central node module sends the medical image recognition model to all user node modules; in the embodiment, the initial parameters of the medical image recognition model are random small values;
[0068] (2) The user node module receives the latest parameters of the medical image recognition model sent by the central node module and updates its local medical image recognition model;
[0069] (3) The user node module uses its own local medical image library to perform rounds of training on its local medical image recognition model; in the embodiment, the number of training rounds is determined by the user node module itself;
[0070] (4) The user node module uses the medical image recognition model trained in this round to generate a twin noise image library of the local medical image library;
[0071] (5) The user node module uses the medical image recognition model trained in this round to calculate the output noiseLogits of each twin noise image in the twin noise image library;
[0072] (6) The user node module sends the parameters of the medical image recognition model trained in this round, the twin noise image library, and the output noiseLogits of each twin noise image in the twin noise image library to the central node module;
[0073] (7) The central node module updates the parameters of the medical image recognition model based on the parameters of the medical image recognition model received from all user node modules in this round;
[0074] (8) The central node module retrains the updated medical image recognition model based on the twin noise image library received from all user node modules in this round and the output noiseLogits of each twin noise image in the twin noise image library;
[0075] (9) The central node module sends the parameters of the medical image recognition model retrained in this round to all the user node modules;
[0076] (10) Repeat steps (2) to (9) until the medical image recognition model is fully trained.
[0077] The inventors have conducted a large number of experiments on the proposed system and method, and the experimental results show that the system and method proposed in the present invention are feasible and effective.
[0078] It should be pointed out that the system and method proposed in the present invention are not only applicable to the field of medical imaging, but can also be widely applied to other fields that require centralized data processing under the premise of data privacy protection.
Claims
1. A system for improving image recognition accuracy under the premise of privacy protection, characterized by: The system includes n user node modules and one central node module, wherein the user node modules and the central node module are interconnected; n is a natural number greater than 1; the specifics of each module are as follows: User node module: The functions of this module are: (1) to train the medical image recognition model in rounds using its own local medical image library; (2) to send the parameters of the medical image recognition model trained in each round to the central node module; (3) Using the medical image recognition model trained in this round, generating a twin noise image library of the local medical image library, and sending the twin noise image library to the central module; (4) Using the medical image recognition model trained in this round, calculate the output noiseLogits of each twin noise image in the twin noise image library, and send the output noiseLogits to the central node module; (4) Receive the latest parameters of the medical image recognition model sent by the central node module, and update the medical image recognition model; Central node module: The functions of this module are: (1) sending the medical image recognition model to all user node modules; (2) updating the parameters of the medical image recognition model based on the parameters of the medical image recognition model received from all user node modules in this round; (3) retraining the updated medical image recognition model based on the twin noise image library received from all user node modules in this round and the output noiseLogits of each twin noise image in the twin noise image library; (4) sending the parameters of the medical image recognition model trained in this round to the user node module; The user node module uses the medical image recognition model trained in this round to generate the twin noise image library of the local medical image library. The specific content is: (1) For each class image in the local medical image library, the average output of the medical image recognition model is calculated according to the following formula: In the above formula, Represents the local medical image library of the i-th user node module, which has a total of images from c0 to c k-1 k categories; avgLogits c Represents the local medical image library for the i-th user node module The average output calculated by the medical image recognition model for the c-th image in ; represents the output obtained by inputting sample x into the medical image recognition model that has completed the tth round of training; y represents the category of sample x; (2) A random noise image is constructed. The medical image recognition model that has completed the tth round of training is used to update the constructed noise image using the following loss function according to the gradient descent back propagation method: In the above formula, Represents the KL divergence entropy function: Represents the softmax function; noiseLogits c Indicates the output obtained by inputting the noise image into the medical image recognition model that has completed the t-th round of training; after the noise image is updated a set number of times or the loss function value is less than the set threshold, the noise image obtained by the above iterative update is the local medical image library of the i-th user node module. A twin noise image corresponding to the c-th image in .
2. The system for improving image recognition accuracy under the premise of privacy protection according to claim 1, characterized in that: The central node module updates the parameters of the medical image recognition model based on the parameters of the medical image recognition model received from all user node modules in this round. The specific content is: The parameters of the medical image recognition model are updated according to the following formula: In the above formula, SetC t Represents a collection of user nodes; is the parameter of the medical image recognition model sent by the i-th user node module at round t The calculated parameter change is in is the parameter of the medical image recognition model sent by the i-th user node module at round t-1; w t is the parameter of the medical image recognition model of the central node module at round t; γ is a weight parameter.
3. The system for improving image recognition accuracy under the premise of privacy protection according to claim 1, characterized in that: The central node module retrains the updated medical image recognition model based on the twin noise image library received from all user node modules in this round and the output noiseLogits of each twin noise image in the twin noise image library. The specific content is: The data in the twin noise image library and the output noiseLogits of each twin noise image in the twin noise image library are divided into batches. For each batch of data b, the medical image recognition model is retrained according to the following loss function: In the above formula, represents the KL divergence entropy function; represents the softmax function; nd represents a received twin noise image, and nt represents the output noiseLogits of the medical image recognition model of the user node module that sends nd for nd; It represents the output obtained by the central node module when it inputs nd into the medical image recognition model that has completed the t+1th round of training; After a set number of rounds of training, the latest model parameters of the medical image recognition model are obtained.
4. A method for improving image recognition accuracy under the premise of privacy protection according to claim 1, characterized in that: The method comprises the following steps: (1) System initialization: The central node module sends the medical image recognition model to all user node modules; (2) The user node module receives the latest parameters of the medical image recognition model sent by the central node module and updates its own local medical image recognition model; (3) The user node module uses its own local medical image library to perform rounds of training on its local medical image recognition model; (4) The user node module uses the medical image recognition model trained in this round to generate a twin noise image library of the local medical image library; (5) The user node module uses the medical image recognition model trained in this round to calculate the output noiseLogits of each twin noise image in the twin noise image library; (6) The user node module sends the parameters of the medical image recognition model trained in this round, the twin noise image library, and the output noiseLogits of each twin noise image in the twin noise image library to the central node module: (7) The central node module updates the parameters of the medical image recognition model based on the parameters of the medical image recognition model received from all user node modules in this round; (8) The central node module retrains the updated medical image recognition model based on the twin noise image library received from all user node modules in this round and the output noiseLogits of each twin noise image in the twin noise image library; (9) The central node module sends the parameters of the medical image recognition model retrained in this round to all the user node modules; (10) Repeat steps (2) to (9) until the medical image recognition model is fully trained.
Citation Information
Patent Citations
Image recognition method and system based on improved PATE
CN114078203A
Federated learning model-based view display method, apparatus and device, and medium
WO2021219080A1
