A multi-center medical image segmentation method and system based on deep learning

Abstract

The present invention provides a multi-center medical image segmentation method and system based on deep learning, belonging to the fields of machine learning, medical Internet of Things, and information encryption transmission technology. The multi-center medical image segmentation method includes: obtaining a medical image to be segmented; encrypting the medical image to be segmented using an image encryption key to obtain an image ciphertext; segmenting the image ciphertext based on a pre-trained image self-segmentation network to obtain a mask ciphertext; the image self-segmentation network is pre-trained using a training sample set; the training sample set includes multiple training samples; each training sample includes a sample image ciphertext and a corresponding sample mask ciphertext; the training samples in the training sample set come from multiple medical institutions; and decrypting the mask ciphertext using a mask decryption key to obtain the segmented medical image. This method improves the security of medical images and the accuracy of medical image segmentation.

Description

Technical Field

[0001] The present invention relates to the fields of machine learning, medical Internet of Things, and information encryption transmission technology, and in particular to a multi-center medical image segmentation method and system based on deep learning. Background Art

[0002] With the development of a modern information society, the Medical Internet of Things (IoT) is increasingly playing a more important role in public health. The IoT aims to enable intelligent and informatized processes such as equipment management, information transmission management, and supply chain management. The IoT connects numerous medical devices, data systems, and terminals. Patient medical information is stored in hospital information systems via image archiving and communication systems. Hospital information systems then deliver the patient information needed by doctors to their workstations.

[0003] This medical information includes patient examination images obtained by medical imaging devices, such as MRI (Nuclear Magnetic Resonance Imaging) and CT (Computed Tomography). Medical images are part of a medical institution's electronic assets, containing a wealth of sensitive patient information and holding significant significance for disease research. Existing medical Internet of Things (IoT) deployments generally lack privacy protection for medical images. Images are mostly transmitted in plaintext within the IoT network and protected by passive network defenses such as firewalls. This defense approach presents significant security risks. It is not only ineffective in combating new types of external cyberattacks, but also lacks protection against malicious attacks from within. Malicious users can steal medical information, including images, by taking over user terminals or launching malicious attacks against medical devices, resulting in privacy breaches. Therefore, ensuring secure, encrypted transmission of medical images is essential for the development of the IoT.

[0004] Currently, medical image self-segmentation based on deep learning methods such as convolutional neural networks and generative adversarial networks has demonstrated excellent segmentation results. By training deep learning models with labeled medical image data, the models can automatically segment and annotate unlabeled image test data, such as patient organs or lesions. Medical images segmented using deep learning methods can provide reference and support for doctors' diagnoses. However, due to the independent nature of the Medical Internet of Things (IoT) within each medical institution, medical information suffers from severe data silos. Limited by the small number and lack of richness of medical image datasets, the segmentation performance and universality of deep learning models in practical applications still need to be improved.

[0005] To leverage data from various medical institutions to expand training datasets, jointly train deep learning models, achieve multi-center data consensus, and protect data privacy, a trusted distributed training model is required. Currently, one approach for joint training of multi-center data is federated learning, which enables multi-center joint modeling by sharing gradients rather than data. However, since federated learning models, which assume a trusted intermediate server, cannot complete testing tasks such as segmentation without uploading test data, they are not suitable for multi-center private joint modeling and medical image self-segmentation tasks. Another privacy-preserving solution is homomorphic encryption, which achieves absolute data security by encrypting data with homomorphism. However, since ciphertext homomorphism only applies to addition and multiplication operations, it cannot support more complex polynomial function calculations. Furthermore, due to its high computational overhead and slow operation speed, homomorphic encryption is also unsuitable for this type of distributed private joint training task. Summary of the Invention

[0006] The purpose of the present invention is to provide a multi-center medical image segmentation method and system based on deep learning, which can improve the safety of medical images while improving the accuracy of medical image segmentation.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A multi-center medical image segmentation method based on deep learning, comprising:

[0009] Obtaining medical images to be segmented;

[0010] Encrypting the medical image to be segmented using an image encryption key to obtain an image ciphertext;

[0011] Based on a pre-trained image self-segmentation network, the image ciphertext is segmented to obtain a mask ciphertext; the image self-segmentation network is pre-trained using a training sample set; the training sample set includes multiple training samples; each training sample includes a sample image ciphertext and a corresponding sample mask ciphertext; the training samples in the training sample set come from multiple medical institutions;

[0012] The mask decryption key is used to decrypt the mask ciphertext to obtain the segmented medical image.

[0013] Optionally, the step of encrypting the medical image to be segmented using an image encryption key to obtain an image ciphertext specifically includes:

[0014] A pre-trained image encryption network is used to extract features from the medical image to be segmented to obtain an image ciphertext; the image encryption key is a parameter of the trained image encryption network;

[0015] The image encryption network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module and a fifth feature extraction module connected in sequence; the first feature extraction module, the second feature extraction module, the third feature extraction module and the fourth feature extraction module each include two convolutional layers and a maximum pooling layer connected in sequence; the first convolutional layer of the second feature extraction module is also connected to the maximum pooling layer of the first feature extraction module; the first convolutional layer of the third feature extraction module is also connected to the maximum pooling layer of the second feature extraction module; the first convolutional layer of the fourth feature extraction module is also connected to the maximum pooling layer of the third feature extraction module; the fifth feature extraction module includes two convolutional layers connected in sequence; the first convolutional layer of the fifth feature extraction module is also connected to the maximum pooling layer of the fourth feature extraction module.

[0016] Optionally, each convolution layer in the first feature extraction module, the second feature extraction module, the third feature extraction module, the fourth feature extraction module and the fifth feature extraction module includes a convolution kernel of size 3*3 and a ReLU activation function.

[0017] Optionally, the convolution kernel size of each maximum pooling layer in the first feature extraction module, the second feature extraction module, the third feature extraction module and the fourth feature extraction module is 2*2.

[0018] Optionally, the image self-segmentation network includes a first segmentation module, a second segmentation module, a third segmentation module, a fourth segmentation module, a fifth segmentation module, a sixth segmentation module, a seventh segmentation module, an eighth segmentation module, and a ninth segmentation module connected in sequence; the first segmentation module is further connected to the ninth segmentation module; the second segmentation module is further connected to the eighth segmentation module; the third segmentation module is further connected to the seventh segmentation module; and the fourth segmentation module is further connected to the sixth segmentation module;

[0019] The first segmentation module includes two convolutional layers and a pyramid pooling layer connected in sequence; the second segmentation module, the third segmentation module, the fourth segmentation module, and the fifth segmentation module each include an upsampling layer and two convolutional layers connected in sequence; the sixth segmentation module, the seventh segmentation module, and the eighth segmentation module each include a maximum pooling layer and two convolutional layers connected in sequence; the ninth segmentation module includes a maximum pooling layer, two convolutional layers, a pyramid pooling layer, and a convolutional layer connected in sequence;

[0020] The image ciphertext is segmented based on a pre-trained image self-segmentation network to obtain mask ciphertext, specifically including:

[0021] Performing a convolution operation and a pyramid pooling operation on the image ciphertext in sequence through a first segmentation module to obtain a first segmentation ciphertext;

[0022] Performing upsampling and convolution operations on the first segmented ciphertext in sequence through a second segmentation module to obtain a second segmented ciphertext;

[0023] performing upsampling and convolution operations on the second split ciphertext in sequence through a third split module to obtain a third split ciphertext;

[0024] performing upsampling and convolution operations on the third segmented ciphertext in sequence through a fourth segmentation module to obtain a fourth segmented ciphertext;

[0025] performing upsampling and convolution operations on the fourth segmented ciphertext in sequence through a fifth segmentation module to obtain a fifth segmented ciphertext;

[0026] Performing maximum pooling and dimensionality expansion operations on the fifth segmented ciphertext in sequence through a sixth segmentation module to obtain a first intermediate ciphertext, and concatenating the first intermediate ciphertext with the fourth segmented ciphertext and performing a convolution operation to obtain a sixth segmented ciphertext;

[0027] Performing maximum pooling and dimensionality expansion operations on the sixth segmented ciphertext in sequence through the seventh segmentation module to obtain a second intermediate ciphertext, and concatenating the second intermediate ciphertext with the third segmented ciphertext and performing a convolution operation to obtain a seventh segmented ciphertext;

[0028] Performing maximum pooling and dimensionality expansion operations on the seventh segmented ciphertext in sequence through an eighth segmentation module to obtain a third intermediate ciphertext, and concatenating the third intermediate ciphertext with the second segmented ciphertext and performing a convolution operation thereon to obtain an eighth segmented ciphertext;

[0029] The ninth segmentation module sequentially performs maximum pooling and dimension expansion operations on the eighth segmentation ciphertext to obtain a fourth intermediate ciphertext, and the fourth intermediate ciphertext is concatenated with the first segmentation ciphertext and sequentially performs convolution operations, pyramid pooling operations, and convolution operations to obtain a mask ciphertext.

[0030] Optionally, the training process of the image self-segmentation network includes:

[0031] For any medical institution, a plurality of sample images and corresponding plurality of sample masks are obtained through the medical institution;

[0032] Encrypting each sample image using the medical institution's image encryption key to obtain corresponding multiple sample image ciphertexts;

[0033] Encrypting each sample mask using the medical institution's mask encryption key to obtain corresponding multiple sample mask ciphertexts;

[0034] According to the training samples of various medical institutions, the image self-segmentation network is trained based on the loss function until the loss function converges to obtain a trained image self-segmentation network.

[0035] Optionally, each sample mask is encrypted using the mask encryption key of the medical institution to obtain corresponding multiple sample mask ciphertexts, specifically including:

[0036] For any sample mask, a pre-trained mask encryption network is used to extract features from the sample mask to obtain the corresponding sample mask ciphertext; the mask encryption key is the parameter of the trained mask encryption network;

[0037] The mask encryption network includes a sixth feature extraction module, a seventh feature extraction module, an eighth feature extraction module, a ninth feature extraction module and a tenth feature extraction module connected in sequence; the sixth feature extraction module, the seventh feature extraction module, the eighth feature extraction module and the ninth feature extraction module each include two convolutional layers and a maximum pooling layer connected in sequence; the first convolutional layer of the seventh feature extraction module is also connected to the maximum pooling layer of the sixth feature extraction module; the first convolutional layer of the eighth feature extraction module is also connected to the maximum pooling layer of the seventh feature extraction module; the first convolutional layer of the ninth feature extraction module is also connected to the maximum pooling layer of the eighth feature extraction module; the tenth feature extraction module includes two convolutional layers connected in sequence; the first convolutional layer of the tenth feature extraction module is also connected to the maximum pooling layer of the ninth feature extraction module.

[0038] Optionally, decrypting the mask ciphertext using a mask decryption key to obtain the segmented medical image specifically includes:

[0039] Decrypting the mask ciphertext using a pre-trained mask decryption network to obtain the segmented medical image; the mask decryption key is the parameters of the trained mask decryption network;

[0040] The mask decryption network includes a first decryption module, a second decryption module, a third decryption module and a fourth decryption module connected in sequence; the first decryption module, the second decryption module and the third decryption module each include an upsampling layer and two convolutional layers connected in sequence; the fourth decryption module includes an upsampling layer and three convolutional layers connected in sequence.

[0041] To achieve the above object, the present invention also provides the following solution:

[0042] A multi-center medical image segmentation system based on deep learning, including: a key management server, medical equipment, an image archiving communication subsystem, a hospital information subsystem, and a doctor workstation;

[0043] The key management server is used to store the image encryption key and mask decryption key of the corresponding medical institution;

[0044] The medical device is used to collect medical images to be segmented;

[0045] The image archiving communication subsystem includes:

[0046] an image encryption unit, connected to the medical device, the key management server, and the hospital information subsystem, respectively, for encrypting the medical image to be segmented using an image encryption key to obtain an image ciphertext, and sending the image ciphertext to the hospital information subsystem;

[0047] An image segmentation unit is connected to the image acquisition unit and the hospital information subsystem, respectively, and is used to segment the image ciphertext based on a pre-trained image self-segmentation network to obtain mask ciphertext, and send the mask ciphertext to the hospital information subsystem; the image self-segmentation network is pre-trained using a training sample set; the training sample set includes multiple training samples; each training sample includes a sample image ciphertext and a corresponding sample mask ciphertext; the training samples in the training sample set come from multiple medical institutions;

[0048] The mask decryption unit is connected to the image segmentation unit, the key management server, the hospital information subsystem and the doctor workstation respectively, and is used to use the mask decryption key to decrypt the mask ciphertext to obtain the segmented medical image, and send the segmented medical image to the doctor workstation.

[0049] Optionally, the image archiving communication subsystem is further configured to send an encryption request to the key management server when encrypting the medical image to be segmented, and to send a decryption request to the key management server when decrypting the mask ciphertext;

[0050] The key management server is further configured to perform identity authentication according to the encryption request or the decryption request, and after the identity authentication is passed, send the image encryption key or the mask decryption key to the image archiving communication subsystem.

[0051] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: first, an image encryption key is used to encrypt the medical image to be segmented to obtain an image ciphertext; then, the image ciphertext is segmented based on a pre-trained image self-segmentation network to obtain a mask ciphertext; the image self-segmentation network is pre-trained using a training sample set, the training sample set includes multiple training samples, each training sample includes a sample image ciphertext and a corresponding sample mask ciphertext, and the training samples in the training sample set come from multiple medical institutions; finally, the mask ciphertext is decrypted using a mask decryption key to obtain the segmented medical image. By encrypting the medical image, the security of the medical image is improved, and by training the image self-segmentation network in a multi-center manner, the accuracy of medical image segmentation is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is a flowchart of the multi-center medical image segmentation method based on deep learning of the present invention;

[0054] Figure 2 This is a schematic diagram of the structure of the image encryption network and the image decryption network;

[0055] Figure 3 This is a structural diagram of the image self-segmentation network;

[0056] Figure 4 It is a structural diagram of the mask encryption network and the mask decryption network;

[0057] Figure 5 Schematic diagram of the training process of the image self-segmentation network;

[0058] Figure 6 Schematic diagram of the multi-center medical image segmentation system based on deep learning of the present invention. DETAILED DESCRIPTION

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0060] The purpose of the present invention is to provide a multi-center medical image segmentation method and system based on deep learning, to realize the encrypted transmission of image data from multiple medical institutions, and to use the encrypted image data to jointly train the segmentation model, to achieve high-precision automatic segmentation of medical images based on deep learning, to provide assistance and reference for doctors' diagnosis of diseases, and to solve the problems of privacy protection and data silos.

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] Example 1

[0063] like Figure 1 As shown, the multi-center medical image segmentation method based on deep learning provided in this embodiment includes:

[0064] S1: Obtain the medical image to be segmented.

[0065] S2: Encrypt the medical image to be segmented using an image encryption key to obtain an image ciphertext.

[0066] S3: Segment the image ciphertext based on a pre-trained image self-segmentation network to obtain a mask ciphertext. The image self-segmentation network is pre-trained using a training sample set. The training sample set includes multiple training samples. Each training sample includes a sample image ciphertext and a corresponding sample mask ciphertext. The training samples in the training sample set come from multiple medical institutions.

[0067] S4: Decrypt the mask ciphertext using the mask decryption key to obtain the segmented medical image.

[0068] The present invention mainly includes an image encryption network, an image self-segmentation network and a mask decryption network.

[0069] Furthermore, step S2 specifically includes: using a pre-trained image encryption network to extract features from the medical image to be segmented, and mapping the medical image to be segmented into a high-dimensional domain through the image encryption network to obtain image ciphertext. The image encryption key is a parameter of the trained image encryption network.

[0070] In this embodiment, the image ciphertext can also be decrypted through the image decryption network. The image encryption and decryption network uses a symmetric convolutional network to achieve image reconstruction, where the image encryption network is the part of the complete network that obtains the image ciphertext from the medical image, and is intended to encrypt the medical image into a 128-channel image ciphertext. Figure 2As shown, the image encryption network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module and a fifth feature extraction module connected in sequence. The first feature extraction module, the second feature extraction module, the third feature extraction module and the fourth feature extraction module each include two convolutional layers and a maximum pooling layer connected in sequence. The first convolutional layer of the second feature extraction module is also connected to the maximum pooling layer of the first feature extraction module. The first convolutional layer of the third feature extraction module is also connected to the maximum pooling layer of the second feature extraction module. The first convolutional layer of the fourth feature extraction module is also connected to the maximum pooling layer of the third feature extraction module. The fifth feature extraction module includes two convolutional layers connected in sequence. The first convolutional layer of the fifth feature extraction module is also connected to the maximum pooling layer of the fourth feature extraction module.

[0071] The image decryption network includes a first image decryption module, a second image decryption module, a third image decryption module, and a fourth image decryption module, which are connected in sequence. The first, second, and third image decryption modules each include an upsampling layer and two convolutional layers connected in sequence. The fourth image decryption module includes an upsampling layer and three convolutional layers.

[0072] Specifically, each convolutional layer in the first feature extraction module, the second feature extraction module, the third feature extraction module, the fourth feature extraction module, and the fifth feature extraction module includes a 3*3 convolution kernel and a ReLU activation function. The convolution kernel size of each maximum pooling layer in the first feature extraction module, the second feature extraction module, the third feature extraction module, and the fourth feature extraction module is 2*2. Each convolutional layer in the first image decryption module, the second image decryption module, and the third image decryption module includes a 3*3 convolution kernel and a ReLU activation function. The first two convolutional layers in the fourth image decryption module include a 3*3 convolution kernel and a ReLU activation function, and the third convolutional layer includes a 1*1 convolution kernel and a ReLU activation function. The convolution kernel size of the upsampling layer in the first image decryption module, the second image decryption module, the third image decryption module, and the fourth image decryption module is 2*2.

[0073] The image encryption network uses a convolutional neural network to extract features. It performs multiple 3x3 convolutions and 2x2 max pooling operations on medical images, using the ReLU function as an activation function to increase network sparsity and avoid overfitting. Through multiple convolutions and max pooling operations, a single-channel medical image is converted into a 128-channel ciphertext.

[0074] Furthermore, the image self-segmentation network in step S3 obtains the segmented 128-channel mask ciphertext from the 128-channel image ciphertext. Figure 3As shown, the image self-segmentation network includes a first segmentation module, a second segmentation module, a third segmentation module, a fourth segmentation module, a fifth segmentation module, a sixth segmentation module, a seventh segmentation module, an eighth segmentation module, and a ninth segmentation module, which are connected in sequence. The first segmentation module is also connected to the ninth segmentation module. The second segmentation module is also connected to the eighth segmentation module. The third segmentation module is also connected to the seventh segmentation module. The fourth segmentation module is also connected to the sixth segmentation module.

[0075] The first segmentation module includes two convolutional layers and a pyramid pooling layer connected in sequence. The second, third, fourth, and fifth segmentation modules each include an upsampling layer and two convolutional layers connected in sequence. The sixth, seventh, and eighth segmentation modules each include a maximum pooling layer and two convolutional layers connected in sequence. The ninth segmentation module includes a maximum pooling layer, two convolutional layers, a pyramid pooling layer, and one convolutional layer connected in sequence.

[0076] Step S3 specifically includes:

[0077] (1) Performing convolution operations and pyramid pooling operations on the image ciphertext in sequence through a first segmentation module to obtain a first segmentation ciphertext.

[0078] (2) Performing upsampling and convolution operations on the first segmented ciphertext in sequence through the second segmentation module to obtain a second segmented ciphertext.

[0079] (3) Perform upsampling and convolution operations on the second segmented ciphertext in sequence through the third segmentation module to obtain a third segmented ciphertext.

[0080] (4) Perform upsampling and convolution operations on the third segmented ciphertext in sequence through the fourth segmentation module to obtain a fourth segmented ciphertext.

[0081] (5) Perform upsampling and convolution operations on the fourth segmented ciphertext in sequence through the fifth segmentation module to obtain a fifth segmented ciphertext.

[0082] (6) Performing maximum pooling and dimension expansion operations on the fifth segmented ciphertext in sequence through the sixth segmentation module to obtain a first intermediate ciphertext, and concatenating the first intermediate ciphertext with the fourth segmented ciphertext and performing a convolution operation to obtain a sixth segmented ciphertext.

[0083] (7) Performing maximum pooling and dimension expansion operations on the sixth segmentation ciphertext in sequence through the seventh segmentation module to obtain a second intermediate ciphertext, and concatenating the second intermediate ciphertext with the third segmentation ciphertext and performing a convolution operation to obtain a seventh segmentation ciphertext.

[0084] (8) Performing maximum pooling and dimension expansion operations on the seventh segmentation ciphertext in sequence through the eighth segmentation module to obtain a third intermediate ciphertext, and concatenating the third intermediate ciphertext with the second segmentation ciphertext and performing a convolution operation to obtain an eighth segmentation ciphertext.

[0085] (9) Performing maximum pooling and dimension expansion operations on the eighth segmentation ciphertext in sequence through the ninth segmentation module to obtain a fourth intermediate ciphertext, and concatenating the fourth intermediate ciphertext with the first segmentation ciphertext and performing convolution operations, pyramid pooling operations, and convolution operations in sequence to obtain a mask ciphertext.

[0086] The image self-segmentation network first performs two convolution operations on the 128-channel image ciphertext obtained by the image encryption network, followed by pyramid pooling. Pyramid pooling applies multiple different pooling operations to the features extracted by the convolutional network. Specifically, the feature maps are pooled into four different sizes and the number of channels is reduced to one-quarter through 1x1 convolution. The four feature maps are then bilinearly upsampled to restore them to their original size and concatenated with the original feature map to generate a new feature map with twice the number of channels. Pyramid pooling aims to obtain different receptive fields and global network context, thereby increasing global information utilization efficiency. Four sets of convolutions and upsampling are performed on the new feature map to enlarge it to a feature map with 32 channels. Max pooling is then performed on the feature map, and dimension padding is performed to match the dimensions of the intermediate result with the same number of channels during the upsampling process. The max-pooled and dimensionally padded feature map is then concatenated with the feature map with the same number of channels during the upsampling process and convolved twice. After performing this operation four times, pyramid pooling is performed and the number of channels is reduced to the original number through 1*1 convolution to obtain a masked ciphertext with 128 channels.

[0087] In this embodiment, the mask ciphertext is the ciphertext of the image that has been automatically segmented and labeled for the patient's organs or lesion areas.

[0088] The image self-segmentation network of the present invention is based on a convolutional neural network. It trains and segments image ciphertexts through methods such as pyramid pooling and intermediate feature splicing, generating masked ciphertexts that can be decrypted by a mask decryption network. This image self-segmentation network achieves high-precision segmentation while protecting image privacy, reducing physician workload and assisting in diagnosis.

[0089] Furthermore, step S4 specifically includes: using a pre-trained mask decryption network to decrypt the mask ciphertext to obtain the segmented medical image. The mask decryption key is a parameter of the pre-trained mask decryption network.

[0090] Specifically, the mask decryption network has a similar structure to the image encryption and decryption network, employing a symmetric convolutional network to reconstruct the mask. The mask decryption network is the portion of the mask encryption and decryption network that derives the mask from the mask ciphertext. Its purpose is to decrypt the 128-channel mask ciphertext encrypted by the mask encryption network into the mask image, which is the segmented medical image.

[0091] like Figure 4 As shown, the mask decryption network includes a first decryption module, a second decryption module, a third decryption module, and a fourth decryption module connected in sequence. The first decryption module, the second decryption module, and the third decryption module each include an upsampling layer and two convolutional layers connected in sequence. The fourth decryption module includes an upsampling layer and three convolutional layers connected in sequence.

[0092] In this embodiment, multi-center medical imaging data and mask images from multiple medical institutions are used to jointly train the image self-segmentation network, thereby obtaining a higher performance and more universal image self-segmentation network. During the training process, the original data of each medical institution is used locally to train the local encryption and decryption network. The image ciphertext and mask ciphertext used to jointly train the image self-segmentation network can only be decrypted using their local decryption network, and cannot be decrypted using the decryption network of other medical institutions. This protects the privacy and security of the original data of each medical institution, i.e., the medical images. The training process of the image self-segmentation network is as follows: Figure 5 As shown in Figure 2. The training process of the image self-segmentation network includes:

[0093] S5: For any medical institution, a plurality of sample images and corresponding plurality of sample masks are obtained through the medical institution.

[0094] S6: Encrypt each sample image using the image encryption key of the medical institution to obtain corresponding multiple sample image ciphertexts.

[0095] S7: Encrypt each sample mask using the mask encryption key of the medical institution to obtain corresponding multiple sample mask ciphertexts.

[0096] Specifically, for any sample mask, a pre-trained mask encryption network is used to extract features from the sample mask to obtain a corresponding sample mask ciphertext. The mask encryption key is a parameter of the pre-trained mask encryption network.

[0097] In which, the mask encryption network includes a sixth feature extraction module, a seventh feature extraction module, an eighth feature extraction module, a ninth feature extraction module and a tenth feature extraction module connected in sequence. The sixth feature extraction module, the seventh feature extraction module, the eighth feature extraction module and the ninth feature extraction module each include two convolutional layers and a maximum pooling layer connected in sequence. The first convolutional layer of the seventh feature extraction module is also connected to the maximum pooling layer of the sixth feature extraction module; the first convolutional layer of the eighth feature extraction module is also connected to the maximum pooling layer of the seventh feature extraction module. The first convolutional layer of the ninth feature extraction module is also connected to the maximum pooling layer of the eighth feature extraction module; the tenth feature extraction module includes two convolutional layers connected in sequence. The first convolutional layer of the tenth feature extraction module is also connected to the maximum pooling layer of the ninth feature extraction module.

[0098] S8: Based on the training samples of each medical institution and the loss function, the image self-segmentation network is trained until the loss function converges to obtain a trained image self-segmentation network.

[0099] First, each medical institution uses the hospital's medical images and their mask images to train the image encryption and decryption network and the mask encryption and decryption network on the local server respectively.

[0100] Specifically, the deep learning model is trained based on the loss function, and the loss function of the image encryption and decryption network adopts the mean square error function:

[0101]

[0102] Among them, MSE(y,y') is the loss function value of the image encryption and decryption network, m is the number of sample images, y i is the actual image ciphertext of sample image i, y' i The image ciphertext predicted by the image encryption and decryption network for sample image i.

[0103] The loss function of the mask encryption and decryption network adopts the cross entropy function:

[0104] H(p,q)=-∑ x (p(x)logq(x)+(1-p(x))log(1-q(x)));

[0105] Among them, H(p,q) is the loss function value of the mask encryption and decryption network, p(x) is the true probability distribution of the mask encryption and decryption network for sample mask x, and q(x) is the non-true probability distribution of the mask encryption and decryption network for sample mask x.

[0106] The encryption and decryption network is trained to minimize the loss function, that is, the decrypted image is as close to the original image as possible.

[0107] After completing the training, each medical institution will store the final parameters of its encryption and decryption network in its own key management server as encryption and decryption keys.

[0108] Finally, the image ciphertext obtained by the image encryption network of each medical institution is used, and the mask ciphertext is obtained by using the mask encryption network to form a sample image ciphertext set and a sample mask ciphertext set. The mean square error loss function is used to train the image segmentation network using the sample image ciphertext set and the sample mask ciphertext set to complete the multi-center medical image self-segmentation and encryption transmission network training process.

[0109] The present invention addresses the privacy protection risks and data silos in the medical Internet of Things system, realizes the encrypted transmission of image data from multiple medical institutions, and uses the encrypted multi-center image data to jointly train the image self-segmentation network, breaking the medical image information silos between hospitals, and realizing high-precision automatic segmentation of medical images based on deep learning. By expanding the training sample set, an image self-segmentation network with stronger performance and universality is obtained, while protecting the image data privacy of each hospital, and realizing data cooperation and mutual benefit among multiple hospitals while ensuring information security.

[0110] Example 2

[0111] In order to execute the method corresponding to the above-mentioned embodiment 1 and achieve the corresponding functions and technical effects, a multi-center medical image segmentation system based on deep learning is provided below.

[0112] like Figure 6 As shown, the multi-center medical image segmentation system based on deep learning provided in this embodiment includes: a key management server, medical equipment, an image archiving communication subsystem, a hospital information subsystem and a doctor workstation.

[0113] The key management server is used to store the image encryption key and mask decryption key of the corresponding medical institution.

[0114] The medical device is used to collect medical images to be segmented.

[0115] The image archiving communication subsystem includes: an image encryption unit, an image segmentation unit and a mask decryption unit.

[0116] Among them, the image encryption unit is connected to the medical device, the key management server and the hospital information subsystem respectively. The image encryption unit is used to encrypt the medical image to be segmented using the image encryption key, obtain the image ciphertext, and send the image ciphertext to the hospital information subsystem.

[0117] The image segmentation unit is connected to the image acquisition unit and the hospital information subsystem respectively. The image segmentation unit is used to segment the image ciphertext based on a pre-trained image self-segmentation network to obtain mask ciphertext, and send the mask ciphertext to the hospital information subsystem; the image self-segmentation network is pre-trained using a training sample set; the training sample set includes multiple training samples; each training sample includes a sample image ciphertext and a corresponding sample mask ciphertext; the training samples in the training sample set come from multiple medical institutions.

[0118] The mask decryption unit is connected to the image segmentation unit, the key management server, the hospital information subsystem and the doctor workstation respectively. The mask decryption unit is used to decrypt the mask ciphertext using the mask decryption key to obtain the segmented medical image and send the segmented medical image to the doctor workstation.

[0119] The image archiving and communication subsystem encrypts medical images obtained by medical devices through an image encryption network. The medical images are then mapped to a high-dimensional domain via the encryption network, resulting in an image ciphertext. This image ciphertext is then stored in the hospital information subsystem and processed through the image self-segmentation network to obtain a mask ciphertext. After image segmentation, the image archiving and communication subsystem decrypts the mask ciphertext through a mask decryption network to obtain the segmented image. This image is then stored in the hospital information subsystem and delivered to the doctor's workstation to assist in diagnosis.

[0120] The hospital information subsystem is responsible for storing the image ciphertext and mask ciphertext from the image archiving and communication subsystem. When a doctor sends a request to the hospital information subsystem to retrieve an existing image, the mask ciphertext is sent to the image archiving and communication subsystem for decryption and then delivered to the doctor's workstation.

[0121] Furthermore, the image archiving communication subsystem is also used to send an encryption request to the key management server when encrypting the medical image to be segmented, and to send a decryption request to the key management server when decrypting the mask ciphertext.

[0122] The key management server is further configured to perform identity authentication and security review based on the encryption request or decryption request, and after the identity authentication is passed and the request is confirmed to be legitimate, send the image encryption key or mask decryption key to the image archiving communication subsystem. The image encryption network parameters serve as the image encryption key, and the mask decryption network parameters serve as the mask decryption key.

[0123] The doctor's workstation receives the segmented medical images from the image archiving and communication subsystem and provides the patient's medical images and masks to assist the doctor in diagnosis. When the doctor needs to retrieve existing patient medical image data, the doctor's workstation sends a request to the image archiving and communication subsystem, which retrieves the corresponding mask ciphertext from the hospital information subsystem. The key management server performs security authentication and provides the decryption key, which is then decrypted and delivered to the doctor's workstation.

[0124] Compared with the prior art, the multi-center medical image segmentation system based on deep learning provided in this embodiment has the same beneficial effects as the multi-center medical image segmentation method based on deep learning provided in Example 1, which will not be repeated here.

[0125] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0126] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A multi-center medical image segmentation method based on deep learning, characterized in that: The multi-center medical image segmentation method based on deep learning includes: Obtaining medical images to be segmented; The medical image to be segmented is encrypted using an image encryption key to obtain an image ciphertext, specifically comprising: extracting features from the medical image to be segmented using a pre-trained image encryption network to obtain an image ciphertext; the image encryption key is a parameter of the trained image encryption network; the image encryption network comprises a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module and a fifth feature extraction module connected in sequence; the first feature extraction module, the second feature extraction module, the third feature extraction module and the fourth feature extraction module each comprise two convolutional layers and a maximum pooling layer connected in sequence; the first convolutional layer of the second feature extraction module is also connected to the maximum pooling layer of the first feature extraction module; the first convolutional layer of the third feature extraction module is also connected to the maximum pooling layer of the second feature extraction module; the first convolutional layer of the fourth feature extraction module is also connected to the maximum pooling layer of the third feature extraction module; the fifth feature extraction module comprises two convolutional layers connected in sequence; the first convolutional layer of the fifth feature extraction module is also connected to the maximum pooling layer of the fourth feature extraction module; Based on a pre-trained image self-segmentation network, the image ciphertext is segmented to obtain a mask ciphertext; the image self-segmentation network is pre-trained using a training sample set; the training sample set includes multiple training samples; each training sample includes a sample image ciphertext and a corresponding sample mask ciphertext; the training samples in the training sample set come from multiple medical institutions; The mask decryption key is used to decrypt the mask ciphertext to obtain the segmented medical image.

2. The multi-center medical image segmentation method based on deep learning according to claim 1, characterized in that: Each convolution layer in the first feature extraction module, the second feature extraction module, the third feature extraction module, the fourth feature extraction module and the fifth feature extraction module includes a 3*3 convolution kernel and a ReLU activation function.

3. The multi-center medical image segmentation method based on deep learning according to claim 1, characterized in that: The convolution kernel size of each maximum pooling layer in the first feature extraction module, the second feature extraction module, the third feature extraction module and the fourth feature extraction module is 2*2.

4. The multi-center medical image segmentation method based on deep learning according to claim 1, characterized in that: The image self-segmentation network includes a first segmentation module, a second segmentation module, a third segmentation module, a fourth segmentation module, a fifth segmentation module, a sixth segmentation module, a seventh segmentation module, an eighth segmentation module, and a ninth segmentation module connected in sequence; the first segmentation module is further connected to the ninth segmentation module; the second segmentation module is further connected to the eighth segmentation module; the third segmentation module is further connected to the seventh segmentation module; and the fourth segmentation module is further connected to the sixth segmentation module; The first segmentation module includes two convolutional layers and a pyramid pooling layer connected in sequence; the second segmentation module, the third segmentation module, the fourth segmentation module, and the fifth segmentation module each include an upsampling layer and two convolutional layers connected in sequence; the sixth segmentation module, the seventh segmentation module, and the eighth segmentation module each include a maximum pooling layer and two convolutional layers connected in sequence; the ninth segmentation module includes a maximum pooling layer, two convolutional layers, a pyramid pooling layer, and a convolutional layer connected in sequence; The image ciphertext is segmented based on a pre-trained image self-segmentation network to obtain mask ciphertext, specifically including: Performing a convolution operation and a pyramid pooling operation on the image ciphertext in sequence through a first segmentation module to obtain a first segmentation ciphertext; Performing upsampling and convolution operations on the first segmented ciphertext in sequence through a second segmentation module to obtain a second segmented ciphertext; performing upsampling and convolution operations on the second split ciphertext in sequence through a third split module to obtain a third split ciphertext; performing upsampling and convolution operations on the third segmented ciphertext in sequence through a fourth segmentation module to obtain a fourth segmented ciphertext; performing upsampling and convolution operations on the fourth segmented ciphertext in sequence through a fifth segmentation module to obtain a fifth segmented ciphertext; Performing maximum pooling and dimensionality expansion operations on the fifth segmented ciphertext in sequence through a sixth segmentation module to obtain a first intermediate ciphertext, and concatenating the first intermediate ciphertext with the fourth segmented ciphertext and performing a convolution operation to obtain a sixth segmented ciphertext; Performing maximum pooling and dimensionality expansion operations on the sixth segmented ciphertext in sequence through the seventh segmentation module to obtain a second intermediate ciphertext, and concatenating the second intermediate ciphertext with the third segmented ciphertext and performing a convolution operation to obtain a seventh segmented ciphertext; Performing maximum pooling and dimensionality expansion operations on the seventh segmented ciphertext in sequence through an eighth segmentation module to obtain a third intermediate ciphertext, and concatenating the third intermediate ciphertext with the second segmented ciphertext and performing a convolution operation thereon to obtain an eighth segmented ciphertext; The ninth segmentation module sequentially performs maximum pooling and dimension expansion operations on the eighth segmentation ciphertext to obtain a fourth intermediate ciphertext, and the fourth intermediate ciphertext is concatenated with the first segmentation ciphertext and sequentially performs convolution operations, pyramid pooling operations, and convolution operations to obtain a mask ciphertext.

5. The multi-center medical image segmentation method based on deep learning according to claim 1, characterized in that: The training process of the image self-segmentation network includes: For any medical institution, a plurality of sample images and corresponding plurality of sample masks are obtained through the medical institution; Encrypting each sample image using the medical institution's image encryption key to obtain corresponding multiple sample image ciphertexts; Encrypting each sample mask using the medical institution's mask encryption key to obtain corresponding multiple sample mask ciphertexts; According to the training samples of various medical institutions, the image self-segmentation network is trained based on the loss function until the loss function converges to obtain a trained image self-segmentation network.

6. The multi-center medical image segmentation method based on deep learning according to claim 1, characterized in that: Each sample mask is encrypted using the medical institution's mask encryption key to obtain corresponding multiple sample mask ciphertexts, specifically including: For any sample mask, a pre-trained mask encryption network is used to extract features from the sample mask to obtain the corresponding sample mask ciphertext; the mask encryption key is the parameter of the trained mask encryption network; The mask encryption network includes a sixth feature extraction module, a seventh feature extraction module, an eighth feature extraction module, a ninth feature extraction module and a tenth feature extraction module connected in sequence; the sixth feature extraction module, the seventh feature extraction module, the eighth feature extraction module and the ninth feature extraction module each include two convolutional layers and a maximum pooling layer connected in sequence; the first convolutional layer of the seventh feature extraction module is also connected to the maximum pooling layer of the sixth feature extraction module; the first convolutional layer of the eighth feature extraction module is also connected to the maximum pooling layer of the seventh feature extraction module; the first convolutional layer of the ninth feature extraction module is also connected to the maximum pooling layer of the eighth feature extraction module; the tenth feature extraction module includes two convolutional layers connected in sequence; the first convolutional layer of the tenth feature extraction module is also connected to the maximum pooling layer of the ninth feature extraction module.

7. The multi-center medical image segmentation method based on deep learning according to claim 1, characterized in that: Decrypting the mask ciphertext using the mask decryption key to obtain the segmented medical image specifically includes: Decrypting the mask ciphertext using a pre-trained mask decryption network to obtain the segmented medical image; the mask decryption key is the parameters of the trained mask decryption network; The mask decryption network includes a first decryption module, a second decryption module, a third decryption module and a fourth decryption module connected in sequence; the first decryption module, the second decryption module and the third decryption module each include an upsampling layer and two convolutional layers connected in sequence; the fourth decryption module includes an upsampling layer and three convolutional layers connected in sequence.

8. A multi-center medical image segmentation system based on deep learning, characterized in that: The multi-center medical image segmentation system based on deep learning includes: a key management server, medical equipment, an image archiving communication subsystem, a hospital information subsystem and a doctor workstation; The key management server is used to store the image encryption key and mask decryption key of the corresponding medical institution; The medical device is used to collect medical images to be segmented; The image archiving communication subsystem includes: An image encryption unit is connected to the medical device, the key management server and the hospital information subsystem respectively, and is used to encrypt the medical image to be segmented using an image encryption key to obtain an image ciphertext, and send the image ciphertext to the hospital information subsystem; wherein, a pre-trained image encryption network is used to extract features of the medical image to be segmented to obtain an image ciphertext; the image encryption key is a parameter of the trained image encryption network; the image encryption network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module and a fifth feature extraction module connected in sequence; the first feature extraction module, The second feature extraction module, the third feature extraction module, and the fourth feature extraction module each include two convolutional layers and a maximum pooling layer connected in sequence; the first convolutional layer of the second feature extraction module is also connected to the maximum pooling layer of the first feature extraction module; the first convolutional layer of the third feature extraction module is also connected to the maximum pooling layer of the second feature extraction module; the first convolutional layer of the fourth feature extraction module is also connected to the maximum pooling layer of the third feature extraction module; the fifth feature extraction module includes two convolutional layers connected in sequence; the first convolutional layer of the fifth feature extraction module is also connected to the maximum pooling layer of the fourth feature extraction module; An image segmentation unit is connected to the image acquisition unit and the hospital information subsystem, respectively, and is used to segment the image ciphertext based on a pre-trained image self-segmentation network to obtain mask ciphertext, and send the mask ciphertext to the hospital information subsystem; the image self-segmentation network is pre-trained using a training sample set; the training sample set includes multiple training samples; each training sample includes a sample image ciphertext and a corresponding sample mask ciphertext; the training samples in the training sample set come from multiple medical institutions; The mask decryption unit is connected to the image segmentation unit, the key management server, the hospital information subsystem and the doctor workstation respectively, and is used to use the mask decryption key to decrypt the mask ciphertext to obtain the segmented medical image, and send the segmented medical image to the doctor workstation.

9. The multi-center medical image segmentation system based on deep learning according to claim 8, characterized in that: The image archiving communication subsystem is further configured to send an encryption request to the key management server when encrypting the medical image to be segmented, and to send a decryption request to the key management server when decrypting the mask ciphertext; The key management server is further configured to perform identity authentication according to the encryption request or the decryption request, and after the identity authentication is passed, send the image encryption key or the mask decryption key to the image archiving communication subsystem.

Images