A data privacy protection method and device for magnetic resonance image reconstruction

Abstract

The present invention provides a data privacy protection method for magnetic resonance image reconstruction, including: obtaining the original k-space data of a magnetic resonance imaging image; performing random masking on the original k-space data of the magnetic resonance imaging image to obtain masked k-space data; performing downsampling on the masked k-space data to obtain downsampled k-space data; performing differential privacy secure release on the downsampled k-space data to obtain a set of initial reconstruction data; repeating the above steps until a termination condition is satisfied to obtain multiple sets of initial reconstruction data, and the data provider transmits the multiple sets of initial reconstruction data to the data processing end for processing to obtain the final reconstruction data. The present invention also provides a data privacy protection device for magnetic resonance image reconstruction. The method of the present invention solves the problem of privacy protection in the joint training of magnetic resonance imaging image data, makes up for the problem of insufficient data computing power at the data provider end, and improves the robustness of the algorithm and the accuracy of the reconstructed image.

Description

Technical Field

[0001] The present invention relates to the field of computer networks, and in particular, to a method and device for data privacy protection in magnetic resonance image reconstruction. Background Art

[0002] In recent years, both data users and data providers have increasingly emphasized the security and privacy protection of personal data. However, in some scenarios of large-scale training data and large-scale model training, the terminal cannot pre-store relevant images in advance, or the computing power of the terminal is insufficient to complete image processing. Image data needs to be transmitted between different terminals, and privacy protection during the transmission of image data is a particularly severe challenge and problem.

[0003] Medical big data has the characteristics of high volume, high speed, and multiple types. Among them, magnetic resonance imaging is a widely used medical technology. In order to avoid misdiagnosis caused by reasons such as uneven brightness in the reconstructed images of magnetic resonance imaging, the image reconstruction model of magnetic resonance imaging needs to be trained using a large amount of magnetic resonance image data. In real scenarios, the data processing end does not distinguish between general features and individual features when processing images. Therefore, the model released by the data processing end may inadvertently disclose the individual features in the training set, resulting in the leakage of patients' privacy information. At the same time, magnetic resonance images are distributed in different institutions. Due to data security and privacy issues, the transmission and use of magnetic resonance image data are restricted by laws and regulations and cannot be directly transmitted to the data processing end as the large amount of training data required by the deep learning model.

[0004] Chinese patent document with publication number CN104040934B discloses a method for protecting image privacy, including: receiving an original image; dividing the original image into a first sub-image and a second sub-image, each of the first sub-image and the second sub-image having the same number of pixels as the original image; for each of the multiple pixel values of the first sub-image, adding a first corresponding key stream value selected from multiple key stream values; generating a first encrypted sub-image; for each of the multiple pixel values of the second sub-image, adding a second corresponding key stream value selected from the multiple key stream values, the second corresponding key stream value being different from the first corresponding key stream value; generating a second encrypted sub-image; and storing the first encrypted sub-image and the second encrypted sub-image as data for transmission to a publicly accessible storage facility. This method randomly encrypts image information and cannot guarantee both the privacy protection of image data and the accuracy of image reconstruction.

[0005] A Chinese patent document with the publication number CN113889232A designed a privacy protection method based on medical images. First, information extraction is performed on the personal privacy area of the original image based on contour detection. Then, with the help of the complex sequence generated by the iteration of the Logistic chaotic system, pixel value transformation is performed on the extracted area. Finally, the transformation results of all text areas are embedded into the original image, and the image is scrambled after overall encryption, so as to protect the personal privacy area in this encryption form, and further protect the privacy of patients. This method protects the target area, and the image is prone to distortion when used for training.

[0006] A Chinese patent document with the publication number CN113536382A discloses a privacy protection method for medical data sharing based on blockchain using federated learning. Without the original data leaving the local area, joint modeling of the data is performed. In addition, differential privacy noise is added during the local model training process to prevent attackers from inferring users' sensitive information using the learned model. This method is not applicable to systems with strong computing power for training data, and the usability of the learning method is insufficient.

[0007] In addition, compared with existing medical imaging technologies such as CT, magnetic resonance imaging technology has the characteristics of strong soft tissue contrast, low radiation dose, and wide examination range. However, its problems are high data processing complexity, the need for noise removal and image reconstruction, which pose new challenges to medical image reconstruction. At the same time, medical images contain a large amount of users' privacy information, so it is necessary and feasible to perform privacy protection for medical image reconstruction.

[0008] Therefore, it is necessary to solve the problem of proposing a privacy protection preprocessing method for the image data of magnetic resonance imaging, while ensuring the privacy security of patients' individuals, improving the accuracy of server-side reconstruction and diagnosis tasks. Summary of the Invention

[0009] The purpose of the present invention is to provide a data privacy protection method and device for magnetic resonance image reconstruction, solve the problem of privacy protection in the joint training of magnetic resonance imaging image data, and further provide a reference for medical diagnosis.

[0010] A data privacy protection method for magnetic resonance image reconstruction includes the following steps:

[0011] Step 1, the data provider obtains the original grayscale medical image, and performs Fourier transform on the original grayscale medical image to obtain the original k-space data of the magnetic resonance imaging image;

[0012] Step 2, perform random masking on the original k-space data of the magnetic resonance imaging image to obtain masked k-space data;

[0013] Step 3, downsample the masked k-space data to obtain downsampled k-space data;

[0014] Step 4, perform differential privacy secure release on the downsampled k-space data to obtain a set of initial reconstruction data;

[0015] Step 5, repeat Step 2 to Step 4 until the termination condition is met, obtaining multiple sets of initial reconstruction data. The data provider transmits the multiple sets of initial reconstruction data to the data processing end for processing to obtain the final reconstruction data.

[0016] Furthermore, in Step 1, the method of performing Fourier transform on the original grayscale medical image to obtain the original k-space data of the magnetic resonance imaging image is as follows: Apply 2D Fourier transform to the signal data obtained by medical instrument scanning, and perform numerical integration to obtain the original k-space data in the frequency space.

[0017] Furthermore, in Step 2, the method of randomly masking the original k-space data of the magnetic resonance imaging image to obtain the masked k-space data is as follows: Randomly generate a binary mask matrix with the same size as the original k-space data and each element randomly taking a value of 0 or 1, and multiply the corresponding elements of the original k-space data and the binary mask matrix to obtain the masked k-space data.

[0018] Furthermore, in Step 3, the method of downsampling the masked k-space data to obtain the downsampled k-space data is as follows: Divide the masked k-space data into several sub-space data equally, randomly select the frequency coordinates of one of the sub-space data, and set the masked k-space data corresponding to the remaining frequency coordinates to zero to obtain the downsampled k-space data.

[0019] Furthermore, in Step 4, the method of performing differential privacy secure release on the downsampled k-space data to obtain a set of initial reconstruction data is as follows: First, standardize the downsampled k-space data, and then add isotropic Gaussian noise to the standardized k-space data to obtain a set of initial reconstruction data.

[0020] Furthermore, the formula used for standardizing the downsampled k-space data is:

[0021]

[0022] where F s (u, v) is the downsampled k-space data, is the downsampled k-space data after standardization.

[0023] Furthermore, in Step 5, the termination condition is: When the iteration number hyperparameter or the degree of decrease of the iteration error hyperparameter meets the preset value, the repetition stops.

[0024] Specifically, the iteration number hyperparameter is preset as T, where T can be any positive integer ≥ 5, or the iteration error hyperparameter is preset as e, where e can be 10^{-k} and k is an integer.

[0025] Further, in step 5, the specific steps for the data provider to transfer multiple groups of initial reconstruction data to the data processor for processing to obtain the final reconstruction data are as follows:

[0026] Step 5.1, the data provider transfers multiple groups of initial reconstruction data to the data processor, and the data processor performs image reconstruction on the multiple groups of initial reconstruction data to obtain multiple groups of intermediate reconstruction data;

[0027] Step 5.2, average the multiple groups of intermediate reconstruction data to obtain the final reconstruction data; the data processor returns the final reconstruction data to the data provider.

[0028] Further, in step 5.1, image reconstruction is performed by the compressive sensing method, and the calculation formula is:

[0029]

[0030] where is the intermediate reconstruction data, g is an element in the physical space of the image to be reconstructed, F s (u, v) is the downsampled k-space data, F is the Fourier transform, and W is a linear transform that preserves data sparsity in the physical domain.

[0031] The present invention performs image reconstruction on the initial reconstruction data provided by the data provider through the compressive sensing method, improving the usability of differentially private released images in downstream tasks.

[0032] The present invention also provides a data privacy protection device for magnetic resonance image reconstruction, including a memory and one or more processors at the data provider end. The memory stores executable code, and the one or more processors execute the executable code to implement the data privacy protection method for brain magnetic resonance image reconstruction as described above.

[0033] The present invention also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, it implements the data privacy protection method for brain magnetic resonance image reconstruction as described above.

[0034] Compared with the prior art, the present invention has at least the following beneficial effects:

[0035] The image privacy protection system for magnetic resonance imaging provided by the present invention ensures the privacy of the output data of the data provider through masking and downsampling; through the joint calculation of the data provider and the data processing end, it makes up for the problem of insufficient data computing power of the data provider and improves the usability of the learning method; through the standardization of the initial reconstructed data, it gives a consistent data preprocessing method for the magnetic resonance data image reconstruction problem, realizing the interconnection and interoperability between different data providers and the data processing end; by solving the intermediate reconstructed data multiple times and taking the average value, it improves the robustness of the algorithm and the accuracy of the reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic flowchart of the data privacy protection method for magnetic resonance image reconstruction in this embodiment.

[0037] Figure 2 It is a schematic flowchart of the data processing process of the data processing end in this embodiment.

[0038] Figure 3 It is a schematic diagram of the information interaction between the data provider and the data processing end in this embodiment.

[0039] Figure 4 It is a comparison diagram of the initial medical image and the medical image obtained after processing in this embodiment, where Figure 4 a is the initial medical image; Figure 4 b is the original k-space image obtained after Fourier transform; Figure 4 c is the downsampled k-space image; Figure 4 d is the initial reconstructed image obtained by differential privacy secure release.

[0040] Figure 5 For Figure 4 The initial reconstructed image obtained by directly performing differential privacy secure release on the original k-space image in b. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0042] Figure 1 It is a schematic flowchart of the data privacy protection method for magnetic resonance image reconstruction. As Figure 1 shown, it is the scenario where the data provider preprocesses the image of magnetic resonance imaging. The preprocessing of the image of magnetic resonance imaging includes the following steps:

[0043] Step 1, Acquisition of original k-space data: The data provider acquires the original grayscale medical image, and performs Fourier transform on the original grayscale medical image to obtain the original k-space data of the magnetic resonance imaging (MRI) image.

[0044] Fourier transform is performed on the original grayscale medical image to obtain the original k-space data. Specifically, 2D Fourier transform is applied to the signal data obtained by medical instrument scanning, and numerical integration is performed to obtain the original k-space data of the MRI image in the frequency space. The calculation formula is as follows:

[0045] F(u,v)=∫∫f(x,y)e -i2π(ux+vy) dxdy

[0046] where f(x,y) is the original medical image data, and F(u,v) is the original k-space data after Fourier transform. x,y are the position coordinates in the real domain space, and u,v are the dual coordinates in the frequency domain space.

[0047] Step 2, Numerical processing of the original k-space data: Random masking is performed on the original k-space data of the MRI image to obtain the masked k-space data.

[0048] Random masking is performed on the original k-space data of the MRI image. For the original k-space data of the MRI image obtained in Step 1, elements are randomly selected for masking. Specifically, a binary masking matrix M(a,b) of the same size as the original k-space data is randomly generated. Each element in the binary masking matrix M(a,b) randomly takes a value of 0 or 1. The original k-space data F(u,v) obtained in Step 1 is multiplied by the binary masking matrix M(a,b) element by element to obtain the masked k-space data. The calculation formula is as follows:

[0049] F m (u,v)=M(a,b)F(u,v)

[0050] where F m (u,v) is the masked k-space data.

[0051] Step 3, Frequency processing of the masked k-space data: Downsampling is performed on the masked k-space data to obtain the downsampled k-space data.

[0052] Downsample the masked k-space data to reduce the number of pixels required for image reconstruction. Specifically, the spatial frequency after the Fourier transform of the image corresponds to the spatial frequency information of the image features. The masked k-space data is equally divided into several sub-space data. Randomly select the frequency coordinates of one of the sub-space data, and set the remaining sub-space data corresponding to the frequency coordinates to zero to obtain the downsampled k-space data. The calculation formula is as follows:

[0053]

[0054] Among them, F s (u, v) is the downsampled k-space data, and S is the frequency coordinate range of the sub-space data randomly selected for downsampling.

[0055] Step 4, perform differential privacy secure release on the downsampled k-space data to obtain a set of initial reconstruction data.

[0056] If the data provider directly transmits accurate data information to the data processing end, the data processing end may reverse-infer the private information through the data information, resulting in the leakage of the patient's private information. Differential privacy distorts sensitive data by adding noise, and can remove individual characteristics while retaining general characteristics to protect patient privacy.

[0057] Specifically, standardize the downsampled k-space data obtained in step 3. Standardization can remove the data measurement differences between different institutions and make the data of different institutions at the same quantity level. The formula used for standardization is as follows:

[0058]

[0059] Among them, is the downsampled k-space data after standardization.

[0060] Add isotropic Gaussian noise to the standardized k-space data to obtain a set of initial reconstruction data. The calculation formula is as follows:

[0061]

[0062] Among them, is the initial reconstruction data, n is the isotropic Gaussian noise, and σ is determined by the degree of privacy protection. The larger σ is, the stronger the privacy protection degree, and the smaller σ is, the weaker the privacy protection intensity.

[0063] In this embodiment, the masking and downsampling methods are combined with the differential privacy method. The masking itself can enhance the image recognition, and at the same time, combined with differential privacy, it can enhance the accuracy of image recognition while protecting user privacy. Downsampling itself is to reduce the computational amount of data, but usually it will lose precision. However, combined with differential privacy, it can discard some noise, thereby improving the accuracy of image recognition.

[0064] Step 5. Repeat steps 2 to 4 until the termination condition is met, obtaining multiple groups of initial reconstruction data. The data provider transmits the multiple groups of initial reconstruction data to the data processing end for processing to obtain the final reconstruction data.

[0065] Specifically, the termination condition of this embodiment is: when the degree of decrease of the iteration number hyperparameter or the iteration error hyperparameter meets the preset value, the repetition step terminates.

[0066] Figure 2 FIG. is a schematic flow chart of the process in which the data provider transmits multiple groups of initial reconstruction data to the data processing end for processing to obtain the final reconstruction data, including:

[0067] Step 5.1. The data provider transmits multiple groups of initial reconstruction data to the data processing end, and the data processing end performs image reconstruction on the multiple groups of initial reconstruction data to obtain multiple groups of intermediate reconstruction data;

[0068] The data processing end performs row image reconstruction according to the initial reconstruction data provided by the data provider, and reconstructs the original image through a numerical method. The original image can be obtained by using an iterative method such as compressed sensing and parallel computing on downsampled k-space data.

[0069] The compressed sensing method is used to acquire and reconstruct sparse signals, and can restore the overall signal from fewer measurement values. In this embodiment, through the compressed sensing method, specifically, the conjugate gradient iteration method is used to optimize the following compressed sensing problem with constraints:

[0070]

[0071] where is the intermediate reconstruction data, g is an element in the physical space of the image to be reconstructed, F s (u, v) is the downsampled k-space data, F is the Fourier transform, and W is a linear transform that maintains data sparsity in the physical domain.

[0072] Step 5.2. Average the multiple groups of intermediate reconstruction data to obtain the final reconstruction data; the data processing end returns the final reconstruction data to the data provider.

[0073] The specific calculation formula for averaging the intermediate reconstruction data obtained in step 5.1 to obtain the final reconstruction data is:

[0074]

[0075] Among them, is the final reconstructed data, is the intermediate reconstructed data, N is the number of groups of intermediate reconstructed data, and after the reconstruction is completed, the final reconstructed data is returned to the data provider.

[0076] Figure 3 is a schematic diagram of the information interaction between the data provider and the data processor in this embodiment. N data providers preprocess the privacy-protected magnetic resonance images, and provide the processed data to the data processor. The data processor reconstructs the preprocessed data, and uses the iterative method to solve the final reconstructed data. The preprocessing includes steps 1 to 4 and repeating steps 2 to 4 in step 5 until the termination condition is met, obtaining multiple groups of initial reconstructed data. After preprocessing, multiple groups of initial reconstructed data are transmitted to the data processor for data reconstruction, so as to obtain the final reconstructed data.

[0077] Figure 4 is a comparison diagram of the initial medical image and the medical image obtained after processing in this embodiment, where Figure 4 a is the initial medical image, obtained by visualizing the slice with a frequency space of 3, and is the sliced image of the brain's mri image in the frequency space. Figure 4 b is the original k-space image obtained after Fourier transform, corresponding to the 0th, 5th, and 10th images in the 3D scan respectively. Figure 4 c is the downsampled k-space image, which is visually Figure 4 blurrier than Figure 4 b. Figure 4 d is the initial reconstructed image obtained by differentially private secure release, which is visually

[0078] Figure 5 is Figure 4 the reconstructed medical image directly reconstructed from the original k-space image in Figure 4 b. It can be seen from the comparison between Figure 5 and

[0079] that the reconstructed medical image obtained by the method of this embodiment is relatively blurred, while the reconstructed medical image directly reconstructed from the original k-space image is relatively clear. This shows that the method of this embodiment can well protect the privacy image of medical data.

[0080] This embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the data privacy protection method for brain magnetic resonance imaging reconstruction described above.

[0081] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data privacy protection method for magnetic resonance image reconstruction, characterized in that, It includes the following steps: Step 1, the data provider obtains the original grayscale medical image, performs Fourier transform on the original grayscale medical image to obtain the original k-space data of the magnetic resonance imaging (MRI) image; Step 2, perform random masking on the original k-space data of the MRI image to obtain masked k-space data; Step 3, downsample the masked k-space data to obtain downsampled k-space data; The method of downsampling the masked k-space data to obtain downsampled k-space data is: divide the masked k-space data into several sub-space data equally, randomly select the frequency coordinates of one of the sub-space data, and set the masked k-space data corresponding to the remaining frequency coordinates to zero to obtain downsampled k-space data; Step 4, perform differential privacy secure release on the downsampled k-space data to obtain a set of initial reconstructed data; The method of performing differential privacy secure release on the downsampled k-space data to obtain a set of initial reconstructed data is: first standardize the downsampled k-space data, and then add isotropic Gaussian noise to the standardized k-space data to obtain a set of initial reconstructed data; Step 5, repeat Step 2 to Step 4 until the termination condition is met to obtain multiple sets of initial reconstructed data. The data provider transfers the multiple sets of initial reconstructed data to the data processing end for processing to obtain the final reconstructed data; The specific steps for the data provider to transfer multiple sets of initial reconstructed data to the data processing end for processing to obtain the final reconstructed data are: Step 5.1, the data provider transfers multiple sets of initial reconstructed data to the data processing end, and the data processing end performs image reconstruction on the multiple sets of initial reconstructed data to obtain multiple sets of intermediate reconstructed data; Step 5.2, average the multiple sets of intermediate reconstructed data to obtain the final reconstructed data. The data processing end returns the final reconstructed data to the data provider.

2. The data privacy protection method for magnetic resonance image reconstruction according to claim 1, characterized in that In Step 1, the method of performing Fourier transform on the original grayscale medical image to obtain the original k-space data of the MRI image is: apply 2D Fourier transform to the signal data obtained by medical instrument scanning and perform numerical integration to obtain the original k-space data in the frequency space.

3. The data privacy protection method for magnetic resonance image reconstruction according to claim 1, characterized in that In Step 2, the method of performing random masking on the original k-space data of the MRI image to obtain masked k-space data is: randomly generate a binary mask matrix with the same size as the original k-space data and each element randomly taking values of 0 or 1, and multiply the corresponding elements of the original k-space data and the binary mask matrix to obtain masked k-space data.

4. The data privacy protection method for magnetic resonance image reconstruction according to claim 1, wherein The formula for standardizing the downsampled k-space data is: Among them, F s (u, v) is the downsampled k-space data, and is the downsampled k-space data after normalization.

5. The data privacy protection method for magnetic resonance image reconstruction according to claim 1, wherein In Step 5, the termination condition is: when the iteration number hyperparameter or the degree of decrease of the iteration error hyperparameter meets the preset value, the repetition stops.

6. A data privacy protection device for magnetic resonance image reconstruction, comprising a memory and one or more processors at a data providing end, wherein executable code is stored in the memory, and is characterized in that, The one or more processors execute the executable code to implement the data privacy protection method for magnetic resonance image reconstruction according to any one of claims 1-5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the data privacy protection method for magnetic resonance image reconstruction according to any one of claims 1-5.

Images