Data privacy protection method and apparatus for magnetic resonance imaging reconstruction

Abstract

Provided in the present invention is a data privacy protection method for magnetic resonance imaging reconstruction. The method comprises: acquiring original k-space data of a magnetic-resonance-imaging image; performing random masking on the original k-space data of the magnetic-resonance-imaging image, so as to obtain masked k-space data; performing down-sampling on the masked k-space data, so as to obtain down-sampled k-space data; performing differential-privacy-based secure publishing on the down-sampled k-space data, so as to obtain a group of initial reconstructed data; and repeating the above steps until a termination condition is met, so as to obtain a plurality of groups of initial reconstructed data, and a data providing end transmitting the plurality of groups of initial reconstructed data to a data processing end for processing, such that final reconstructed data is obtained. Further provided in the present invention is a data privacy protection apparatus for magnetic resonance imaging reconstruction. The method in the present invention solves the problem of privacy protection in joint data training of a magnetic-resonance-imaging image, remedies the insufficiency of data computing power provided by a data providing end, and improves the robustness of an algorithm and the accuracy of a reconstructed image.

Description

Data privacy protection method and device for magnetic resonance image reconstruction Technical Field

[0001] The present invention relates to the field of computer networks, and in particular to a method and device for protecting data privacy in magnetic resonance image reconstruction. Background Art

[0002] In recent years, both data users and data providers have increasingly attached importance to the security and privacy protection of personal data. However, in some scenarios of large-scale training data and large-scale model training, the terminal cannot pre-store relevant images, or the terminal computing power is insufficient to complete image processing. Image data needs to be transmitted between different terminals. Privacy protection during the transmission of image data is a particularly severe challenge and problem.

[0003] Medical big data is characterized by high capacity, high speed, and diverse types. Magnetic resonance imaging (MRI) is a widely used medical technology. To prevent misdiagnoses caused by uneven brightness and other factors in MRI reconstructed images, MRI image reconstruction models require training using large amounts of MRI image data. In real-world scenarios, data processing endpoints do not distinguish between common and individual features when processing images. Therefore, models released by these endpoints may inadvertently reveal individual features in the training set, leading to the leakage of patient privacy information. Furthermore, MRI images are distributed across different institutions. Due to data security and privacy concerns, the transmission and use of MRI image data is subject to legal and regulatory restrictions, making it impossible to directly transmit them to the data processing endpoints as the large amounts of training data required for deep learning models.

[0004] Chinese patent publication CN104040934B discloses a method for protecting image privacy, comprising: receiving an original image; segmenting the original image into a first sub-image and a second sub-image, each of the first sub-image and the second sub-image having the same number of pixels as the original image; adding a first corresponding key stream value selected from a plurality of key stream values ​​to each of a plurality of pixel values ​​of the first sub-image; generating a first encrypted sub-image; adding a second corresponding key stream value selected from the plurality of key stream values ​​to each of a plurality of pixel values ​​of the second sub-image, the second corresponding key stream value being different from the first corresponding key stream value; generating a second encrypted sub-image; and storing the first encrypted sub-image and the second encrypted sub-image as data for transmission to a publicly accessible storage facility. This method randomly encrypts image information and cannot simultaneously guarantee image data privacy protection and image reconstruction accuracy.

[0005] Chinese patent publication CN113889232A proposes a privacy protection method for medical imaging. First, it extracts information from the original image's privacy regions using contour detection. Then, using a complex sequence generated by iterative logistic chaotic systems, it transforms the pixel values ​​of the extracted regions. Finally, the transformed results for all text regions are embedded into the original image. The image is then encrypted and scrambled. This encryption method protects privacy regions and, by extension, the patient's privacy. This method protects the target region, but images are susceptible to distortion during training.

[0006] Chinese patent publication CN113536382A discloses a blockchain-based method for sharing and protecting medical data privacy using federated learning. This method jointly models the data without leaving the local machine. Furthermore, differential privacy noise is added during local model training to prevent attackers from exploiting the learned model to infer sensitive user information. This method is not suitable for data training on systems with high computing power, and the learning method is limited in its usability.

[0007] In addition, compared with existing medical imaging technologies such as CT, magnetic resonance imaging technology has the characteristics of strong soft tissue contrast, low radiation dose and wide examination range. However, its problem is that the data processing is highly complex and requires noise removal and image reconstruction, which poses new challenges for medical image reconstruction. At the same time, medical images contain a large amount of user privacy information, so privacy-protected medical image reconstruction is necessary and feasible.

[0008] Therefore, a privacy protection preprocessing method for magnetic resonance imaging image data is proposed to ensure the privacy security of individual patients while improving the accuracy of server-side reconstruction and diagnosis tasks.

[0009] Summary of the Invention

[0010] The purpose of the present invention is to provide a data privacy protection method and device for magnetic resonance image reconstruction, solve the problem of privacy protection in joint training of magnetic resonance imaging image data, and further provide a reference for medical diagnosis.

[0011] A data privacy protection method for magnetic resonance image reconstruction includes the following steps:

[0012] Step 1: The data provider obtains an original grayscale medical image, performs Fourier transform on the original grayscale medical image, and obtains original k-space data of the magnetic resonance imaging image;

[0013] Step 2, randomly masking the original k-space data of the magnetic resonance imaging image to obtain masked k-space data;

[0014] Step 3, downsampling the masked k-space data to obtain downsampled k-space data;

[0015] Step 4: Perform differential privacy security publishing on the downsampled k-space data to obtain a set of initial reconstructed data;

[0016] Step 5: Repeat steps 2 to 4 until the termination condition is met to obtain multiple sets of initial reconstructed data. The data provider transmits the multiple sets of initial reconstructed data to the data processing end for processing to obtain the final reconstructed data.

[0017] Furthermore, in step 1, the method of performing Fourier transform on the original grayscale medical image to obtain the original k-space data of the magnetic resonance imaging image is: applying 2D Fourier transform to the signal data obtained by scanning the medical instrument, performing numerical integration, and obtaining the original k-space data in the frequency space.

[0018] Furthermore, in step 2, the original k-space data of the magnetic resonance imaging image is randomly masked to obtain the masked k-space data by randomly generating a binary mask matrix of the same size as the original k-space data and in which each element is randomly 0 or 1, and multiplying the original k-space data by the corresponding elements in the binary mask matrix to obtain the masked k-space data.

[0019] Furthermore, in step 3, the method for downsampling the masked k-space data to obtain the downsampled k-space data is as follows: the masked k-space data is equally divided into a number of subspace data, the frequency coordinate of one of the subspace data is randomly selected, and the masked k-space data corresponding to the remaining frequency coordinates are reset to zero to obtain the downsampled k-space data.

[0020] Furthermore, in step 4, the method for differentially privately publishing the downsampled k-space data to obtain a set of initial reconstructed data is: first, the downsampled k-space data is normalized, and then isotropic Gaussian noise is added to the normalized k-space data to obtain a set of initial reconstructed data.

[0021] Furthermore, the formula used to normalize the downsampled k-space data is:

[0022] Among them, F s (u,v) is the downsampled k-space data, is the normalized downsampled k-space data.

[0023] Furthermore, in step 5, the termination condition is: when the degree of decrease of the iteration number hyperparameter or the iteration error hyperparameter meets the preset value, the repetition step is terminated.

[0024] Specifically, the number of iterations hyperparameter is preset to T, where T can be any positive integer ≥ 5, or the iteration error hyperparameter is preset to e, where e can be 10^{-k}, where k is a certain integer.

[0025] Furthermore, in step 5, the data provider transmits multiple sets of initial reconstructed data to the data processing end for processing. The specific steps for obtaining the final reconstructed data are as follows:

[0026] Step 5.1: The data provider transmits multiple sets of initial reconstruction data to the data processing end, and the data processing end reconstructs the images of the multiple sets of initial reconstruction data to obtain multiple sets of intermediate reconstruction data;

[0027] In step 5.2, multiple sets of intermediate reconstructed data are averaged to obtain final reconstructed data; the data processing end returns the final reconstructed data to the data providing end.

[0028] Furthermore, in step 5.1, the image is reconstructed using the compressed sensing method, and the calculation formula is:

[0029] in, is the intermediate reconstruction data, g is an element of the physical space of the image to be reconstructed, F s (u,v) is the downsampled k-space data, F is the Fourier transform, and W is the linear transformation in the physical domain to maintain data sparsity.

[0030] The present invention reconstructs row images of the initial reconstructed data provided by the data provider through a compressed sensing method, thereby improving the availability of differentially private published images in downstream tasks.

[0031] The present invention also provides a data privacy protection device for magnetic resonance image reconstruction, comprising a memory at a data provider end and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the data privacy protection method for brain magnetic resonance image reconstruction described above.

[0032] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned data privacy protection method for brain magnetic resonance imaging reconstruction.

[0033] Compared with the prior art, the present invention has at least the following beneficial effects:

[0034] The magnetic resonance imaging image privacy protection system provided by the present invention ensures the privacy of data output by the data provider through masking and downsampling; through joint calculation between the data provider and the data processing end, it makes up for the problem of insufficient data computing power on the data provider end and improves the usability of the learning method; through the standardization of initial reconstructed data, a consistent data preprocessing method for the magnetic resonance data image reconstruction problem is provided, and different data providers can be interconnected with the data processing end; by solving the intermediate reconstructed data multiple times and taking the average value, the robustness of the algorithm and the accuracy of the reconstructed image are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG1 is a flow chart of a data privacy protection method for magnetic resonance image reconstruction according to an embodiment of the present invention.

[0036] FIG2 is a schematic diagram of the data processing flow at the data processing end of this embodiment.

[0037] FIG3 is a schematic diagram of information interaction between a data providing end and a data processing end in this embodiment.

[0038] Figure 4 is a comparison diagram of the initial medical image and the processed medical image of this embodiment, where Figure 4a is the initial medical image; Figure 4b is the original k-space image obtained after Fourier transform; Figure 4c is the downsampled k-space image; and Figure 4d is the initial reconstructed image obtained by differential privacy security release.

[0039] Figure 5 shows the initial reconstructed image obtained by directly performing differential privacy security publishing on the original k-space image in Figure 4b. DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0041] FIG1 is a flow chart of a data privacy protection method for magnetic resonance image reconstruction. As shown in FIG1 , a scenario in which a data provider preprocesses magnetic resonance imaging images includes the following steps:

[0042] Step 1: Acquisition of original k-space data: The data provider obtains the original grayscale medical image, performs Fourier transform on the original grayscale medical image, and obtains the original k-space data of the magnetic resonance imaging image.

[0043] Perform a Fourier transform on the original grayscale medical image to obtain the original k-space data. Specifically, a 2D Fourier transform is applied to the signal data obtained by scanning with a medical instrument, and numerical integration is performed to obtain the original k-space data of the magnetic resonance imaging image in the frequency space. The calculation formula used is as follows: F(u,v)=∫∫f(x,y)e-i2π(ux+vy)dxdy

[0044] Where f(x,y) is the original medical image data, and F(u,v) is the original k-space data after Fourier transformation. x and y are the position coordinates in the real domain, and u and v are the dual coordinates in the frequency domain.

[0045] Step 2: Numerical processing of raw k-space data: performing random masking on the raw k-space data of the magnetic resonance imaging image to obtain masked k-space data.

[0046] Randomly mask the original k-space data of the magnetic resonance imaging image. For the original k-space data of the magnetic resonance imaging image obtained in step 1, randomly select elements therein for masking. Specifically, randomly generate a binary mask matrix M(a,b) of the same size as the original k-space data. Each element in the binary mask matrix M(a,b) is randomly taken as 0 or 1. The original k-space data F(u,v) obtained in step 1 is multiplied by the binary mask matrix M(a,b) to obtain the masked k-space data. The calculation formula is as follows: F m (u,v)=M(a,b)F(u,v)

[0047] Among them, F m (u,v) is the mask k-space data.

[0048] Step 3, frequency processing of masked k-space data: downsampling the masked k-space data to obtain downsampled k-space data.

[0049] The masked k-space data is downsampled to reduce the number of pixels required for image reconstruction. Specifically, the spatial frequency of the image after Fourier transform corresponds to the spatial frequency information of the image features. The masked k-space data is divided into several subspace data. The frequency coordinate of one of the subspace data is randomly selected, and the subspace data corresponding to the other frequency coordinates are reset to zero to obtain the downsampled k-space data. The calculation formula is:

[0050] Among them, F s (u, v) is the downsampled k-space data, and S is the frequency coordinate range of the randomly selected subspace data for downsampling.

[0051] Step 4: Perform differential privacy security on the downsampled k-space data to obtain a set of initial reconstructed data.

[0052] If the data provider directly transmits accurate data to the data processor, the data processor may infer private information from the data, resulting in the leakage of patients' private information. Differential privacy distorts sensitive data by adding noise to the data. This can remove individual features while retaining common features to protect patient privacy.

[0053] Specifically, the downsampled k-space data obtained in step 3 are normalized. Normalization can remove the measurement differences of data from different institutions and make the data from different institutions at the same quantitative level. The formula used for normalization is as follows:

[0054] in, is the normalized downsampled k-space data.

[0055] Add isotropic Gaussian noise to the normalized k-space data to obtain a set of initial reconstructed data. The calculation formula is as follows:

[0056] in, is the initial reconstructed data, n is isotropic Gaussian noise, and σ is determined by the degree of privacy protection. The larger σ is, the stronger the privacy protection is, and the smaller σ is, the weaker the privacy protection is.

[0057] This embodiment combines masking and downsampling with differential privacy. Masking alone can enhance image recognition, while combined with differential privacy, it can enhance image recognition accuracy while protecting user privacy. Downsampling itself reduces the amount of computation required, but typically results in a loss of accuracy. However, combined with differential privacy, it can remove some noise, thereby improving image recognition accuracy.

[0058] Step 5: Repeat steps 2 to 4 until the termination condition is met to obtain multiple sets of initial reconstructed data. The data provider transmits the multiple sets of initial reconstructed data to the data processing end for processing to obtain the final reconstructed data.

[0059] Specifically, the termination condition of this embodiment is: when the degree of decrease of the iteration number hyperparameter or the iteration error hyperparameter meets the preset value, the repetition step is terminated.

[0060] FIG2 is a flow chart showing a process in which a data provider transmits multiple sets of initial reconstructed data to a data processor for processing to obtain final reconstructed data, including:

[0061] Step 5.1: The data provider transmits multiple sets of initial reconstruction data to the data processing end, and the data processing end reconstructs the images of the multiple sets of initial reconstruction data to obtain multiple sets of intermediate reconstruction data;

[0062] The data processing end reconstructs the row image based on the initial reconstruction data provided by the data provider, and reconstructs the original image through numerical methods. The original image can be obtained by downsampling the k-space data and performing iterative methods such as compressed sensing and parallel computing.

[0063] Compressed sensing is used to acquire and reconstruct sparse signals and can restore the overall signal from fewer measurement values. This embodiment uses the compressed sensing method, specifically the conjugate gradient iteration method, to optimize the following constrained compressed sensing problem:

[0064] in, is the intermediate reconstruction data, g is an element of the physical space of the image to be reconstructed, F s (u,v) is the downsampled k-space data, F is the Fourier transform, and W is the linear transformation in the physical domain to maintain data sparsity.

[0065] In step 5.2, multiple sets of intermediate reconstructed data are averaged to obtain final reconstructed data; the data processing end returns the final reconstructed data to the data providing end.

[0066] The specific calculation formula for averaging the intermediate reconstructed data obtained in step 5.1 to obtain the final reconstructed data is:

[0067] in, is the final reconstructed data, is the intermediate reconstructed data, N is the number of groups of intermediate reconstructed data, and after the reconstruction is completed, the final reconstructed data is returned to the data provider.

[0068] Figure 3 is a schematic diagram of the information exchange between the data provider and data processing end of this embodiment. N data providers preprocess the privacy-preserving magnetic resonance images and provide the resulting data to the data processing end. The data processing end reconstructs the preprocessed data and iterates to obtain the final reconstructed data. The preprocessing includes steps 1 through 4, and in step 5, steps 2 through 4 are repeated until a termination condition is met, resulting in multiple sets of initial reconstructed data. The resulting sets of initial reconstructed data are then transmitted to the data processing end for data reconstruction, thereby obtaining the final reconstructed data.

[0069] Figure 4 shows a comparison of the initial and processed medical images of this embodiment. Figure 4a shows the initial medical image, obtained by visualizing slices in frequency space 3, representing a sliced ​​image of a brain MRI image in frequency space. Figure 4b shows the original k-space image obtained after Fourier transform, corresponding to images 0, 5, and 10 in the stereo scan. Figure 4c shows the downsampled k-space image, which is visually blurrier than Figure 4b. Figure 4d shows the initial reconstructed image obtained through differential privacy secure publishing, which is even blurrier than Figure 4c.

[0070] Figure 5 shows a reconstructed medical image obtained by directly reconstructing the original k-space image in Figure 4b. Comparing Figures 4 and 5, it can be seen that the reconstructed medical image obtained by the method of this embodiment is relatively blurry, while the reconstructed medical image obtained by directly reconstructing the original k-space image is relatively clear. This demonstrates that the method of this embodiment can effectively protect medical data privacy images.

[0071] This embodiment also provides a data privacy protection device for magnetic resonance image reconstruction, including a memory at a data provider end and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the data privacy protection method for brain magnetic resonance image reconstruction described above.

[0072] This embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the data privacy protection method for brain magnetic resonance imaging reconstruction.

[0073] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data privacy protection method for magnetic resonance image reconstruction, characterized in that: The following steps are involved: Step 1: The data provider obtains an original grayscale medical image, performs Fourier transform on the original grayscale medical image, and obtains original k-space data of the magnetic resonance imaging image; Step 2, randomly masking the original k-space data of the magnetic resonance imaging image to obtain masked k-space data; Step 3, downsampling the masked k-space data to obtain downsampled k-space data; Step 4, performing differential privacy security publishing on the downsampled k-space data to obtain a set of initial reconstructed data; Step 5, repeating steps 2 to 4 until the termination condition is met, obtaining multiple sets of initial reconstructed data, and the data provider transmits the multiple sets of initial reconstructed data to the data processor for processing to obtain final reconstructed data.

2. The data privacy protection method for magnetic resonance image reconstruction according to claim 1, characterized in that: In step 1, the method of performing Fourier transform on the original grayscale medical image to obtain the original k-space data of the magnetic resonance imaging image is: applying 2D Fourier transform to the signal data obtained by scanning the medical instrument, performing numerical integration, and obtaining the original k-space data in the frequency space.

3. The data privacy protection method for magnetic resonance image reconstruction according to claim 1, characterized in that: In step 2, the original k-space data of the magnetic resonance imaging image is randomly masked to obtain the masked k-space data by randomly generating a binary mask matrix of the same size as the original k-space data and in which each element randomly takes a value of 0 or 1, and multiplying the original k-space data with the corresponding elements in the binary mask matrix to obtain the masked k-space data.

4. The data privacy protection method for magnetic resonance image reconstruction according to claim 1, characterized in that: In step 3, the method for downsampling the masked k-space data to obtain the downsampled k-space data is to equally divide the masked k-space data into a plurality of subspace data, randomly select the frequency coordinate of one of the subspace data, and reset the masked k-space data corresponding to the remaining frequency coordinates to zero to obtain the downsampled k-space data.

5. The data privacy protection method for magnetic resonance image reconstruction according to claim 1, characterized in that: In step 4, the method for differentially privately publishing the downsampled k-space data to obtain a set of initial reconstructed data is: first, the downsampled k-space data is standardized, and then isotropic Gaussian noise is added to the standardized k-space data to obtain a set of initial reconstructed data.

6. The data privacy protection method for magnetic resonance image reconstruction according to claim 5, characterized in that: The formula for normalizing the downsampled k-space data is: Among them, F s (u,v) is the downsampled k-space data, is the normalized downsampled k-space data.

7. The data privacy protection method for magnetic resonance image reconstruction according to claim 1, characterized in that: In step 5, the termination condition is: when the degree of decrease of the iteration number hyperparameter or the iteration error hyperparameter meets the preset value, the repetition step is terminated.

8. The data privacy protection method for magnetic resonance image reconstruction according to claim 1, characterized in that: In step 5, the data provider transmits multiple groups of initial reconstructed data to the data processor for processing, and the specific steps for obtaining the final reconstructed data are: Step 5.1, the data provider transmits multiple sets of initial reconstruction data to the data processing end, and the data processing end reconstructs the images of the multiple sets of initial reconstruction data to obtain multiple sets of intermediate reconstruction data; Step 5.2, averaging multiple sets of intermediate reconstructed data to obtain final reconstructed data; the data processing end returns the final reconstructed data to the data providing end.

9. A data privacy protection device for magnetic resonance image reconstruction, comprising a memory at a data provider end and one or more processors, wherein the memory stores executable code, characterized in that: The one or more processors execute the executable code to implement the data privacy protection method for magnetic resonance image reconstruction according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, the data privacy protection method for brain magnetic resonance imaging reconstruction is implemented.

Images