Data amplification method and device, model training method and device, computer equipment and medium

Abstract

The invention relates to a data amplification method and device, a model training method and device, computer equipment and a medium. The method comprises the following steps: acquiring gold standard magnetic resonance data; performing filling processing on an edge region in a target K space corresponding to the gold standard magnetic resonance data to obtain sample magnetic resonance data; training the initial generative adversarial network based on the sample magnetic resonance data and the gold standard magnetic resonance data to obtain a target generative adversarial network; the target generative adversarial network comprises a target generator network; and the target generator network is used for performing amplification processing on the gold standard magnetic resonance data. By adopting the method, high-resolution magnetic resonance data which are difficult to obtain can be amplified, so that a large amount of high-resolution magnetic resonance data can be obtained; moreover, the amplified high-resolution magnetic resonance data is closer to actually acquired high-resolution magnetic resonance data, the precision and accuracy of data amplification processing are guaranteed, and the training accuracy of a deep learning model can be improved.

Description

Technical Field

[0001] The present application relates to the field of image technology, and in particular to a data augmentation method, a model training method, an apparatus, a computer device and a medium. Background Art

[0002] With the development and application of artificial intelligence in the medical field, the application of deep learning technology to magnetic resonance systems has become a development trend. For example, through the deep learning model, the target tissue in the magnetic resonance scan image is segmented to obtain the target tissue segmentation result.

[0003] For deep learning models, a large amount of training data is often required to train a high-precision deep learning model. Among them, the training data needs to include a large amount of high-resolution magnetic resonance data. However, due to the difficulty in obtaining high-resolution magnetic resonance data, it is difficult to obtain a large amount of high-resolution magnetic resonance data through direct scanning, resulting in a small amount of training data for deep learning models, which in turn causes the accuracy of the trained deep learning models to be low.

[0004] Therefore, how to obtain a large amount of high-resolution magnetic resonance data to expand the training data of the deep learning model and thus improve the accuracy of the deep learning model has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] Based on this, it is necessary to provide a data amplification method, model training method, device, computer equipment, computer-readable storage medium and computer program product that can obtain a large amount of training data based on a small amount of collected high-resolution magnetic resonance data and amplify the small amount of high-resolution magnetic resonance data to address the above-mentioned technical problems.

[0006] In a first aspect, the present application provides a data augmentation method. The method comprises:

[0007] Acquire gold standard MRI data;

[0008] Filling the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain sample magnetic resonance data;

[0009] Based on sample magnetic resonance data and gold standard magnetic resonance data, an initial generative adversarial network is trained to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network; the target generator network is used to amplify the gold standard magnetic resonance data.

[0010] In one embodiment, filling processing is performed on the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain the sample magnetic resonance data, including:

[0011] Extracting magnetic resonance data of a central area in the target K-space corresponding to the gold standard magnetic resonance data according to a preset size to obtain central magnetic resonance data;

[0012] Based on the first preset filling value, a filling process is performed on the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain edge magnetic resonance data; the edge area of ​​the target K space is the K space area remaining after the central area is removed from the target K space;

[0013] Based on the central magnetic resonance data and the edge magnetic resonance data, sample magnetic resonance data are obtained.

[0014] In one embodiment, the initial generative adversarial network includes an initial generator network and an initial discriminator network; based on the sample magnetic resonance data and the gold standard magnetic resonance data, the initial generative adversarial network is trained to obtain a target generative adversarial network, including:

[0015] Inputting the sample magnetic resonance data into the initial generator network to obtain amplified sample magnetic resonance data;

[0016] The amplified sample magnetic resonance data and the gold standard magnetic resonance data are input into the initial discriminator network to obtain the discrimination result;

[0017] Based on the discrimination results, the initial generator network and the initial discriminator network are trained to obtain the target generative adversarial network.

[0018] In one embodiment, the sample magnetic resonance data is input into the initial generator network to obtain the amplified sample magnetic resonance data, including:

[0019] Inputting the sample magnetic resonance data into the initial generator network to obtain candidate magnetic resonance data;

[0020] The magnetic resonance data of the central area of ​​the target K space corresponding to the candidate magnetic resonance data is replaced with the central magnetic resonance data to obtain the amplified sample magnetic resonance data.

[0021] In one embodiment, the amplified sample magnetic resonance data and the gold standard magnetic resonance data are both magnetic resonance scan raw data; the initial discriminator network includes a first initial discriminator network and a second initial discriminator network; the amplified sample magnetic resonance data and the gold standard magnetic resonance data are input into the initial discriminator network to obtain a discrimination result, including:

[0022] Inputting the amplified sample magnetic resonance data and the gold standard magnetic resonance data into a first initial discriminator network to obtain a first discrimination result;

[0023] Performing image reconstruction on the amplified sample magnetic resonance data to obtain a sample magnetic resonance image; and performing image reconstruction on the gold standard magnetic resonance data to obtain a gold standard magnetic resonance image;

[0024] Inputting the sample magnetic resonance image and the gold standard magnetic resonance image into a second initial discriminator network to obtain a second discrimination result;

[0025] A discrimination result is obtained according to the first discrimination result and the second discrimination result.

[0026] In one embodiment, the method further comprises:

[0027] Acquiring initial magnetic resonance data; the resolution of the initial magnetic resonance data is less than the resolution of the gold standard magnetic resonance data;

[0028] Based on the second preset filling value, a filling process is performed on the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data, and the filled initial magnetic resonance data is used as the sample magnetic resonance data.

[0029] In one embodiment, the method further comprises:

[0030] Inputting the gold standard magnetic resonance data into the target generator network for amplification processing to obtain amplified gold standard magnetic resonance data;

[0031] The gold standard MRI data and the amplified gold standard MRI data are used as training data for the deep learning network;

[0032] Based on the training data, the deep learning network is trained to obtain a trained deep learning model.

[0033] In one embodiment, the training data serves as sample data for a deep learning network, or serves as gold standard data for a deep learning network.

[0034] In a second aspect, the present application also provides a model training method. The method comprises:

[0035] Acquire gold standard MRI data;

[0036] Acquiring sample magnetic resonance data; the method of acquiring the sample magnetic resonance data includes filling and acquiring the edge area in the target K space corresponding to the gold standard magnetic resonance data, and / or filling and acquiring the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data;

[0037] Based on the sample magnetic resonance data and the gold standard magnetic resonance data, the initial generative adversarial network is trained to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network;

[0038] Inputting the gold standard magnetic resonance data into the target generator network for amplification processing to obtain amplified gold standard magnetic resonance data;

[0039] Based on the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data, the deep learning network is trained to obtain a trained deep learning model.

[0040] In a third aspect, the present application also provides a data amplification device. The device comprises:

[0041] an acquisition module for acquiring gold standard magnetic resonance data;

[0042] A processing module is used to fill in the edge area of ​​the target K space corresponding to the gold standard magnetic resonance data to obtain sample magnetic resonance data; and / or to fill in the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data to obtain sample magnetic resonance data;

[0043] The first training module is used to train the initial generative adversarial network based on sample magnetic resonance data and gold standard magnetic resonance data to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network; the target generator network is used to amplify the gold standard magnetic resonance data.

[0044] In a fourth aspect, the present application also provides a model training device. The device comprises:

[0045] A first acquisition module, used to acquire gold standard magnetic resonance data;

[0046] A second acquisition module is used to acquire sample magnetic resonance data; the acquisition method of the sample magnetic resonance data includes filling and acquiring the edge area in the target K space corresponding to the gold standard magnetic resonance data, and / or filling and acquiring the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data;

[0047] A first training module is used to train an initial generative adversarial network based on sample magnetic resonance data and gold standard magnetic resonance data to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network;

[0048] an amplification module, used for inputting the gold standard magnetic resonance data into the target generator network for amplification processing to obtain amplified gold standard magnetic resonance data;

[0049] The second training module is used to train the deep learning network based on the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data to obtain a trained deep learning model.

[0050] In a fifth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the data augmentation method in the first aspect and / or the model training method in the second aspect when executing the computer program.

[0051] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the data augmentation method in the first aspect and / or the model training method in the second aspect.

[0052] In the seventh aspect, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the data augmentation method in the first aspect and / or the model training method in the second aspect.

[0053] The above-mentioned data augmentation method, model training method, device, computer equipment, storage medium and computer program product, the computer equipment obtains gold standard magnetic resonance data; then, fills the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain sample magnetic resonance data; and based on the sample magnetic resonance data and the gold standard magnetic resonance data, the initial generative adversarial network is trained to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network; the target generator network can be used to augment the gold standard magnetic resonance data. It can be seen that in the embodiment of the present application, for a small amount of high-resolution magnetic resonance data that is actually collected, by performing different filling processes on the edge area in its target K space, a large amount of sample magnetic resonance data with a resolution that is almost consistent with the resolution of the magnetic resonance data that is actually collected can be obtained; then, the small amount of high-resolution magnetic resonance data is used as the gold standard magnetic resonance data, and together with the large amount of sample magnetic resonance data obtained by processing, it is used as the training data of the generative adversarial network to realize the training of the generative adversarial network; due to the large amount of training data, a high-precision target generator network can be obtained. Next, based on the high-precision target generator network, data augmentation processing is performed on a small amount of actually collected high-resolution magnetic resonance data, which not only can obtain a large amount of high-resolution magnetic resonance data, but also the high-resolution magnetic resonance data after data amplification using this method is closer to the actually collected high-resolution magnetic resonance data, and the precision and accuracy of the data augmentation processing are guaranteed; furthermore, when a large amount of high-resolution magnetic resonance data after data augmentation processing in this way is used as training data for a deep learning model, it can provide strong data support for the training of the deep learning model and can improve the training precision and model accuracy of the deep learning model.

[0054] In addition, the method proposed in the embodiment of the present application can not only expand the high-resolution magnetic resonance data required by the deep learning model but difficult to actually collect, obtain a large amount of training data, and improve the efficiency of obtaining training data; but also solve a series of problems faced in the actual process of collecting high-resolution magnetic resonance data, such as the difficulty of collection and possible motion artifacts, thereby improving the accuracy of high-resolution magnetic resonance data and the accuracy of training data; and further, it can improve the training accuracy of the deep learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is an application environment diagram of a data augmentation method in an embodiment;

[0056] Figure 2 Schematic diagram of a data augmentation method in one embodiment;

[0057] Figure 3 is a flow chart of a data augmentation method in another embodiment;

[0058] Figure 4 is a flow chart of a data augmentation method in another embodiment;

[0059] Figure 5 is a flow chart of a data augmentation method in another embodiment;

[0060] Figure 6 is a flow chart of a data augmentation method in another embodiment;

[0061] Figure 7 is a flow chart of a data augmentation method in another embodiment;

[0062] Figure 8 A network structure for generating an adversarial network in one embodiment;

[0063] Fig. 9 A network structure for generating an adversarial network in another embodiment;

[0064] Fig.10 A schematic diagram of a flow chart of a model training method in one embodiment;

[0065] Fig.11 is a structural block diagram of a data amplification device in one embodiment;

[0066] Fig.12 is a structural block diagram of a model training device in one embodiment;

[0067] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] With the development of artificial intelligence, more and more deep learning technologies are used in magnetic resonance imaging systems. For example, based on deep learning models, target tissues in high-resolution magnetic resonance imaging images are segmented to obtain high-quality target tissue segmentation results. For deep learning models, a large amount of training data is often required to train high-precision deep learning models.

[0070] Therefore, in order to train the above-mentioned high-precision deep learning model for segmenting target tissues in high-resolution magnetic resonance scan images, it is necessary to collect a large number of high-resolution magnetic resonance scan images as training sample images, and obtain gold standard label images for marking target tissues for each high-resolution magnetic resonance scan image. Only in this way can training be performed based on the training sample images and the corresponding gold standard label images to obtain a high-precision deep learning model.

[0071] However, high-resolution MRI data usually takes a long time to scan, and because the voxels are very small, they are very sensitive to the slightest movement of the patient. In addition, long scanning times make it very easy for motion artifacts to appear in high-resolution MRI data. Therefore, it is difficult to acquire high-resolution MRI data without motion artifacts. It is difficult to obtain a large amount of high-resolution MRI data through direct scanning, which hinders the application of deep learning on high-resolution MRI data, such as accelerating imaging and reducing noise.

[0072] Therefore, how to obtain a large amount of high-resolution magnetic resonance data to expand the training data of the deep learning model and thus improve the accuracy of the deep learning model has become a technical problem that needs to be solved urgently.

[0073] Based on this, an embodiment of the present application proposes a data augmentation method, which amplifies a small amount of collected high-resolution magnetic resonance data based on a generative adversarial network to construct a training data set for deep learning.

[0074] The data augmentation method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. The computer device 102 can be a terminal or a server. When the computer device 102 is a terminal, it includes but is not limited to various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. When the computer device 102 is a server, it can be implemented with an independent server or a server cluster consisting of multiple servers.

[0075] In one embodiment, Figure 2 As shown, a data augmentation method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:

[0076] Step 202: Acquire gold standard magnetic resonance data.

[0077] The gold standard magnetic resonance data may include high-resolution magnetic resonance data that is actually collected. For example, the gold standard magnetic resonance data may be magnetic resonance data with high field imaging resolution obtained by scanning with a high field strength magnetic resonance device; or may be high-resolution magnetic resonance data obtained by scanning for a long time with a low field strength magnetic resonance device, and the scanning time should meet the scanning time required for high-resolution magnetic resonance data.

[0078] In some other optional implementations, the gold standard magnetic resonance data may also be high-resolution magnetic resonance data generated based on low-resolution magnetic resonance data; wherein the low-resolution magnetic resonance data may include actually acquired magnetic resonance data; for example: high-resolution magnetic resonance data may be generated by inputting the low-resolution magnetic resonance data into a pre-trained data processing model for processing; wherein the data processing model may be obtained after self-supervised training based on a large amount of low-resolution magnetic resonance data, and the data processing model is used to convert the low-resolution magnetic resonance data into high-resolution magnetic resonance data.

[0079] In addition, the gold standard magnetic resonance data may be K-space data or scanned image data, etc.; this embodiment of the present application does not specifically limit this.

[0080] Step 204 , fill the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain sample magnetic resonance data.

[0081] The resolution of the sample magnetic resonance data may be the same as the resolution of the gold standard magnetic resonance data. Of course, the difference between the resolution of the sample magnetic resonance data and the resolution of the gold standard magnetic resonance data may also be less than a preset difference threshold.

[0082] Exemplarily, the computer device may use a first preset filling value to fill the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain sample magnetic resonance data. Exemplarily, the first preset filling value may be 0, or other random values, such as 1, -1, etc., or random noise, etc.

[0083] For the gold standard magnetic resonance data, it may include magnetic resonance data of the central area of ​​the target K space and magnetic resonance data of the edge area of ​​the target K space, and may also include magnetic resonance data of the central area of ​​the target K space, magnetic resonance data of the middle annular area of ​​the target K space, and magnetic resonance data of the edge area of ​​the target K space, etc.; that is, the computer device can divide the target K space corresponding to the gold standard magnetic resonance data into spatial areas based on different preset sizes and division rules, so as to determine the edge area in the target K space corresponding to the gold standard magnetic resonance data.

[0084] Optionally, the computer device may also divide the target K space corresponding to the gold standard magnetic resonance data into spatial regions based on evaluation parameters of the magnetic resonance data, such as resolution, quality, noise, accuracy and other parameters, so as to determine the edge region in the target K space corresponding to the gold standard magnetic resonance data; in this embodiment, the determination method and size range of the edge region in the K space are not specifically limited.

[0085] Exemplarily, when the computer device determines the edge area in the target K space corresponding to the gold standard magnetic resonance data, it can use the first preset filling data to fill the edge area, and then determine the sample magnetic resonance data based on the K space data after the filling process. Wherein, when the data type of the gold standard magnetic resonance data is K space data, the K space data after the filling process can be directly used as the sample magnetic resonance data; when the data type of the gold standard magnetic resonance data is image data, the computer device can perform image reconstruction based on the K space data after the filling process to obtain the sample magnetic resonance data.

[0086] It should be noted that, when the data type of the gold standard magnetic resonance data is image data, the computer device may also directly perform image interpolation processing on the gold standard magnetic resonance data to obtain sample magnetic resonance data whose data type is image data.

[0087] Exemplarily, when performing the filling process, the computer device may replace all data in the edge area of ​​the first magnetic resonance data with the first preset filling data; or the data in the edge area of ​​the first magnetic resonance data may be deleted first, and then the first preset filling value may be filled into each data position in the edge area, thereby obtaining the filled K-space data.

[0088] Step 206, based on the sample magnetic resonance data and the gold standard magnetic resonance data, the initial generative adversarial network is trained to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network; the target generator network is used to amplify the gold standard magnetic resonance data.

[0089] A generative adversarial network usually includes a generator network and a discriminator network, wherein the generator network is responsible for capturing the distribution of sample data, and the discriminator network is usually a binary classification network, which is used to determine whether the data input into the discriminator network is real sample data or data generated by the generator network. When training a generative adversarial network, the initial generative adversarial network is trained to obtain a trained target generative adversarial network; wherein the initial generative adversarial network is an untrained generative adversarial network formed using the initial network parameters. The initial generative adversarial network may include an initial generator network and an initial discriminator network.

[0090] Based on this, the computer device can input the sample magnetic resonance data into the initial generator network for processing to obtain amplified sample magnetic resonance data; optionally, the amplified sample magnetic resonance data can be the magnetic resonance data generated by the initial generator network; it can also be the magnetic resonance data obtained after post-processing operations on the magnetic resonance data generated by the initial generator network, etc.; then, the computer device can input the amplified sample magnetic resonance data and the gold standard magnetic resonance data into the initial discriminator network to obtain a discrimination result; wherein the discrimination result can be used to characterize the probability that the amplified sample magnetic resonance data belongs to the sample magnetic resonance data initially input; finally, based on the discrimination result, the initial generator network and the initial discriminator network are trained to obtain a target generative adversarial network.

[0091] Exemplarily, during network training, the initial generator network and the initial discriminator network may be trained alternately and iteratively; that is, one of the networks may be fixed first, and the other network may be trained iteratively, thereby performing alternating training to obtain a trained target generator network and target discriminator network.

[0092] For example, in the first training stage, the computer device can first fix the initial discriminator network, and train the initial generator network based on the discrimination results; at the beginning, the parameters of the initial generator network have not been trained well, and the generated data can be easily judged as false by the initial discriminator network. However, with continuous iterative training, the parameters of the generator network are optimized better and better, and the generated data is more and more realistic, then the initial discriminator network will not be able to accurately judge the authenticity of the generated data.

[0093] Then, entering the second training stage, the computer equipment can fix the network parameters of the generator network obtained in the first training stage, and train the initial discriminator network based on the discrimination results; through continuous iterative training, the parameters of the discriminator network are optimized better and better, and the discrimination ability is stronger and stronger, and it can accurately judge that the data generated by the generator network is false.

[0094] In the third training stage, the training operations of the first training stage and the second training stage can be repeated, so that the parameter optimization of the discriminator network and the generator network is getting better and better; finally, a target generator network and a target discriminator network with very good effects are obtained.

[0095] Among them, based on the target generator network, the high-resolution gold standard magnetic resonance data can be amplified to obtain a large amount of high-resolution magnetic resonance data. This large amount of high-resolution magnetic resonance data can be used as training data for the deep learning model, so as to train a high-precision and high-accuracy deep learning model for segmenting the target tissue.

[0096] In the above data augmentation method, the computer device obtains gold standard magnetic resonance data; then, the edge area in the target K space corresponding to the gold standard magnetic resonance data is filled to obtain sample magnetic resonance data; and based on the sample magnetic resonance data and the gold standard magnetic resonance data, the initial generative adversarial network is trained to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network; the target generator network can be used to augment the gold standard magnetic resonance data. It can be seen that in the embodiment of the present application, for a small amount of high-resolution magnetic resonance data that is actually collected, a large amount of sample magnetic resonance data with a resolution that is almost consistent with the resolution of the magnetic resonance data that is actually collected can be obtained by performing different filling processes on the edge areas in the target K space; further, the small amount of high-resolution magnetic resonance data is used as the gold standard magnetic resonance data, and together with the large amount of sample magnetic resonance data obtained by processing, is used as training data for the generative adversarial network to achieve training of the generative adversarial network; due to the large amount of training data, a high-precision target generator network can be obtained. Next, based on the high-precision target generator network, data augmentation processing is performed on a small amount of actually collected high-resolution magnetic resonance data, which not only can obtain a large amount of high-resolution magnetic resonance data, but also the high-resolution magnetic resonance data after data amplification using this method is closer to the actually collected high-resolution magnetic resonance data, and the precision and accuracy of the data augmentation processing are guaranteed; furthermore, when a large amount of high-resolution magnetic resonance data after data augmentation processing in this way is used as training data for a deep learning model, it can provide strong data support for the training of the deep learning model and can improve the training precision and model accuracy of the deep learning model.

[0097] In addition, the method proposed in the embodiment of the present application can not only expand the high-resolution magnetic resonance data required by the deep learning model but difficult to actually collect, obtain a large amount of training data, and improve the efficiency of obtaining training data; but also solve a series of problems faced in the actual process of collecting high-resolution magnetic resonance data, such as the difficulty of collection and possible motion artifacts, thereby improving the accuracy of high-resolution magnetic resonance data and the accuracy of training data; and further, it can improve the training accuracy of the deep learning model.

[0098] Figure 3 FIG. 1 is a flow chart of a data augmentation method in another embodiment. This embodiment relates to an optional implementation process in which a computer device fills the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain sample magnetic resonance data. Based on the above embodiment, Figure 3 As shown, the above step 204 includes:

[0099] Step 302 : extracting magnetic resonance data of a central region in a target K-space corresponding to the gold standard magnetic resonance data according to a preset size to obtain central magnetic resonance data.

[0100] The preset size is smaller than the size range of the target K space corresponding to the gold standard magnetic resonance data. Exemplarily, the computer device can determine the central area corresponding to the preset size from the central position of the target K space corresponding to the gold standard magnetic resonance data according to the preset size; then, extract part of the data in the central area of ​​the target K space corresponding to the gold standard magnetic resonance data as the central magnetic resonance data.

[0101] Step 304 : Based on the first preset filling value, fill processing is performed on the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain edge magnetic resonance data.

[0102] The edge region of the target K space is the K space region remaining after the central region is removed from the target K space.

[0103] Exemplarily, the computer device may replace all magnetic resonance data in the edge region of the target K space corresponding to the gold standard magnetic resonance data with the first preset filling value based on the first preset filling value, thereby obtaining filled edge magnetic resonance data. Exemplarily, the first preset filling data may include but is not limited to zero or any other arbitrary value, and may also include random noise, etc.

[0104] Step 306 , obtaining sample magnetic resonance data based on the central magnetic resonance data and the edge magnetic resonance data.

[0105] Exemplarily, the computer device may merge the central magnetic resonance data and the filled edge magnetic resonance data to obtain sample magnetic resonance data having a data volume, that is, resolution, that is, almost the same as that of the gold standard magnetic resonance data.

[0106] In this embodiment, the computer device extracts the magnetic resonance data of the central area of ​​the target K space corresponding to the gold standard magnetic resonance data according to a preset size to obtain the central magnetic resonance data; then, based on the first preset filling value, the edge area in the target K space corresponding to the gold standard magnetic resonance data is filled to obtain the filled edge magnetic resonance data; wherein the edge area of ​​the target K space is the K space area remaining after the central area is removed from the target K space; finally, based on the central magnetic resonance data and the filled edge magnetic resonance data, the sample magnetic resonance data can be obtained. That is to say, the magnetic resonance data of the central area of ​​the target K space corresponding to the gold standard magnetic resonance data is used as the magnetic resonance data of the central area of ​​the K space corresponding to the sample magnetic resonance data; then, data is filled around the magnetic resonance data of the central area to obtain the low-frequency data in the gold standard magnetic resonance data, that is, the sample magnetic resonance data whose image structure is consistent; in this way, when the sample magnetic resonance data is learned by the generative network in the generative adversarial network, the high-frequency part of the sample magnetic resonance data can be effectively learned to obtain the amplified sample magnetic resonance data with image details different from the gold standard magnetic resonance data, so that the image structure of the amplified sample magnetic resonance data is almost consistent with that of the gold standard magnetic resonance data, and the image details are changed, thereby improving the accuracy of the amplified high-resolution magnetic resonance data.

[0107] In an exemplary embodiment, when the sample magnetic resonance data includes central magnetic resonance data and edge magnetic resonance data (i.e., the first preset filling value), there may be another implementation method for the above process of obtaining the amplified sample magnetic resonance data by inputting the sample magnetic resonance data into the initial generator network; on the basis of the above embodiment, Figure 4 As shown, the implementation process may also include:

[0108] Step 402: input the sample magnetic resonance data into the initial generator network to obtain candidate magnetic resonance data.

[0109] Exemplarily, the candidate magnetic resonance data may be magnetic resonance data directly outputted after the initial generator network processes the sample magnetic resonance data; that is, the candidate magnetic resonance data may be magnetic resonance data generated by the initial generator network based on the sample magnetic resonance data.

[0110] Exemplarily, the candidate magnetic resonance data may also be magnetic resonance data generated from sample magnetic resonance data, after one or more post-processing operations are performed on the magnetic resonance data; the post-processing operations may include but are not limited to performing processing operations such as denoising, smoothing, and interpolation on the magnetic resonance data generated from the sample magnetic resonance data.

[0111] Step 404 : Replace the magnetic resonance data of the central region of the target K space corresponding to the candidate magnetic resonance data with the central magnetic resonance data to obtain the amplified sample magnetic resonance data.

[0112] Among them, the central area of ​​the candidate magnetic resonance data can be the central area determined based on the above-mentioned preset size, that is, the central area of ​​the target K control corresponding to the gold standard magnetic resonance data, or the central area of ​​the target K space corresponding to the sample magnetic resonance data, the corresponding area.

[0113] Exemplarily, when the sample magnetic resonance data is input into the initial generator network for processing, the initial generator network not only processes the central magnetic resonance data corresponding to the central area of ​​the target K space of the sample magnetic resonance data, but also processes the edge magnetic resonance data corresponding to the edge area of ​​the target K space of the sample magnetic resonance data.

[0114] In this embodiment, the computer device can further replace the magnetic resonance data of the central area of ​​the target K space corresponding to the candidate magnetic resonance data with the magnetic resonance data of the central area of ​​the target K space corresponding to the sample magnetic resonance data (that is, the magnetic resonance data of the central area of ​​the target K space corresponding to the gold standard magnetic resonance data), based on the candidate magnetic resonance data generated by the initial generator network; that is, the central magnetic resonance data of the candidate magnetic resonance data is replaced with the central magnetic resonance data of the sample magnetic resonance data (that is, the central magnetic resonance data of the gold standard magnetic resonance data); thereby obtaining the amplified sample magnetic resonance data corresponding to the sample magnetic resonance data; at this time, the magnetic resonance data of the central area of ​​the target K space between the amplified sample magnetic resonance data and the input sample magnetic resonance data (or the gold standard magnetic resonance data) are exactly the same, and only the magnetic resonance data of the edge area is different.

[0115] In this way, when the gold standard magnetic resonance data is K-space data, since the central area data of the K-space data corresponds to the low-frequency information of the image (i.e., the general shape and outline), the edge area data of the K-space data corresponds to the high-frequency information of the image (i.e., the edge, details); in this way, the low-frequency data of the K-space containing the main energy can be guaranteed not to be modified, thereby ensuring the consistency of the image structure. At the same time, the high-frequency data of the edge area of ​​the K-space is optimized and updated, so that the details of the image are changed, thereby obtaining the amplified high-resolution magnetic resonance data.

[0116] In this embodiment, in the process of processing the sample magnetic resonance data through the initial generator network to obtain the amplified sample magnetic resonance data, the sample magnetic resonance data is first input into the initial generator network for processing to obtain candidate magnetic resonance data; then, the magnetic resonance data of the central area of ​​the target K space corresponding to the candidate magnetic resonance data is replaced with the central magnetic resonance data of the target K space corresponding to the sample magnetic resonance data to obtain the amplified sample magnetic resonance data. Since in this embodiment, the magnetic resonance data corresponding to the central area of ​​the target K space corresponding to the candidate magnetic resonance data generated by the initial generator network is replaced with the central magnetic resonance data corresponding to the central area of ​​the target K space corresponding to the sample magnetic resonance data, the amplified sample magnetic resonance data can be consistent with the low-frequency information of the sample magnetic resonance data initially input, that is, it is ensured that the amplified sample magnetic resonance data can be consistent with the low-frequency information of the gold standard magnetic resonance data, and then the amplified sample magnetic resonance data can also maintain the anatomical information such as the general shape and contour of the scanned object in the gold standard magnetic resonance data, thereby improving the accuracy of the amplified sample magnetic resonance data.

[0117] In an exemplary embodiment, when the amplified sample magnetic resonance data and the gold standard magnetic resonance data are both magnetic resonance scan data, that is, K-space data, the above-mentioned initial discriminator network may also include a first initial discriminator network and a second initial discriminator network; wherein the first initial discriminator network may be used to discriminate the true or false of the first type of magnetic resonance data, for example: discriminate the true or false of K-space data; the second initial discriminator network may be used to discriminate the true or false of the second type of magnetic resonance data, for example: discriminate the true or false of image data, and the image data may be obtained after reconstruction of K-space data. Of course, the first initial discriminator network may also be used to discriminate the true or false of image data, and the second initial discriminator network may also be used to discriminate the true or false of K-space data.

[0118] Based on this, there may be another implementation method for the above process of obtaining the discrimination result by inputting the amplified sample magnetic resonance data and the gold standard magnetic resonance data into the initial discriminator network; on the basis of the above embodiment, Figure 5 As shown, the implementation process may also include:

[0119] Step 502: input the amplified sample magnetic resonance data and the gold standard magnetic resonance data into a first initial discriminator network to obtain a first discrimination result.

[0120] Exemplarily, the gold standard magnetic resonance data can be K-space data, so the amplified sample magnetic resonance data is also K-space data; after the computer device inputs the sample magnetic resonance data into the initial generator network for processing to generate amplified sample magnetic resonance data, the amplified sample magnetic resonance data and the gold standard magnetic resonance data can be input into the first initial discriminator network to perform true or false discrimination on the amplified sample magnetic resonance data output by the initial generator network to obtain a first discrimination result.

[0121] Exemplarily, the first discrimination result may be that the amplified sample magnetic resonance data is true or false, that is, a true or false discrimination result; the first discrimination result may also be the probability that the amplified sample magnetic resonance data is true. Then, the closer the first discrimination result is to 1, the higher the probability that the amplified sample magnetic resonance data is true, which means that the degree of similarity between the amplified sample magnetic resonance data and the gold standard magnetic resonance data is higher; of course, the first discrimination result may also be the probability that the amplified sample magnetic resonance data is false. In this case, the closer the first discrimination result is to 1, the higher the probability that the amplified sample magnetic resonance data is false, which means that the degree of similarity between the amplified sample magnetic resonance data and the gold standard magnetic resonance data is lower.

[0122] Step 504 , performing image reconstruction on the amplified sample magnetic resonance data to obtain a sample magnetic resonance image; and performing image reconstruction on the gold standard magnetic resonance data to obtain a gold standard magnetic resonance image.

[0123] Exemplarily, when the gold standard magnetic resonance data is K-space data, the computer device may also perform image reconstruction processing on the gold standard magnetic resonance data to obtain a gold standard magnetic resonance image corresponding to the gold standard magnetic resonance data; for example, the computer device may perform an inverse Fourier transform on the gold standard magnetic resonance data to obtain a reconstructed gold standard magnetic resonance image. Similarly, when the computer device obtains the amplified sample magnetic resonance data, it may also perform an inverse Fourier transform on the amplified sample magnetic resonance data to obtain an amplified sample magnetic resonance image corresponding to the amplified sample magnetic resonance data.

[0124] Step 506: input the sample magnetic resonance image and the gold standard magnetic resonance image into a second initial discriminator network to obtain a second discrimination result.

[0125] Among them, the gold standard magnetic resonance image is true, and the second initial discriminator network can judge the authenticity of the amplified sample magnetic resonance image based on the gold standard magnetic resonance image to obtain a second discrimination result; illustratively, the second discrimination result can be that the amplified sample magnetic resonance image is true, or false, that is, a true or false discrimination result; the second discrimination result can also be the probability that the amplified sample magnetic resonance image is true, then, the closer the second discrimination result is to 1, the higher the probability that the amplified sample magnetic resonance image is true, that is, the higher the similarity between the amplified sample magnetic resonance image and the gold standard magnetic resonance image; of course, the second discrimination result can also be the probability that the amplified sample magnetic resonance image is false, in this case, if the second discrimination result is closer to 1, the higher the probability that the amplified sample magnetic resonance image is false, that is, the lower the similarity between the amplified sample magnetic resonance image and the gold standard magnetic resonance image.

[0126] Step 508: Obtain a discrimination result according to the first discrimination result and the second discrimination result.

[0127] Exemplarily, when the first discrimination result indicates that the amplified sample magnetic resonance data is true, and the second discrimination result indicates that the amplified sample magnetic resonance image is true, the discrimination result is true, which means that the output result of the initial generator network is true, which means that the amplification result of the initial generator network is valid. Otherwise, the discrimination result is false, which means that the output result of the initial generator network is false, and the amplification result of the initial generator network is invalid.

[0128] In this embodiment, when the amplified sample magnetic resonance data and the gold standard magnetic resonance data are both magnetic resonance scan raw data, an initial discriminator network can be set up to include a first initial discriminator network and a second initial discriminator network; the computer device obtains a first discrimination result by inputting the gold standard magnetic resonance data and the amplified sample magnetic resonance data into the first initial discriminator network; at the same time, the gold standard magnetic resonance data is image reconstructed to obtain a gold standard magnetic resonance image; and the amplified sample magnetic resonance data is image reconstructed to obtain an amplified sample magnetic resonance image; and the gold standard magnetic resonance image and the amplified sample magnetic resonance image are input into the second initial discriminator network to obtain a second discrimination result; then, a discrimination result is obtained according to the first discrimination result and the second discrimination result. That is, in this embodiment, in order to further improve the accuracy of the generator network, based on the characteristics of magnetic resonance data in K space and image domain, the discriminator network is designed as a dual discriminant mechanism; that is, two discriminator networks are used to participate in the judgment at the same time, wherein the first discriminator network judges whether the augmented K space data generated by the generator network is true based on the complete K space data (i.e., high-resolution scanned data) actually collected; the second discriminator network judges whether the augmented image data obtained by inverse Fourier transform of the augmented K space data generated by the generator network is true based on the complete image data (i.e., high-resolution image data) obtained by inverse Fourier transform of the complete K space data actually collected; only when both discriminator networks output true, the amplification result of the generator network is judged to be valid; the adoption of the dual discriminant mechanism can further improve the accuracy of the generator network.

[0129] In an exemplary embodiment, based on any of the above embodiments, the sample magnetic resonance data is obtained by taking the central magnetic resonance data of the central area in the target K space corresponding to the gold standard magnetic resonance data and filling the surrounding of the central magnetic resonance data. In order to further increase the data volume of the sample magnetic resonance data, on this basis, another possible implementation method can also be used in this embodiment to obtain the sample magnetic resonance data. Figure 6 As shown, the method of acquiring sample magnetic resonance data may also include:

[0130] Step 602, acquiring initial magnetic resonance data; the resolution of the initial magnetic resonance data is smaller than the resolution of the gold standard magnetic resonance data.

[0131] That is to say, when the gold standard MRI data is high-resolution MRI data, since high-resolution MRI data is difficult to obtain, while low-resolution MRI data is easier to obtain, when obtaining sample MRI data, a portion of low-resolution MRI data, namely initial MRI data, can also be obtained to generate sample MRI data with a resolution that is almost consistent with that of the high-resolution MRI data.

[0132] Exemplarily, the initial magnetic resonance data may be magnetic resonance data with low field imaging resolution obtained by scanning with a magnetic resonance device of low field strength; or may be low resolution magnetic resonance data obtained by undersampling.

[0133] Step 604 : Based on the second preset filling value, fill processing is performed on the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data, and the filled initial magnetic resonance data is used as sample magnetic resonance data.

[0134] The second preset filling value may be the same as or different from the first preset filling value; the second preset filling value may include but is not limited to 0 and other random values.

[0135] Exemplarily, the computer device may fill the peripheral area in the initial K space corresponding to the initial magnetic resonance data in the same manner as the edge area in the target K space corresponding to the gold standard magnetic resonance data, and use the filled initial magnetic resonance data as the sample magnetic resonance data. The specific implementation method may refer to the relevant implementation method described in step 204 above, which will not be repeated here.

[0136] In this embodiment, not only can sample magnetic resonance data be obtained by filling high-resolution gold standard magnetic resonance data, but sample magnetic resonance data can also be obtained by filling low-resolution initial magnetic resonance data; thereby increasing the amount of sample magnetic resonance data and improving the training effect of the generative adversarial network.

[0137] In an exemplary embodiment, based on any of the above embodiments, when a target generative adversarial network is obtained through training, the high-resolution gold standard magnetic resonance data can be amplified based on the target generator network in the target generative adversarial network, thereby increasing the amount of high-resolution magnetic resonance data; that is, increasing the amount of training data for the deep learning model. Figure 7 As shown, the training process of the deep learning model may include:

[0138] Step 702: input the gold standard magnetic resonance data into the target generator network for amplification processing to obtain amplified gold standard magnetic resonance data.

[0139] Step 704 , using the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data as training data for the deep learning network.

[0140] Step 706: Train the deep learning network based on the training data to obtain a trained deep learning model.

[0141] Exemplarily, in the process of training a deep learning network to obtain a deep learning model based on gold standard magnetic resonance data and amplified gold standard magnetic resonance data, the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data can be used as sample data of the deep learning network, or, can also be used as gold standard data of the deep learning network; this is not specifically limited in the embodiments of the present application.

[0142] In an optional implementation, the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data (hereinafter referred to as training data) can be used as sample data of the deep learning network, and the target tissue label data corresponding to each training data can be used as gold standard data to train the deep learning network, thereby obtaining a deep learning model that can segment the target tissue in high-resolution magnetic resonance data.

[0143] In another optional implementation, the above training data can be used as gold standard data of the deep learning network, and the low-resolution magnetic resonance data can be used as sample data of the deep learning network to train the deep learning network, thereby obtaining a deep learning model that can convert low-resolution magnetic resonance data into high-resolution magnetic resonance data. Exemplarily, the low-resolution magnetic resonance data here can be obtained by undersampling the gold standard data (i.e., high-resolution magnetic resonance data).

[0144] Alternatively, high-resolution magnetic resonance data of one modality may be used as sample data for a deep learning network, and high-resolution magnetic resonance data of another modality may be used as training data for the deep learning network, and the deep learning network may be trained to obtain a deep learning model capable of modal conversion of high-resolution magnetic resonance data.

[0145] In this embodiment, for high-resolution magnetic resonance data that is more difficult to obtain, a pre-trained target generator network is used to expand the high-resolution magnetic resonance data, thereby obtaining more high-resolution magnetic resonance data, thereby expanding the training data set of the deep learning model; therefore, with the support of a large amount of high-resolution magnetic resonance data, a high-precision and high-quality deep learning model can be trained, greatly improving the accuracy of the deep learning model.

[0146] In an exemplary embodiment, a complete embodiment of a data augmentation method is provided. The complete implementation process of data augmentation includes a training process of a generative adversarial network and an augmentation process of high-resolution magnetic resonance data based on a trained target generator network; wherein, reference Figure 8 As shown, it shows a network structure of a generative adversarial network; based on the network structure, the training process of the generative adversarial network may include the following steps:

[0147] Step 1: Acquire high-resolution magnetic resonance data, that is, complete K-space data collected truly; and use the high-resolution magnetic resonance data as the gold standard magnetic resonance data for generating adversarial networks.

[0148] Optionally, a certain amount of low-resolution magnetic resonance data may also be acquired.

[0149] Step 2: for each high-resolution magnetic resonance data, extract the central magnetic resonance data of the central area in the target K space corresponding to the high-resolution magnetic resonance data, and fill the edge area of ​​the central magnetic resonance data in the target K space to obtain sample magnetic resonance data; Figure 8 The sample K-space data composed of A+B is shown in the figure; wherein A is the central magnetic resonance data of the central area in the target K-space corresponding to the high-resolution magnetic resonance data, and B is the periphery of the central area in the target K-space, that is, the edge magnetic resonance data filled by the edge area, and the filling value can be 0 or other random noise values, etc.

[0150] That is, the magnetic resonance data of the target K-space center area corresponding to the gold standard magnetic resonance data is extracted and recorded as A, and zero or randomly generated noise (recorded as area B) is filled in the periphery of A to form sample magnetic resonance data; the random noise can be composed of the same value or different values.

[0151] Optionally, based on the low-resolution magnetic resonance data, sample magnetic resonance data can be acquired by filling the peripheral area of ​​the initial K space corresponding to the low-resolution magnetic resonance data, thereby further increasing the data volume of the sample magnetic resonance data.

[0152] Step 3, input the sample magnetic resonance data into the generator network to obtain the amplified sample magnetic resonance data corresponding to the sample magnetic resonance data; that is, Figure 8 The K-space data consisting of A'+C is shown in .

[0153] Exemplarily, the generator network may focus on optimizing the high-frequency portion of the sample magnetic resonance data, ie, region B, and output optimized K-space data, ie, A'+C.

[0154] Step 4: input the gold standard MRI data and the amplified sample MRI data (ie, the amplified K-space data A'+C) into the discriminator network-1 to judge the authenticity of the amplified K-space data A'+C and obtain the first discrimination result.

[0155] Step 5, for each gold standard magnetic resonance data, the gold standard magnetic resonance data is inverse Fourier transformed to obtain a gold standard magnetic resonance image corresponding to the gold standard magnetic resonance data; similarly, for each amplified sample magnetic resonance data, the amplified sample magnetic resonance data is also inverse Fourier transformed to obtain an amplified sample magnetic resonance image corresponding to the amplified sample magnetic resonance data.

[0156] Step 6: input the gold standard magnetic resonance image and the amplified sample magnetic resonance image into the discriminator network-2 to judge the authenticity of the amplified sample magnetic resonance image and obtain a second discrimination result.

[0157] Step 7, based on the first discrimination result and the second discrimination result, obtain the discrimination result of the generator network; illustratively, when both discrimination results are true, it can be determined that the generated data is true.

[0158] Step 8, based on the discrimination result obtained in step 7, iteratively train the generator network, the discriminator network-1 and the discriminator network-2 until the preset iteration stop condition is met, thereby obtaining a trained target generative adversarial network; the target generative adversarial network includes the trained target generator network; the preset iteration stop condition may include that the central magnetic resonance data in the target K space corresponding to the amplified sample magnetic resonance data generated by the generator network and the central magnetic resonance data in the target K space corresponding to the originally acquired gold standard magnetic resonance data (that is, the central magnetic resonance data in the target K space corresponding to the sample magnetic resonance data) are more similar than a preset similarity threshold, that is, A' is approximately equal to A; or, the similarity between the amplified sample magnetic resonance data and the gold standard magnetic resonance data is greater than a preset similarity threshold, etc.

[0159] For example, during the training process, A' can be made closer and closer to A through the loss function, or A' can be directly replaced by the input data A through the data fidelity term; that is, in the above step 3, A' in the K-space data composed of A'+C generated by the generator network can be directly replaced by A, and the K-space data composed of A+C can be obtained as the amplified sample magnetic resonance data; then, the amplified sample magnetic resonance data composed of A+C and the originally collected gold standard magnetic resonance data are input into the discriminator network-1 for true or false judgment; at the same time, the gold standard magnetic resonance data The corresponding gold standard magnetic resonance image and the amplified sample magnetic resonance image obtained by inverse Fourier transform of the amplified sample magnetic resonance data composed of A+C are input into the discriminator network-2 for true or false judgment, and the discrimination result of the generator network is obtained; then based on the discrimination result, the generator network, the discriminator network-1 and the discriminator network-2 are iteratively trained until the preset iteration stop condition is met, so as to obtain a trained target generative adversarial network; it should be noted that the preset iteration stop condition here may also include but is not limited to meeting the preset number of iterations, meeting the preset loss function value, etc.

[0160] This training method can ensure that the low-frequency data of the K space containing the main energy is not modified, and the accuracy of the image structure can be guaranteed; at the same time, the high-frequency data outside the center of the K space is optimized and updated, so that the details of the image are changed, and the amplified magnetic resonance data are obtained.

[0161] In addition, in this embodiment, in order to ensure the accuracy of the generator network, based on the characteristics of the magnetic resonance data in the K space and image domains, the discriminator network is designed as a dual discriminant mechanism; that is, two discriminator networks are involved in the judgment at the same time, and the discriminator network-1 judges whether the augmented K space data generated by the generator network is true based on the complete K space data actually collected; the discriminator network-2 judges whether the image data obtained by inverse Fourier transform of the augmented K space data generated by the generator network is true based on the complete image data obtained by inverse Fourier transform of the complete K space data actually collected; only when both discriminator networks output true, can the amplification result of the generator network be judged to be valid; the use of a dual discriminant mechanism can further improve the accuracy of the generator network.

[0162] In one embodiment, for the generator network, it can also use image domain data directly as network input, that is, the high-resolution gold standard magnetic resonance data can also include high-resolution gold standard magnetic resonance images; Fig. 9 As shown, it shows a schematic diagram of the structure of training a generative adversarial network based on image domain data.

[0163] Exemplarily, when complete K-space data is acquired, the complete K-space data can be inverse Fourier transformed to obtain a complete magnetic resonance image, and the complete magnetic resonance image can be used as a high-resolution gold standard magnetic resonance image; at the same time, based on the gold standard magnetic resonance image, a sample magnetic resonance image can be obtained by performing interpolation processing such as filling zeros or random noise on the gold standard magnetic resonance image.

[0164] Furthermore, the sample magnetic resonance image is input into the generator network to obtain an amplified sample magnetic resonance image corresponding to the sample magnetic resonance image; that is, Fig. 9 Then, the amplified sample magnetic resonance image and the gold standard magnetic resonance image can be input into the discriminator network to judge the authenticity of the amplified sample magnetic resonance image and obtain a discrimination result; the discrimination result may include that the amplified sample magnetic resonance image is true, or that the amplified sample magnetic resonance image is false.

[0165] Furthermore, based on the discrimination result, the generator network and the discriminator network can be iteratively trained until a preset iteration stop condition is met, thereby obtaining a trained target generative adversarial network; the target generative adversarial network includes the trained target generator network. Exemplarily, the preset iteration stop condition may include but is not limited to the similarity between the amplified sample magnetic resonance image and the gold standard magnetic resonance image being greater than a preset similarity threshold, meeting a preset number of iterations, meeting a preset loss function value, etc.

[0166] In this embodiment, the computer device can perform an inverse Fourier transform on the real collected K-space data to obtain a gold standard magnetic resonance image, and then interpolate the gold standard magnetic resonance image to obtain a sample magnetic resonance image, such as: the sample magnetic resonance image can be obtained after filling 0 interpolation on the gold standard magnetic resonance image; then, the sample magnetic resonance image is input into the generator network for optimization processing to obtain an optimized magnetic resonance image; then, the gold standard magnetic resonance image and the optimized magnetic resonance image are input into the discriminator network for true and false judgment to obtain a judgment result; based on the judgment result, the generator network and the discriminator network are iteratively trained to finally obtain a trained target generative adversarial network, including a target generator network and a target discriminator network. That is, in the image domain, based on the high-resolution gold standard magnetic resonance image, the training of the generative adversarial network can also be realized, so as to obtain a target generator network for amplifying the high-resolution gold standard magnetic resonance image, so as to realize the amplification of the high-resolution image that is difficult to obtain, and obtain a large number of high-resolution images, so as to provide a large amount of training data support for the training of the deep learning model, and improve the training effect of the deep learning model.

[0167] In one embodiment, Fig.10As shown, a model training method is also provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:

[0168] Step 1002: Acquire gold standard magnetic resonance data.

[0169] Step 1004, acquiring sample magnetic resonance data; the method of acquiring the sample magnetic resonance data includes filling and acquiring the edge area in the target K space corresponding to the gold standard magnetic resonance data, and / or filling and acquiring the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data.

[0170] Step 1006, based on the sample magnetic resonance data and the gold standard magnetic resonance data, the initial generative adversarial network is trained to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network.

[0171] Step 1008: input the gold standard magnetic resonance data into the target generator network for amplification processing to obtain amplified gold standard magnetic resonance data.

[0172] Step 1010: training a deep learning network based on the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data to obtain a trained deep learning model.

[0173] In this embodiment, the method of amplifying data in step 1002 and step 1008 may refer to any of the above embodiments, and will not be repeated here.

[0174] It should be noted that in the above step 1004, the sample magnetic resonance data can be obtained from the gold standard magnetic resonance data or the initial magnetic resonance data; of course, part of the sample magnetic resonance data can be obtained from the gold standard magnetic resonance data, and the other part can be obtained from the initial magnetic resonance data. The resolution of the initial magnetic resonance data can be less than the resolution of the gold standard magnetic resonance data; illustratively, the initial magnetic resonance data can be obtained by scanning based on undersampling, and the gold standard magnetic resonance data can be obtained by scanning based on full sampling.

[0175] Based on this, after obtaining the augmented training data, the initial deep learning network can be trained based on the augmented training data, including the real high-resolution magnetic resonance data and the augmented high-resolution magnetic resonance data, so as to obtain a trained deep learning model. The specific implementation method can refer to the above data augmentation method. Figure 7 The embodiments shown will not be repeated here.

[0176] In this embodiment, for high-resolution magnetic resonance data that is more difficult to obtain, a pre-trained target generator network is used to expand the high-resolution magnetic resonance data, thereby obtaining more high-resolution magnetic resonance data, thereby expanding the training data set of the deep learning model; therefore, with the support of a large amount of high-resolution magnetic resonance data, a high-precision and high-quality deep learning model can be trained, greatly improving the accuracy of the deep learning model.

[0177] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0178] Based on the same inventive concept, the embodiment of the present application also provides a data amplification device for implementing the data amplification method involved above, and provides a model training device for implementing the model training method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data amplification device embodiments provided below can refer to the limitations on the data amplification method above, and the specific limitations in the model training device embodiments provided can refer to the limitations on the model training method above, which will not be repeated here.

[0179] In one embodiment, Fig.11 As shown, a data augmentation device is provided, including: an acquisition module 1102, a processing module 1104 and a first training module 1106, wherein:

[0180] The acquisition module 1102 is used to acquire gold standard magnetic resonance data.

[0181] The processing module 1104 is used to fill the edge area of ​​the target K space corresponding to the gold standard MRI data to obtain sample MRI data; and / or fill the peripheral area of ​​the initial K space corresponding to the initial MRI data to obtain sample MRI data.

[0182] The first training module 1106 is used to train the initial generative adversarial network based on the sample magnetic resonance data and the gold standard magnetic resonance data to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network; the target generator network is used to amplify the gold standard magnetic resonance data.

[0183] In one embodiment, the processing module 1104 includes:

[0184] An extraction submodule, used to extract the magnetic resonance data of the central area in the target K space corresponding to the gold standard magnetic resonance data according to a preset size to obtain central magnetic resonance data;

[0185] A filling submodule is used to fill the edge area in the target K space corresponding to the gold standard magnetic resonance data based on a first preset filling value to obtain edge magnetic resonance data; the edge area of ​​the target K space is the K space area remaining after the central area is removed from the target K space;

[0186] The generation submodule is used to obtain sample magnetic resonance data based on the central magnetic resonance data and the edge magnetic resonance data.

[0187] In one embodiment, the initial generative adversarial network includes an initial generator network and an initial discriminator network; the training module 1106 includes:

[0188] A processing submodule, used for inputting the sample magnetic resonance data into the initial generator network to obtain amplified sample magnetic resonance data;

[0189] The discriminant submodule is used to input the amplified sample magnetic resonance data and the gold standard magnetic resonance data into the initial discriminator network to obtain a discrimination result;

[0190] The training submodule is used to train the initial generator network and the initial discriminator network based on the discrimination results to obtain the target generative adversarial network.

[0191] In one embodiment, the processing submodule includes:

[0192] a processing unit, configured to input the sample magnetic resonance data into an initial generator network to obtain candidate magnetic resonance data;

[0193] The replacement unit is used to replace the magnetic resonance data of the central area of ​​the target K space corresponding to the candidate magnetic resonance data with the central magnetic resonance data to obtain the amplified sample magnetic resonance data.

[0194] In one embodiment, the amplified sample magnetic resonance data and the gold standard magnetic resonance data are both magnetic resonance scan raw data; the initial discriminator network includes a first initial discriminator network and a second initial discriminator network; the discriminant submodule includes:

[0195] A first discriminant unit, used for inputting the amplified sample magnetic resonance data and the gold standard magnetic resonance data into a first initial discriminator network to obtain a first discriminant result;

[0196] A reconstruction unit, configured to reconstruct the amplified sample magnetic resonance data to obtain a sample magnetic resonance image; and to reconstruct the gold standard magnetic resonance data to obtain a gold standard magnetic resonance image;

[0197] A first discriminant unit is used to input the sample magnetic resonance image and the gold standard magnetic resonance image into a second initial discriminator network to obtain a second discriminant result;

[0198] The generating unit is used to obtain a discrimination result according to the first discrimination result and the second discrimination result.

[0199] In one embodiment, the device further comprises:

[0200] an amplification module, used for inputting the gold standard magnetic resonance data into the target generator network for amplification processing to obtain amplified gold standard magnetic resonance data;

[0201] A determination module is used to use the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data as training data for a deep learning network;

[0202] The second training module is used to train the deep learning network based on the training data to obtain a trained deep learning model.

[0203] In one embodiment, the training data serves as sample data for a deep learning network, or serves as gold standard data for a deep learning network.

[0204] In one embodiment, Fig.12 As shown, a model training device is provided, including: a first acquisition module 1202, a second acquisition module 1204, a first training module 1206, an amplification module 1208 and a second training module 1210, wherein:

[0205] The first acquisition module 1202 is used to acquire gold standard magnetic resonance data.

[0206] The second acquisition module 1204 is used to acquire sample magnetic resonance data; the acquisition method of the sample magnetic resonance data includes filling and acquiring the edge area in the target K space corresponding to the gold standard magnetic resonance data, and / or filling and acquiring the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data.

[0207] The first training module 1206 is used to train the initial generative adversarial network based on the sample magnetic resonance data and the gold standard magnetic resonance data to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network.

[0208] The amplification module 1208 is used to input the gold standard magnetic resonance data into the target generator network for amplification processing to obtain amplified gold standard magnetic resonance data.

[0209] The second training module 1210 is used to train the deep learning network based on the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data to obtain a trained deep learning model.

[0210] Each module in the above-mentioned data augmentation device and model training device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0211] In one embodiment, a computer device is provided. The computer device may be a terminal or a server. The internal structure diagram thereof may be as follows: Fig.13 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the collected magnetic resonance data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data augmentation method and / or a model training method is implemented. In the case where the computer device is a terminal, the computer device may also include a display screen and an input device (not shown in the figure), the display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0212] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0213] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the data augmentation method and / or model training method in any of the above embodiments are implemented.

[0214] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data augmentation method and / or model training method in any of the above embodiments are implemented.

[0215] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the data augmentation method and / or model training method in any of the above embodiments.

[0216] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0217] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0218] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0219] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data augmentation method, characterized in that: The method comprises: Acquire gold standard MRI data; Filling the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain sample magnetic resonance data; Based on the sample magnetic resonance data and the gold standard magnetic resonance data, an initial generative adversarial network is trained to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network; the target generator network is used to amplify the gold standard magnetic resonance data.

2. The method according to claim 1, characterized in that The filling process is performed on the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain the sample magnetic resonance data, including: Extracting magnetic resonance data of a central area in the target K space corresponding to the gold standard magnetic resonance data according to a preset size to obtain central magnetic resonance data; Based on a first preset filling value, a filling process is performed on the edge area in the target K space corresponding to the gold standard magnetic resonance data to obtain edge magnetic resonance data; the edge area of ​​the target K space is the K space area remaining after the central area is removed from the target K space; The sample magnetic resonance data is obtained based on the central magnetic resonance data and the edge magnetic resonance data.

3. The method according to claim 2, characterized in that The initial generative adversarial network includes an initial generator network and an initial discriminator network; the initial generative adversarial network is trained based on the sample magnetic resonance data and the gold standard magnetic resonance data to obtain a target generative adversarial network, including: Inputting the sample magnetic resonance data into the initial generator network to obtain amplified sample magnetic resonance data; Inputting the amplified sample magnetic resonance data and the gold standard magnetic resonance data into the initial discriminator network to obtain a discrimination result; Based on the discrimination result, the initial generator network and the initial discriminator network are trained to obtain the target generative adversarial network.

4. The method according to claim 3, characterized in that The step of inputting the sample magnetic resonance data into the initial generator network to obtain amplified sample magnetic resonance data comprises: inputting the sample magnetic resonance data into the initial generator network to obtain candidate magnetic resonance data; The magnetic resonance data of the central area of ​​the target K space corresponding to the candidate magnetic resonance data is replaced with the central magnetic resonance data to obtain the amplified sample magnetic resonance data.

5. The method according to claim 3 or 4, characterized in that: The amplified sample magnetic resonance data and the gold standard magnetic resonance data are both magnetic resonance scan raw data; the initial discriminator network includes a first initial discriminator network and a second initial discriminator network; the amplified sample magnetic resonance data and the gold standard magnetic resonance data are input into the initial discriminator network to obtain a discrimination result, including: Inputting the amplified sample magnetic resonance data and the gold standard magnetic resonance data into the first initial discriminator network to obtain a first discrimination result; Performing image reconstruction on the amplified sample magnetic resonance data to obtain a sample magnetic resonance image; and performing image reconstruction on the gold standard magnetic resonance data to obtain a gold standard magnetic resonance image; Inputting the sample magnetic resonance image and the gold standard magnetic resonance image into the second initial discriminator network to obtain a second discrimination result; The discrimination result is obtained according to the first discrimination result and the second discrimination result.

6. The method according to claim 1, characterized in that The method further comprises: Acquiring initial magnetic resonance data; wherein the resolution of the initial magnetic resonance data is less than the resolution of the gold standard magnetic resonance data; Based on the second preset filling value, a filling process is performed on the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data, and the filled initial magnetic resonance data is used as the sample magnetic resonance data.

7. The method according to claim 1, characterized in that The method further comprises: Inputting the gold standard magnetic resonance data into the target generator network for amplification processing to obtain amplified gold standard magnetic resonance data; Using the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data as training data for a deep learning network; Based on the training data, the deep learning network is trained to obtain a trained deep learning model.

8. The method according to claim 7, characterized in that The training data serves as sample data of the deep learning network, or serves as gold standard data of the deep learning network.

9. A model training method, characterized in that: The method comprises: Acquire gold standard MRI data; Acquiring sample magnetic resonance data; the method of acquiring the sample magnetic resonance data includes filling and acquiring the edge area in the target K space corresponding to the gold standard magnetic resonance data, and / or filling and acquiring the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data; Based on the sample magnetic resonance data and the gold standard magnetic resonance data, an initial generative adversarial network is trained to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network; Inputting the gold standard magnetic resonance data into the target generator network for amplification processing to obtain amplified gold standard magnetic resonance data; Based on the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data, a deep learning network is trained to obtain a trained deep learning model.

10. A data amplification device, characterized in that: The device comprises: an acquisition module for acquiring gold standard magnetic resonance data; A processing module, configured to fill in the edge area of ​​the target K space corresponding to the gold standard magnetic resonance data to obtain sample magnetic resonance data; and / or fill in the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data to obtain sample magnetic resonance data; The first training module is used to train an initial generative adversarial network based on the sample magnetic resonance data and the gold standard magnetic resonance data to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network; the target generator network is used to amplify the gold standard magnetic resonance data.

11. A model training device, characterized in that: The device comprises: A first acquisition module, used to acquire gold standard magnetic resonance data; A second acquisition module is used to acquire sample magnetic resonance data; the sample magnetic resonance data is acquired by filling and acquiring the edge area in the target K space corresponding to the gold standard magnetic resonance data, and / or filling and acquiring the peripheral area of ​​the initial K space corresponding to the initial magnetic resonance data; A first training module is used to train an initial generative adversarial network based on the sample magnetic resonance data and the gold standard magnetic resonance data to obtain a target generative adversarial network; the target generative adversarial network includes a target generator network; an amplification module, used for inputting the gold standard magnetic resonance data into the target generator network for amplification processing to obtain amplified gold standard magnetic resonance data; The second training module is used to train the deep learning network based on the gold standard magnetic resonance data and the amplified gold standard magnetic resonance data to obtain a trained deep learning model.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images