Kidney nuclear magnetic sequence conversion and image segmentation method and system

Abstract

The invention provides a kidney nuclear magnetic sequence conversion and image segmentation method and system, and the method comprises the following steps: obtaining a kidney image, and dividing the obtained kidney image into a labeled group image and an unlabeled group image; constructing a kidney image sequence conversion network through the unlabeled group images; constructing a kidney image automatic annotation and segmentation network through the annotated group images; and combining the kidney image sequence conversion network and the kidney image automatic annotation and segmentation network to construct a kidney image sequence conversion and image segmentation unit. Through the kidney image sequence conversion and image segmentation unit, a T1c sequence of a kidney nuclear magnetic image can generate a T1 sequence, a T2 sequence, a T2c sequence, a T2FS sequence, a DWI sequence and other sequences, and the steps of nuclear magnetic examination are greatly reduced; meanwhile, differences among the sequences are reduced, marked and segmented kidney nuclear magnetic images can be directly generated, investment of manpower and material resources is reduced, and convenience is provided for clinical and scientific research.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a kidney nuclear magnetic resonance sequence conversion and image segmentation method and system. Background Art

[0002] Kidney cancer is a common malignant tumor. In my country, there are 77,300 new cases of kidney cancer and 24,000 deaths each year, which seriously endangers people's health. Early detection, early diagnosis and early treatment are the key to preventing and controlling kidney cancer. The early diagnosis of kidney cancer mainly relies on imaging examinations. Among various imaging examination methods, magnetic resonance imaging (MRI) is favored by clinicians and researchers because it contains multiple sequences of images and can provide more comprehensive information.

[0003] A complete renal MRI image includes T1, T2, T1c, T2c, T2FS and DWI sequences. It usually takes 30 minutes to complete a renal MRI scan in order to obtain all the image sequences, which means that one machine can only perform 48 scans per day, which is far from meeting the actual clinical needs.

[0004] On the other hand, in clinical or scientific research, we need to mark and segment tumors on kidney MRI images to extract tumor features. Relying on manpower to mark and segment each sequence of each kidney MRI will undoubtedly require huge manpower and material resources, and there are subjective differences in the operation of different people. The development of artificial intelligence has brought hope for solving this problem. However, the existing method is to automatically mark all sequences after a specific MRI sequence, which assumes that all sequences in the MRI are consistent. However, in reality, there are differences between the imaging of different sequences, which leads to large errors in the model trained on a single sequence when generalized to other sequences for marking and segmentation, which cannot meet the needs of clinical and scientific research. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method and system for converting and segmenting kidney nuclear magnetic resonance sequences, which shortens the inspection process, reduces the differences between sequences, and facilitates intelligent labeling and segmentation.

[0006] In a first aspect, an embodiment of the present invention provides a method for converting a renal nuclear magnetic resonance sequence and image segmentation, and the specific technical scheme is as follows: A method for converting a renal nuclear magnetic resonance sequence and image segmentation, comprising:

[0007] Acquire kidney images, and divide the acquired kidney images into a labeled group of images and an unlabeled group of images;

[0008] Construct a kidney image sequence conversion network using unlabeled group images;

[0009] Constructing a kidney image automatic annotation and segmentation network through annotated group images;

[0010] Combining the kidney image sequence conversion network with the kidney image automatic annotation and segmentation network to construct a kidney image sequence conversion and image segmentation unit;

[0011] The unlabeled group of images is input into the kidney image sequence conversion and image segmentation unit to obtain labeled and segmented kidney images of each sequence.

[0012] Furthermore, the step of acquiring kidney images and dividing the acquired kidney images into a labeled group of images and an unlabeled group of images specifically includes:

[0013] After acquiring the kidney images, a portion of the kidney images are manually labeled and segmented to obtain the labeled group of images, and another portion of the images that are not labeled and segmented are the unlabeled group of images.

[0014] Furthermore, the step of constructing a kidney image sequence conversion network using the unlabeled group images specifically includes:

[0015] The unlabeled group images are put into the generator network as input data and target data for image-level domain adaptive training, and finally the kidney image sequence conversion network is formed.

[0016] Furthermore, the step of constructing a kidney image automatic annotation and segmentation network using the annotated group of images specifically includes:

[0017] The labeled group images are input as input data and target data into the encoder for feature extraction, and finally the kidney image automatic labeling and segmentation network is formed.

[0018] Furthermore, in the step of acquiring a kidney image and dividing the acquired kidney image into a labeled group image and an unlabeled group image: the kidney image is acquired as a kidney nuclear magnetic resonance image, and the kidney nuclear magnetic resonance image includes a T1 sequence, a T2 sequence, a T1c sequence, a T2c sequence, a T2FS sequence and a DWI sequence.

[0019] Furthermore, the step of placing the unlabeled group of images as input data and target data into the generator network for image-level domain adaptive training to finally form the kidney image sequence conversion network specifically includes:

[0020] The T1c sequence and T1 sequence, T2 sequence, T2c sequence, T2FS sequence and DWI sequence of all unlabeled group images are respectively put into the generator network as input data and target data for image-level domain adaptive training, and finally the kidney image sequence conversion network that can generate other sequences through the T1c sequence is constructed.

[0021] Furthermore, the step of inputting the labeled group images as input data and target data into the encoder for feature extraction, and finally forming the kidney image automatic labeling and segmentation network specifically includes:

[0022] The pre-annotation sequence and the annotated sequence of the annotated group of images are respectively input into the encoder as input data and target data for feature extraction. In the encoder, these data are collaboratively constrained by the segmenter and the reconstructor, and finally form the automatic annotation and segmentation network of kidney images that can automatically annotate and segment unlabeled sequences.

[0023] Furthermore, the step of inputting the unlabeled group of images into the kidney image sequence conversion and image segmentation unit to obtain labeled and segmented kidney images of each sequence specifically includes:

[0024] The T1c sequence in the unlabeled group of images is input into the kidney image sequence conversion and image segmentation unit for processing to obtain labeled and segmented kidney images of each sequence.

[0025] Furthermore, in the step of inputting the T1c sequence in the unlabeled group of images into the kidney image sequence conversion and image segmentation unit for processing to obtain labeled and segmented kidney images of each sequence:

[0026] The kidney image sequence conversion network of the kidney image sequence conversion and image segmentation unit is used to convert the T1c sequence in the unlabeled group of images into a T1 sequence, a T2 sequence, a T2c sequence, a T2FS sequence and a DWI sequence;

[0027] The renal image automatic annotation and segmentation network of the renal image sequence conversion and image segmentation unit is used to automatically annotate and segment the T1c sequence in the unannotated group of images and the T1 sequence, T2 sequence, T2c sequence, T2FS sequence and DWI sequence generated by the renal image sequence conversion network, and finally the annotated and segmented renal images of each sequence are obtained.

[0028] On the other hand, an embodiment of the present invention provides a kidney nuclear magnetic resonance sequence conversion and image segmentation system, the kidney nuclear magnetic resonance sequence conversion and image segmentation system comprising:

[0029] An acquisition module is used to acquire kidney images and divide the acquired kidney images into a labeled group of images and an unlabeled group of images:

[0030] A conversion network construction module, used to construct a kidney image sequence conversion network using unlabeled group images;

[0031] A segmentation network construction module is used to construct a kidney image automatic annotation and segmentation network using an annotated group of images;

[0032] A combination module, used for combining the kidney image sequence conversion network and the kidney image automatic annotation and segmentation network to construct a kidney image sequence conversion and image segmentation unit;

[0033] The output module is used to input the unlabeled group of images into the kidney image sequence conversion and image segmentation unit to obtain labeled and segmented kidney images of each sequence.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows: a kidney image sequence conversion network is constructed through an unlabeled group of images; a kidney image automatic annotation and segmentation network is constructed through an labeled group of images; and then the two networks are combined to construct a kidney image sequence conversion and image segmentation unit, through which labeled and segmented kidney images of various sequences can be obtained, thereby through which the kidney image sequence conversion and image segmentation unit can generate T1 sequences, T2 sequences, T2c sequences, T2FS sequences, DWI sequences and other sequences from the T1c sequence of kidney nuclear magnetic resonance images, greatly reducing the steps of nuclear magnetic resonance examination; at the same time, the differences between the various sequences are reduced, and labeled and segmented kidney nuclear magnetic resonance images can be directly generated, reducing the investment in manpower and material resources, and providing convenience for clinical and scientific research. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 It is a flow chart of the kidney nuclear magnetic resonance sequence conversion and image segmentation method in the present invention.

[0037] Figure 2 It is a flow chart of the kidney nuclear magnetic resonance sequence conversion and image segmentation system in the present invention.

[0038] Figure 3 It is a flow chart of the kidney image sequence conversion and image segmentation unit in the present invention processing the kidney nuclear magnetic resonance image T1c sequence.

[0039] Figure 4 It is a schematic diagram of the structure of the kidney image sequence conversion network in the present invention.

[0040] Figure 5 It is a schematic diagram of the structure of the kidney image automatic annotation and segmentation network in the present invention. DETAILED DESCRIPTION

[0041] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0042] like Figure 1 FIG. 1 is a first embodiment of the present invention, which provides a method for converting kidney nuclear magnetic resonance sequences and image segmentation, comprising the following steps:

[0043] S1. Acquire kidney images, and divide the acquired kidney images into a labeled group of images and an unlabeled group of images:

[0044] Specifically, after obtaining the kidney image, a part of the kidney image is manually labeled and segmented to obtain the labeled group image, and the other part of the unlabeled and segmented image is the unlabeled group image, wherein the obtained kidney image is a real kidney nuclear magnetic resonance image, and the kidney nuclear magnetic resonance image includes a T1 sequence, a T2 sequence, a T1c sequence, a T2c sequence, a T2FS sequence and a DWI sequence.

[0045] S2, constructing a kidney image sequence conversion network using unlabeled group images;

[0046] It can be understood that the unlabeled group images are put into the generator network as input data and target data for image-level domain adaptive training, and finally the kidney image sequence conversion network is formed;

[0047] Specifically, Figure 4 As shown, the T1c sequence and T1 sequence, T2 sequence, T2c sequence, T2FS sequence and DWI sequence of all unlabeled group images are respectively put into the generator network as input data and target data for image-level domain adaptive training. The training is conducted for approximately 50,000 iterations, and finally the kidney image sequence conversion network that can generate other sequences through the T1c sequence is constructed.

[0048] S3, constructing a kidney image automatic annotation and segmentation network through the annotated group images;

[0049] It can be understood that the labeled group images are input as input data and target data into the encoder for feature extraction, and finally the kidney image automatic labeling and segmentation network is formed;

[0050] Specifically, Figure 5As shown, the pre-annotation sequence and the annotated sequence of the annotated group of images are respectively input into the encoder as input data and target data for feature extraction. In the encoder, these data are collaboratively constrained by the segmenter and the reconstructor. After tens of thousands of times of learning, the automatic annotation and segmentation network for kidney images is finally formed, which can automatically annotate and segment unlabeled sequences.

[0051] S4, combining the kidney image sequence conversion network and the kidney image automatic annotation and segmentation network to construct a kidney image sequence conversion and image segmentation unit;

[0052] It can be understood that the kidney image sequence conversion and image segmentation unit is formed by combining the kidney image sequence conversion network and the kidney image automatic annotation and segmentation network.

[0053] S5. Input the unlabeled group of images into the kidney image sequence conversion and image segmentation unit to obtain labeled and segmented kidney image sequences.

[0054] It can be understood that the T1c sequence in the unlabeled group of images is input into the kidney image sequence conversion and image segmentation unit for processing to obtain labeled and segmented kidney images of each sequence.

[0055] like Figure 3 As shown, the kidney image sequence conversion and image segmentation unit processes the T1c sequence in the unlabeled group image including the following steps:

[0056] The kidney image sequence conversion network of the kidney image sequence conversion and image segmentation unit is used to convert the T1c sequence in the unlabeled group of images into a T1 sequence, a T2 sequence, a T2c sequence, a T2FS sequence, a DWI sequence, and the like;

[0057] The renal image automatic annotation and segmentation network of the renal image sequence conversion and image segmentation unit is used to automatically annotate and segment the T1c sequence in the unannotated group of images and the T1 sequence, T2 sequence, T2c sequence, T2FS sequence, DWI and other sequences generated by the renal image sequence conversion network, and finally the annotated and segmented renal magnetic resonance images of each sequence are obtained.

[0058] In summary, through the above steps, T1 sequence, T2 sequence, T2c sequence, T2FS sequence and DWI sequence can be directly generated through the T1c sequence of renal MRI images, which greatly reduces the steps of MRI examination; at the same time, annotated and segmented renal MRI images can be directly generated, reducing the investment in manpower and material resources, and providing convenience for clinical and scientific research.

[0059] Embodiment 2

[0060] This embodiment provides a structural block diagram of a system corresponding to the method described in Embodiment 1. Figure 2 The figure is a structural block diagram of a kidney nuclear magnetic resonance sequence conversion and image segmentation system according to an embodiment of the present application, the system comprising:

[0061] A kidney nuclear magnetic resonance sequence conversion and image segmentation system, comprising:

[0062] The acquisition module 1 is used to acquire kidney images and divide the acquired kidney images into a labeled group of images and an unlabeled group of images:

[0063] A conversion network construction module 2 is used to construct a kidney image sequence conversion network through unlabeled group images;

[0064] A segmentation network construction module 3, used for constructing a kidney image automatic annotation and segmentation network through annotated group images;

[0065] A combination module 4, used for combining the kidney image sequence conversion network and the kidney image automatic annotation and segmentation network to construct a kidney image sequence conversion and image segmentation unit;

[0066] The output module 5 is used to input the unlabeled group of images into the kidney image sequence conversion and image segmentation unit to obtain labeled and segmented kidney image sequences.

[0067] Furthermore, the acquisition module 1 is specifically used to: after acquiring the kidney image, manually annotate and segment a portion of the kidney image to obtain the annotated group image, and the other portion of the unannotated and segmented image is the unannotated group image; wherein, the kidney image acquired is a kidney nuclear magnetic resonance image, and the kidney nuclear magnetic resonance image includes a T1 sequence, a T2 sequence, a T1c sequence, a T2c sequence, a T2FS sequence and a DWI sequence.

[0068] Furthermore, the conversion network construction module 2 is specifically used to: put the T1c sequence and T1 sequence, T2 sequence, T2c sequence, T2FS sequence and DWI sequence of all unlabeled group images as input data and target data into the generator network for image-level domain adaptive training, and finally construct the kidney image sequence conversion network that can generate other sequences through the T1c sequence.

[0069] Furthermore, the segmentation network construction module 3 is specifically used to: input the pre-annotation sequence and the annotated sequence of the annotated group of images as input data and target data into the encoder for feature extraction, and in the encoder, these data are collaboratively constrained by the segmenter and the reconstructor, and finally form the kidney image automatic annotation and segmentation network that can automatically annotate and segment the unlabeled sequence.

[0070] Furthermore, the output module 5 is specifically used to: input the T1c sequence in the unlabeled group of images into the kidney image sequence conversion and image segmentation unit for processing to obtain labeled and segmented kidney images of each sequence;

[0071] The kidney image sequence conversion network of the kidney image sequence conversion and image segmentation unit is used to convert the T1c sequence in the unlabeled group image into a T1 sequence, a T2 sequence, a T2c sequence, a T2FS sequence and a DWI sequence;

[0072] The renal image automatic annotation and segmentation network of the renal image sequence conversion and image segmentation unit is used to automatically annotate and segment the T1c sequence in the unannotated group of images and the T1 sequence, T2 sequence, T2c sequence, T2FS sequence and DWI sequence generated by the renal image sequence conversion network, and finally the annotated and segmented renal images of each sequence are obtained.

[0073] Embodiment 3

[0074] The embodiment of the present invention further provides a storage medium on which an application program is stored. When the program is executed by a processor, the steps of the kidney nuclear magnetic resonance sequence conversion and image segmentation method in the above-mentioned embodiment 1 are implemented.

[0075] Embodiment 4

[0076] An embodiment of the present invention also provides an intelligent device, including a memory, a processor, and an application stored in the memory and executable on the processor. When the processor executes the program, the steps of the kidney nuclear magnetic resonance sequence conversion and image segmentation method in the above-mentioned embodiment 1 are implemented.

[0077] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0078] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0079] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0080] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for renal nuclear magnetic resonance sequence conversion and image segmentation, characterized in that: include: Acquire kidney images, and divide the acquired kidney images into a labeled group of images and an unlabeled group of images; Construct a kidney image sequence conversion network using unlabeled group images; Constructing a kidney image automatic annotation and segmentation network through annotated group images; Combining the kidney image sequence conversion network with the kidney image automatic annotation and segmentation network to construct a kidney image sequence conversion and image segmentation unit; The unlabeled group of images is input into the kidney image sequence conversion and image segmentation unit to obtain labeled and segmented kidney images of each sequence.

2. The method for renal nuclear magnetic resonance sequence conversion and image segmentation according to claim 1, characterized in that: The step of acquiring kidney images and dividing the acquired kidney images into a labeled group of images and an unlabeled group of images specifically includes: After acquiring the kidney images, a portion of the kidney images are manually labeled and segmented to obtain the labeled group of images, and another portion of the images that are not labeled and segmented are the unlabeled group of images.

3. The method for renal nuclear magnetic resonance sequence conversion and image segmentation according to claim 1, characterized in that: The step of constructing a kidney image sequence conversion network using the unlabeled group images specifically includes: The unlabeled group images are put into the generator network as input data and target data for image-level domain adaptive training, and finally the kidney image sequence conversion network is formed.

4. The method for renal nuclear magnetic resonance sequence conversion and image segmentation according to claim 3, characterized in that: The step of constructing a kidney image automatic annotation and segmentation network using the annotated group of images specifically includes: The labeled group images are input as input data and target data into the encoder for feature extraction, and finally the kidney image automatic labeling and segmentation network is formed.

5. The method for renal nuclear magnetic resonance sequence conversion and image segmentation according to claim 4, characterized in that: In the step of acquiring a kidney image and dividing the acquired kidney image into a labeled group image and an unlabeled group image: the kidney image is acquired as a kidney nuclear magnetic resonance image, and the kidney nuclear magnetic resonance image includes a T1 sequence, a T2 sequence, a T1c sequence, a T2c sequence, a T2FS sequence and a DWI sequence.

6. The method for renal nuclear magnetic resonance sequence conversion and image segmentation according to claim 5, characterized in that: The step of placing the unlabeled group of images as input data and target data into the generator network for image-level domain adaptive training to finally form the kidney image sequence conversion network specifically includes: The T1c sequence and T1 sequence, T2 sequence, T2c sequence, T2FS sequence and DWI sequence of all unlabeled group images are respectively put into the generator network as input data and target data for image-level domain adaptive training, and finally the kidney image sequence conversion network that can generate other sequences through the T1c sequence is constructed.

7. The method for renal nuclear magnetic resonance sequence conversion and image segmentation according to claim 5, characterized in that: The step of inputting the labeled group images as input data and target data into the encoder for feature extraction, and finally forming the kidney image automatic labeling and segmentation network specifically includes: The pre-annotation sequence and the annotated sequence of the annotated group of images are respectively input into the encoder as input data and target data for feature extraction. In the encoder, these data are collaboratively constrained by the segmenter and the reconstructor, and finally form the automatic annotation and segmentation network of kidney images that can automatically annotate and segment unlabeled sequences.

8. The method for renal nuclear magnetic resonance sequence conversion and image segmentation according to claim 5, characterized in that: The step of inputting the unlabeled group of images into the kidney image sequence conversion and image segmentation unit to obtain labeled and segmented kidney images of each sequence specifically includes: The T1c sequence in the unlabeled group of images is input into the kidney image sequence conversion and image segmentation unit for processing to obtain labeled and segmented kidney images of each sequence.

9. The method for renal nuclear magnetic resonance sequence conversion and image segmentation according to claim 8, characterized in that: In the step of inputting the T1c sequence in the unlabeled group of images into the kidney image sequence conversion and image segmentation unit for processing to obtain labeled and segmented kidney images of each sequence: The kidney image sequence conversion network of the kidney image sequence conversion and image segmentation unit is used to convert the T1c sequence in the unlabeled group of images into a T1 sequence, a T2 sequence, a T2c sequence, a T2FS sequence and a DWI sequence; The renal image automatic annotation and segmentation network of the renal image sequence conversion and image segmentation unit is used to automatically annotate and segment the T1c sequence in the unannotated group of images and the T1 sequence, T2 sequence, T2c sequence, T2FS sequence and DWI sequence generated by the renal image sequence conversion network, and finally the annotated and segmented renal images of each sequence are obtained.

10. A kidney nuclear magnetic resonance sequence conversion and image segmentation system, characterized in that: include: An acquisition module is used to acquire kidney images and divide the acquired kidney images into a labeled group of images and an unlabeled group of images: A conversion network construction module, used to construct a kidney image sequence conversion network using unlabeled group images; A segmentation network construction module is used to construct a kidney image automatic annotation and segmentation network using an annotated group of images; A combination module, used for combining the kidney image sequence conversion network and the kidney image automatic annotation and segmentation network to construct a kidney image sequence conversion and image segmentation unit; The output module is used to input the unlabeled group of images into the kidney image sequence conversion and image segmentation unit to obtain labeled and segmented kidney images of each sequence.

Images