Model training method, image reconstruction method and related equipment
By reconstructing CTA images from CTP images using a pre-trained neural network model, the problem of CTA detection dependence in one-stop CT examination in stroke is solved, and the effect of shortening examination time, reducing radiation and contrast agent use is achieved.
Patent Information
- Application Number
- CN202311298638.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-09
- Publication Date
- 2025-04-11
AI Technical Summary
In the prior art, when performing one-stop CT examination for stroke, CTA examination is required first, waiting for the patient to discharge the contrast agent, and then conducting the CTP examination, resulting in a prolonged examination time and the patient with multiple radiation and contrast agent injections.
The pre-trained target neural network model is used to reconstruct high-quality CTA images from the CTP images, thereby reducing dependence on CTA inspections.
By reducing the number of CTA examinations, shortening the examination time, reducing the patient's radiation exposure and the number of contrast agent injections, improving the examination efficiency and safety.
Smart Images
Figure CN119810221A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of medical image processing, and specifically to a model training method, an image reconstruction method, and related equipment. Background Art
[0002] Computed Tomography Angiography (CTA) is a computerized tomography technique that can be used to image blood vessels throughout the human body, that is, to visualize arteries and veins. By injecting contrast agents into blood vessels and creating images, it is possible to find aneurysms (blood vessels that have enlarged and may be in danger of rupture), find blood vessels that have narrowed due to atherosclerosis (fatty substances that form plaques on the walls of arteries), find abnormal blood vessel structures in the brain, identify blood vessels damaged by injuries, find blood clots that may form in leg veins and enter the lungs, and evaluate tumors supplied by blood vessels. CTA can be used to show blood vessels in the heart, aorta and other large blood vessels, lungs, kidneys, head and neck, arms and legs, and can also be used to locate arterial or venous bleeding in the gastrointestinal system. Information from CTA can help prevent strokes or heart attacks.
[0003] Computed Tomography Perfusion (CTP) is a new imaging modality that can assess the microcirculation of the brain. CTP is different from traditional CT imaging that assesses brain structure and morphology, and it collects physiological information about the state of brain perfusion. CTP imaging can significantly impact the initial assessment of patients with acute cerebral infarction by helping to identify the ischemic penumbra. On the other hand, diffusion magnetic resonance imaging can identify dead brain tissue, while CTP imaging can help identify dying brain tissue, which, once identified, can be restored with timely and appropriate treatment. Several studies have demonstrated the potential role and development of CT perfusion imaging in the assessment and tracking of intracranial and extracranial stenotic occlusive diseases, the assessment of vasospasm after subarachnoid hemorrhage, the differentiation of tumors from infection, and the confirmation of brain death.
[0004] Brain CTA images can assist doctors in diagnosing brain disease information, and brain CTP images can reflect the blood flow velocity and blood supply of brain tissue. Therefore, brain CTA and brain CTP are currently important means of examining stroke-related diseases. For patients with acute ischemic cerebrovascular disease with occlusion of the anterior circulation large arteries, if the time window (6 to 24 hours) is exceeded or the onset time is unknown, CTP imaging can be used to evaluate the core infarct area and ischemic penumbra, helping to screen patients suitable for intra-arterial treatment. Therefore, in a one-stop CT examination for stroke, CTP examination should be performed on the basis of plain scan and CTA, that is, plain scan first, then CTA, and finally CTP. However, if CTA is performed first to find abnormal blood vessels, then after the CTA is completed, it is necessary to wait for the patient to excrete the contrast agent, and then inject new contrast agent for CTP examination. This will prolong the examination time for the treatment of stroke, and will cause the patient to endure multiple examinations of radiation and multiple contrast injections. Summary of the invention
[0005] The embodiments of the present application provide a model training method, an image reconstruction method and related equipment for reconstructing a CTA image with high-quality vascular imaging effect from a CTP image using a pre-trained target neural network model.
[0006] A first aspect of an embodiment of the present application provides an image reconstruction method, the method comprising:
[0007] Acquire the target CTP image to be processed;
[0008] Acquire a pre-trained target neural network model, wherein the target neural network model is obtained by training multiple groups of training samples using a machine learning algorithm, and each group of training samples includes paired CTP images and CTA images, and the paired CTP images and CTA images correspond to the same selected layer of the target tissue;
[0009] The target CTP image is input into the target neural network model so that the target neural network model reconstructs the target CTA image corresponding to the target CTP image according to the target CTP image, and obtains the target CTA image output by the target neural network model.
[0010] A second aspect of the embodiments of the present application provides a model training method, the method comprising:
[0011] Acquire multiple groups of training samples, each group of the training samples includes paired CTP images and CTA images, and the paired CTP images and CTA images correspond to the same selected layer of the target tissue;
[0012] Get the initial neural network model;
[0013] Multiple groups of the training samples are input into the initial neural network model so that the initial neural network model generates a predicted CTA image based on the CTP image in the training samples. When the relationship between the predicted CTA image and the CTA image in the training samples meets the convergence condition, the training is stopped to obtain a target neural network model.
[0014] A third aspect of an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.
[0015] A fourth aspect of an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method of the first aspect.
[0016] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:
[0017] The target CTA image corresponding to the target CTP image is reconstructed using the pre-trained target neural network model. The vascular imaging effect of the target CTA image can be close to or even better than the real CTA image, providing a true and accurate reference for the examination of various tissues and organs. Therefore, for brain examinations, patients can undergo one less CTA examination, which speeds up the one-stop CT examination for stroke, avoids the radiation of an additional CTA examination, and reduces the injection of contrast agent, thereby reducing the risk of side effects of contrast agent. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A flow chart of the model training method in the embodiment of the present application;
[0019] Figure 2 This is another flowchart of the image reconstruction method in the embodiment of the present application;
[0020] Figure 3 It is a structural schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION
[0021] The embodiments of the present application provide a model training method, an image reconstruction method and related equipment for reconstructing a CTA image with high-quality vascular imaging effect from a CTP image using a pre-trained target neural network model.
[0022] The model training method in the embodiment of the present application is described below:
[0023] See also Figure 1In the embodiment of the present application, one embodiment of the model training method includes:
[0024] 101. Acquire multiple groups of training samples, each group of the training samples includes paired CTP images and CTA images, and the paired CTP images and CTA images correspond to the same selected layer of the target tissue;
[0025] In this embodiment, the target tissue can be any organ tissue of the human body or animal body, such as the brain, heart, aorta and other large blood vessels, lungs, kidneys and other organ tissues. Taking the brain as an example, the same patient can be subjected to brain CTP and brain CTA medical examinations, and CTP images and CTA images of the same brain structure can be collected, that is, CTP examinations are performed on n selected layers of the brain to obtain CTP images of each selected layer; and CTA examinations are performed on m selected layers of the brain to obtain CTA images of each selected layer, where n and m are both positive integers greater than 1 and m>n.
[0026] Wherein, when obtaining CTP images, n×t images obtained by performing t scans on n selected layers of the target tissue respectively can be obtained, where n and t are both positive integers greater than 1; for each selected layer, t images obtained by performing t scans on the layer in the time sequence of the t scans performed on the layer are superimposed to obtain a CTP image with t timing channels corresponding to the layer; CTA images of m layers of the target tissue are obtained, where the m layers include the n layers; for each layer in the n layers, a CTA image that is paired with the CTP image of the layer is determined in the CTA images of the m layers, and the CTP images and CTA images that are paired on the layer constitute a group of training samples.
[0027] The CTA image that is paired with the CTP image at the m levels is determined from the CTA images at the m levels, and the pairing between the CTP image and the CTA image can be determined manually. The personnel can select the CTA image that is morphologically close or identical to the level corresponding to the CTP image from the CTA images at the m levels, and pair it with the CTP image.
[0028] For example, assuming that the n levels are the three levels A, B, and C of the target tissue, the CTP images of the three levels can be obtained respectively; and the m levels are the five levels A, B, C, D, and E of the target tissue, the CTA images of the five levels can be obtained respectively. For the A level, the CTA image that is similar to or identical to the A level in morphology and structure can be screened out from the CTA images of the five levels A, B, C, D, and E, and then the CTA image is paired with the CTP image of the A level; similarly, for the B level, the CTA image that is similar to or identical to the B level in morphology and structure can be screened out from the CTA images of the five levels A, B, C, D, and E, and then the CTA image is paired with the CTP image of the B level; similarly for the C level, a CTA image paired with the CTP image of the C level can be obtained.
[0029] For example, for a patient, the n selected brain layers are scanned continuously t times, that is, t scanning time points are generated, and n images are generated at each scanning time point. For each layer, the images of t scanning time points of the layer are selected, and the images of these t scanning time points are superimposed in the time sequence of t scans, and finally n CTP images of t time series channels can be obtained, and each CTP image of t time series channel corresponds to a selected layer of the brain.
[0030] In each set of training samples, the paired CTP images and CTA images correspond to the same selected layer of the same brain structure.
[0031] The five dimensions of the CTP images in the training samples include batch size (bs), the number of temporal channels (t), the number of image channels, image length (h), and image width (w). For conventional color images, the number of image channels is 3, namely RGB (red, green, and blue) channels; for CTP images, they are usually single-channel grayscale images, so the number of image channels is 1.
[0032] 102. Obtain an initial neural network model;
[0033] In this embodiment, the initial neural network model includes a backbone neural network and a time series information fusion module. The backbone neural network can be any neural network, such as a convolutional neural network (CNN), a feedforward neural network, a recursive neural network (LSTM / GRU / attention), a generative adversarial network (GAN) and other neural networks. The time series information fusion module can be any functional module that can perform image dimensionality reduction operations. For example, dimensionality reduction is to directly eliminate the dimension on the image channel. The specific operation is that because the present embodiment uses grayscale images for model training, the value on the dimension of the image channel is 1, that is, there is only one image channel. For example, the dimension array of the image is five-dimensional [bs, t, 1, h, w], and the feature map on the image channel is directly taken to become a four-dimensional [bs, t, h, w]. In other words, the present embodiment can directly perform dimensionality reduction operations on the image channel, that is, image dimensionality reduction to remove the dimension of the image channel.
[0034] Among them, the backbone neural network is connected to the time series information fusion module, and the time series information fusion module includes the above-mentioned functional module capable of performing image dimensionality reduction operations and the convolution layer, thereby forming the initial neural network model.
[0035] 103. Inputting the plurality of groups of training samples into the initial neural network model, so that the initial neural network model generates predicted CTA images according to the CTP images in the training samples, and stopping the training when the relationship between the predicted CTA images and the CTA images in the training samples meets a convergence condition, so as to obtain a target neural network model;
[0036] In this embodiment, the five dimensions of the CTP image in the training sample include the number t of timing channels, where t is a positive integer greater than 1. The timing channels correspond to the scanning time points of the CTP image, that is, if there are t scanning time points, the obtained CTP image has t timing channels.
[0037] When the initial neural network model generates a predicted CTA image based on the CTP image in the training sample, the backbone neural network is used to process the four-dimensional data on each time series channel of the CTP image in the training sample to obtain t first four-dimensional feature maps, and the t first four-dimensional feature maps are stacked based on the sequence of the time series channels of the t first four-dimensional feature maps to obtain a five-dimensional feature map; the time series information fusion module is used to perform a dimensionality reduction operation on the image channel of the five-dimensional feature map to obtain a second four-dimensional feature map; the convolution layer in the time series information fusion module is used to perform a convolution operation on the second four-dimensional feature map, which is equivalent to compressing the dimension of the number of time series channels in the second four-dimensional feature map. For example, the dimensions of the second four-dimensional feature map are [bs, t, h, w], where bs refers to batch size, t refers to the number of time series channels, h refers to image length, and w refers to image width. After the convolution operation, the dimension of the number of time series channels is compressed to 1, and the dimension of the predicted CTA image is finally obtained as [bs, 1, h, w]. Furthermore, a loss function can be calculated based on the predicted CTA image and the CTA image in the training sample, and the training of the initial neural network model is stopped when the loss function meets the convergence condition.
[0038] Among them, concatenation means connecting multiple images together at the channel level. For example, there are 3 single-channel images with a length and width of 512, that is, the size of each image is (1, 512, 512). After concatenating these 3 single-channel images, a three-channel image with a size of (3, 512, 512) can be obtained.
[0039] For example, five selected layers of the brain (assuming they are numbered from layer 1 to layer 5) are scanned three times respectively, that is, there are three scanning time points (point A, point B, point C), each scanning time point generates five images, and three images are obtained for each layer, that is, a total of 15 images are obtained. If they are numbered A1 to A5, B1 to B5, and C1 to C5, the three images A1B1C1 of layer 1 can be superimposed, the three images A2B2C2 of layer 2 can be superimposed, the three images A3B3C3 of layer 3 can be superimposed, the three images A4B4C4 of layer 4 can be superimposed, and the three images A5B5C5 of layer 5 can be superimposed, and finally a three-channel image with 3 timing channels corresponding to each layer can be obtained.
[0040] Among them, the loss function satisfies the convergence condition, which can be that the fluctuation range of the loss function value is less than a preset numerical range, or the loss function value is less than a preset threshold, which is not limited here.
[0041] Therefore, by training the neural network model through the above-mentioned model training steps, the model can acquire the ability to reconstruct CTA images from CTP images. Moreover, since the CTP image and the reconstructed CTA image are both aimed at the same level of the target tissue, the reconstructed CTA image can be close to the real CTA image in vascular imaging, or even better than the real CTA image, providing a real and accurate reference basis for the examination of various tissues and organs.
[0042] In the following, Figure 1 Based on the embodiment shown, the embodiment of the present application is further described in detail. Figure 2 In the embodiment of the present application, an image reconstruction method includes:
[0043] 201. Obtain a target CTP image to be processed;
[0044] The target CTP image may be a real CTP image obtained by performing a CTP scan on the target tissue of the subject, that is, a CTP scan is performed on a selected layer of the target tissue, and a CTP image is generated according to the scanning result.
[0045] 202. Obtain a pre-trained target neural network model, wherein the target neural network model is obtained by training multiple groups of training samples using a machine learning algorithm, and each group of training samples includes paired CTP images and CTA images, and the paired CTP images and CTA images correspond to the same selected layer of the target tissue;
[0046] In this embodiment, the target neural network model can be based on Figure 1 The target neural network model obtained by training the model training method of the illustrated embodiment. Wherein, the target tissue includes the brain, and the target CTP image is a brain CTP image obtained by performing a CTP examination on the brain. The machine learning algorithm may be a deep learning algorithm.
[0047] 203. Input the target CTP image into the target neural network model, so that the target neural network model reconstructs a target CTA image corresponding to the target CTP image according to the target CTP image, and obtains the target CTA image output by the target neural network model;
[0048] When the target CTP image is obtained, the target CTP image can be input into the target neural network model, so that the target neural network model processes the target CTP image based on the ability to reconstruct the CTA image from the CTP image obtained during the model training period, and then outputs the target CTA image corresponding to the reconstructed target CTP image. Since the target CTA image is reconstructed based on the target CTP image, the target CTA image and the target CTP image correspond to the same level of the target tissue, and thus the vascular imaging effect of the target CTA image can be close to or even better than the real CTA image, providing a real and accurate reference basis for the examination of various tissues and organs.
[0049] In this embodiment, a pre-trained target neural network model is used to reconstruct the target CTP image into its corresponding target CTA image. The vascular imaging effect of the target CTA image can be close to or even better than the real CTA image, providing a real and accurate reference basis for the examination of various tissues and organs. Therefore, for brain examination, the patient can undergo one less CTA examination, which speeds up the one-stop CT examination of stroke and avoids the radiation of one more CTA examination. At the same time, it reduces the injection of contrast agent and reduces the risk of side effects of contrast agent.
[0050] The computer device in the embodiment of the present application is described below. Figure 3 In the embodiment of the present application, one embodiment of the computer device includes:
[0051] The computer device 300 may include one or more central processing units (CPU) 301 and a memory 305 , wherein the memory 305 stores one or more application programs or data.
[0052] The memory 305 may be a volatile storage or a persistent storage. The program stored in the memory 305 may include one or more modules, each of which may include a series of instruction operations in the computer device. Furthermore, the central processor 301 may be configured to communicate with the memory 305 and execute a series of instruction operations in the memory 305 on the computer device 300.
[0053] The computer device 300 may also include one or more power supplies 302, one or more wired or wireless network interfaces 303, one or more input and output interfaces 304, and / or one or more operating systems, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0054] The CPU 301 can execute the aforementioned Figure 1 to Figure 2 The operations performed by the computer device in the illustrated embodiment will not be described in detail here.
[0055] The present application also provides a computer storage medium, wherein one embodiment includes: the computer storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the aforementioned Figure 1 to Figure 2 The operations performed by the computer device in the illustrated embodiment.
[0056] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0057] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0058] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0059] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0060] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.
Claims
1. An image reconstruction method, characterized in that: The method comprises: Acquire the target CTP image to be processed; Acquire a pre-trained target neural network model, wherein the target neural network model is obtained by training multiple groups of training samples using a machine learning algorithm, and each group of training samples includes paired CTP images and CTA images, and the paired CTP images and CTA images correspond to the same selected layer of the target tissue; The target CTP image is input into the target neural network model so that the target neural network model reconstructs the target CTA image corresponding to the target CTP image according to the target CTP image, and obtains the target CTA image output by the target neural network model.
2. The method according to claim 1, characterized in that: The target tissue includes the brain, and the target CTP image is a brain CTP image obtained by performing a CTP examination on the brain.
3. A model training method, characterized in that: The method comprises: Acquire multiple groups of training samples, each group of the training samples includes paired CTP images and CTA images, and the paired CTP images and CTA images correspond to the same selected layer of the target tissue; Get the initial neural network model; Multiple groups of the training samples are input into the initial neural network model so that the initial neural network model generates a predicted CTA image based on the CTP image in the training samples. When the relationship between the predicted CTA image and the CTA image in the training samples meets the convergence condition, the training is stopped to obtain a target neural network model.
4. The method according to claim 3, characterized in that: The five dimensions of the CTP image in the training sample include the number of time series channels t, where t is a positive integer greater than 1; The initial neural network model includes a backbone neural network and a time series information fusion module; When the initial neural network model generates the predicted CTA image according to the CTP image in the training sample, the backbone neural network is used to process the four-dimensional data on each time series channel of the CTP image in the training sample to obtain t first four-dimensional feature maps, and to stack the t first four-dimensional feature maps based on the sequence of the time series channels of the t first four-dimensional feature maps to obtain a five-dimensional feature map; The temporal information fusion module is used to perform a dimensionality reduction operation on the image channel of the five-dimensional feature map to obtain a second four-dimensional feature map, and the convolution layer in the temporal information fusion module is used to perform a convolution operation on the second four-dimensional feature map to obtain the predicted CTA image.
5. The method according to claim 4, characterized in that The step of stopping the training when the relationship between the predicted CTA image and the CTA image in the training sample satisfies a convergence condition comprises: A loss function is calculated according to the predicted CTA image and the CTA image in the training sample, and the training of the initial neural network model is stopped when the loss function meets a convergence condition.
6. The method according to claim 4, characterized in that The five dimensions of the CTP image in the training sample include batch size, the number of the timing channels, the number of image channels, image length and image width.
7. The method according to claim 3, characterized in that The obtaining of multiple sets of training samples comprises: Obtain n×t images by scanning n selected layers of the target tissue t times, where n and t are both positive integers greater than 1; For each of the selected layers, t images obtained by performing the t scans of the layer in the time sequence of the t scans of the layer are superimposed to obtain a CTP image with t sequential channels corresponding to the layer; Acquire CTA images of m layers of the target tissue, where the m layers include the n layer, and m is a positive integer greater than n; For each of the n layers, a CTA image paired with the CTP image of the layer is determined from the CTA images of the m layers, and the paired CTP images and CTA images at the layer constitute a group of the training samples.
8. The method according to claim 3, characterized in that The paired CTP image and CTA image correspond to the same selected layer of the same brain structure.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.
10. A computer storage medium, characterized in that: The computer storage medium stores instructions, and when the instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 8.