Perfusion imaging system and method

Abstract

A perfusion imaging system is used for perfusion imaging, comprising a data acquisition module for acquiring a perfusion image data set; a preprocessing module for preprocessing the perfusion image data set; and a lesion segmentation module for performing lesion segmentation based on the preprocessed perfusion image data set, comprising a cascaded first neural network and a second neural network, the first neural network generating a perfusion parameter, and the second neural network performing lesion segmentation according to the perfusion parameter. The present invention can improve the stability of perfusion parameter generation and the accuracy of lesion segmentation, and can be applied to perfusion imaging of the brain or other anatomical structures.

Description

Technical Field

[0001] The present invention relates to medical imaging, and more particularly to systems and methods for perfusion imaging. Background Art

[0002] In medical imaging, perfusion imaging provides a functional examination method. Based on the principle that blood flow changes are closely related to the normal physiological functions of tissues and organs and their pathological changes, it evaluates the blood perfusion state of tissue microcirculation, such as evaluating pathological changes or physiological changes such as cerebral infarction and tumors, thereby facilitating the diagnosis and treatment of diseases. At present, the commonly used medical imaging modes of perfusion imaging include computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), etc. Taking CT perfusion imaging as an example, when the contrast agent is rapidly injected intravenously, the region of interest is scanned continuously and multiple times to obtain the time-density curve of the region of interest, and various perfusion parameter values ​​are calculated using different mathematical models, and then the perfusion images are generated using the perfusion parameters.

[0003] Perfusion imaging can be used for multiple organs and tissues throughout the body, such as the brain, liver, heart, pancreas, and bones. For example, for the brain, stroke is a common cerebrovascular disease, and brain perfusion CT imaging is often used as a functional imaging method to understand the blood perfusion status of brain tissue, and is now included in the routine examination method for diagnosing ischemic stroke. Brain CT perfusion imaging can obtain a series of perfusion parameters such as cerebral blood volume (CBV), cerebral blood flow (CBF), mean transit time (MTT), peak time (Tmax), etc., and then generate brain perfusion images to evaluate the perfusion status of brain tissue.

[0004] In the cerebral perfusion imaging method provided in the prior art, the cerebral perfusion parameters are affected by many factors such as the arterial input function (AIF) and the model calculation method, and the repeatability is poor. When determining the cerebral infarction area in the parameter image, medical personnel need to make manual judgments based on their experience, which depends on the experience of medical personnel, has subjective differences, is time-consuming, has low accuracy, and has poor operability.

[0005] Therefore, a new perfusion imaging system and method is needed that can reduce the subjective differences introduced by medical personnel's judgment based on experience, improve the accuracy of lesion judgment, and reduce the lesion segmentation time. Summary of the invention

[0006] In one aspect of the present invention, a perfusion imaging system is provided for perfusion imaging, comprising: a data acquisition module for acquiring a perfusion image data set; a preprocessing module for preprocessing the perfusion image data set; a lesion segmentation module for performing lesion segmentation based on the preprocessed perfusion image data set, the lesion segmentation module comprising a cascaded first neural network and a second neural network, the first neural network generating a perfusion parameter, and the second neural network performing lesion segmentation according to the perfusion parameter.

[0007] In one aspect of the present invention, the first neural network comprises a U-net neural network.

[0008] In one aspect of the present invention, the first neural network includes an encoding path having four groups of convolutional layers, wherein the first two groups of convolutional layers include convolutional layers with 1x1 convolutional kernels.

[0009] In one aspect of the present invention, the second neural network comprises a Dense V-net neural network.

[0010] In one aspect of the present invention, the second neural network includes a downsampling path having three stages of convolutional layers, wherein the input of the first stage convolutional layer includes a concatenation of a downsampled feature map of the perfusion parameters generated by the first neural network and a downsampled feature map of the output of the first group of convolutional layers of the first neural network.

[0011] In one aspect of the present invention, the inputs of the second-stage convolutional layer and the third-stage convolutional layer of the downsampling path of the second neural network respectively include a concatenation of a feature map formed by downsampling the output of the previous stage convolutional layer and a feature map formed by downsampling the output of a group of convolutional layers corresponding to the first neural network.

[0012] In one aspect of the present invention, the preprocessing module includes a normalization module that calculates a mean and a variance of the perfusion image dataset, and subtracts the mean from each pixel value in the perfusion image dataset and divides the result by the variance.

[0013] In one aspect of the present invention, the pre-processing module includes a data selection module for selecting perfusion image data of a specific period from the perfusion image data set.

[0014] In one aspect of the present invention, the perfusion image data set includes a brain perfusion image data set, and the preprocessing module further includes a skull image information removal module for removing skull part data in the brain perfusion image data set.

[0015] In one aspect of the present invention, the perfusion image dataset includes a cerebral perfusion image dataset, and the perfusion parameter includes at least one of cerebral blood volume, cerebral blood flow, mean contrast agent transit time, and contrast agent peak time.

[0016] In another aspect of the present invention, a perfusion imaging method is provided, comprising: acquiring a perfusion image data set; preprocessing the perfusion image data set; and lesion segmentation, comprising inputting the preprocessed perfusion image data set into a cascaded first neural network and a second neural network, the first neural network generating perfusion parameters, and the second neural network performing lesion segmentation based on the perfusion parameters.

[0017] In one aspect of the present invention, the first neural network comprises a U-net neural network.

[0018] In one aspect of the present invention, the first neural network includes an encoding path having four groups of convolutional layers, wherein the first two groups of convolutional layers include convolutional layers with 1x1 convolutional kernels.

[0019] In one aspect of the present invention, the second neural network comprises a Dense V-net neural network.

[0020] In one aspect of the present invention, the second neural network includes a downsampling path having three stages of convolutional layers, wherein the input of the first stage convolutional layer includes the perfusion parameters generated by the first neural network and the feature map output of the output of the first group of convolutional layers of the first neural network after downsampling.

[0021] In one aspect of the present invention, the inputs of the second-stage convolutional layer and the third-stage convolutional layer of the downsampling path of the second neural network respectively include a concatenation of a feature map formed by downsampling the output of the previous stage neural network and a feature map formed by downsampling the output of a group of convolutional layers corresponding to the first neural network.

[0022] In one aspect of the present invention, the perfusion image data set includes a brain perfusion image data set, and the preprocessing includes removing skull image information, which is used to remove skull part data in the brain perfusion image data set.

[0023] In one aspect of the present invention, the perfusion image dataset includes a cerebral perfusion image dataset, and the perfusion parameter includes at least one of cerebral blood volume, cerebral blood flow, mean contrast agent transit time, and contrast agent peak time.

[0024] In another aspect of the present invention, a system is provided, comprising a processor for executing the aforementioned perfusion imaging method.

[0025] In another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned perfusion imaging method is implemented.

[0026] In the present invention, a cascaded first neural network and a second neural network are used, the first neural network generates perfusion parameters, and the second neural network performs lesion segmentation based on the perfusion parameters. It can automatically generate perfusion parameters and lesion segmentation, reduce the subjective differences introduced by medical personnel's judgment based on experience, improve the accuracy of lesion segmentation judgment and reduce the time of lesion segmentation.

[0027] It should be understood that the above brief description is provided to introduce the technical solutions further described in the specific embodiments in a simplified form. This does not mean that the above brief description limits the key or essential features claimed in the present invention, and the scope of protection claimed in the present invention is uniquely defined by the claims. In addition, the claimed invention is not limited to the implementation of solving any disadvantages mentioned above or in any section of this disclosure.

[0028] These and other features and aspects of the present invention will become more apparent from the following detailed description taken with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to be able to understand the present invention in detail, a more specific description of the present invention briefly summarized above can be obtained by reference to the embodiments, some of which are shown in the accompanying drawings. In order to facilitate understanding, the same reference numerals have been used as much as possible to indicate the same elements common to the various figures. However, it should be noted that the drawings only show typical embodiments of the present invention and therefore should not be considered to limit the scope of the present invention, as the present invention may allow other equivalent embodiments. In the accompanying drawings:

[0030] Figure 1 A CT imaging system according to an embodiment of the present invention is schematically shown.

[0031] Figure 2 A block diagram of a CT imaging system according to an embodiment of the present invention is schematically shown.

[0032] Figure 3 A block diagram of a perfusion imaging system according to an embodiment of the present invention is schematically shown.

[0033] Figure 4 The figure schematically shows a block diagram of a preprocessing module of a perfusion imaging system according to an embodiment of the present invention.

[0034] Figure 5 The neural network structure diagram of the lesion segmentation module of the perfusion imaging system according to an embodiment of the present invention is schematically shown.

[0035] Figure 6 The flowchart of the perfusion imaging method according to the embodiment of the present invention is schematically shown.

[0036] Figure 7The flowchart of the preprocessing of the perfusion imaging method according to the embodiment of the present invention is schematically shown.

[0037] Figure 8 An example of an electronic device for executing a perfusion imaging method according to an embodiment of the present invention is schematically shown.

[0038] Fig. 9 An example of a perfusion imaging system for executing a perfusion imaging method according to an embodiment of the present invention is schematically shown.

[0039] It is contemplated that elements of one embodiment of the invention may be beneficially applied to other embodiments without further recitation. DETAILED DESCRIPTION

[0040] The specific embodiments of the present invention will be described below. It should be noted that in the specific description of these embodiments, in order to be concise and clear, this specification cannot provide a detailed description of all the features of the actual embodiments. It should be understood that in the actual implementation of any embodiment, just as in the process of any engineering project or design project, in order to achieve the specific goals of the developer and to meet system-related or business-related restrictions, various specific decisions are often made, and this will also change from one embodiment to another. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the content disclosed by the present invention, some changes such as design, manufacturing or production based on the technical content disclosed by the present invention are just conventional technical means, and should not be understood as the content of the present invention is insufficient.

[0041] Unless otherwise defined, the technical or scientific terms used in the claims and the specification shall have the usual meaning understood by persons with ordinary skills in the technical field to which the invention belongs. The words "first", "second" and similar words used in the patent application specification and the claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "One" or "one" and other similar words do not indicate a quantitative limitation, but indicate the existence of at least one. "Include" or "comprises" and other similar words mean that the elements or objects appearing before "include" or "comprises" include the elements or objects listed after "include" or "comprises" and their equivalent elements, and do not exclude other elements or objects. "Connected" or "connected" and other similar words are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0042] The perfusion imaging system and method described herein can be applied to various medical imaging modalities, including but not limited to computed tomography (CT) equipment, magnetic resonance imaging (MRI) equipment, C-arm imaging equipment, positron emission tomography (PET) equipment, single photon emission computed tomography (SPECT) equipment, or any other suitable medical imaging equipment. The perfusion imaging system may include the aforementioned medical imaging equipment, and may also include a separate computer device connected to the medical imaging equipment, and may also include a computer device connected to the Internet cloud, which is connected to the medical imaging equipment or a storage device storing medical images via the Internet. The perfusion imaging method can be implemented independently or jointly by the aforementioned medical imaging equipment, the computer device connected to the medical imaging equipment, and the computer device connected to the Internet cloud.

[0043] Exemplarily, the present invention is described below in conjunction with an X-ray computed tomography (CT) device. Those skilled in the art will appreciate that the present invention may also be applicable to other medical imaging devices suitable for perfusion imaging.

[0044] Figure 1 A CT imaging apparatus 100 is shown to which a perfusion imaging system and method according to an exemplary embodiment of the present invention is applicable. Figure 2 yes Figure 1 A schematic block diagram of an example CT imaging system 100 is shown.

[0045] See also Figure 1 , the CT imaging system 100 is shown to include a scanning gantry 11. The scanning gantry 11 has an X-ray source 11a that projects an X-ray beam toward a detector assembly or collimator 12 on an opposite side of the scanning gantry 11.

[0046] See also Figure 2 The detector assembly 12 includes a plurality of detector units 12 a and a data acquisition system (DAS) 12 b. The plurality of detector units 12 a sense the projected X-rays 16 passing through the object 10.

[0047] The DAS 12b converts the collected information into projection data for subsequent processing based on the sensing of the detector unit 12a. During the scanning period for collecting X-ray projection data, the scanning gantry 11 and the components mounted thereon rotate around the rotation center 11c.

[0048] The rotation of the scanning gantry 11 and the operation of the X-ray source 11a are controlled by the control mechanism 13 of the CT system 100. The control mechanism 13 includes an X-ray controller 13a that provides power and timing signals to the X-ray source 11a, and a scanning gantry motor controller 13b that controls the rotation speed and position of the scanning gantry 11. The image reconstruction device 14 receives the projection data from the DAS 12b and performs image reconstruction. The reconstructed image is transmitted as input to the computer 15, and the computer 15 stores the image in the mass storage device 16.

[0049] The computer 15 also receives commands and scanning parameters from an operator through a console 17, which has some form of operator interface, such as a keyboard, mouse, voice activated controller, or any other suitable input device. An associated display 18 allows the operator to observe reconstructed images and other data from the computer 15. The commands and parameters provided by the operator are used by the computer 15 to provide control signals and information to the DAS 12b, the x-ray controller 13a, and the scanning gantry motor controller 13b. In addition, the computer 15 operates a patient table motor controller 19a, which controls the patient table 19 to position the subject 10 and the scanning gantry 11. In particular, the patient table 19 moves the subject 10, in whole or in part, through the scanning gantry 11. Figure 1 The scanning frame opening 11d.

[0050] Figure 3 An example block diagram of a perfusion imaging system 200 according to an embodiment of the present invention is shown. The perfusion imaging system 200 includes a data acquisition module 21, a preprocessing module 22, and a lesion segmentation module 23. The acquisition module 21 is used to acquire a perfusion image data set. The preprocessing module 201 is used to preprocess the acquired perfusion image data set. The lesion segmentation module 23 is used to perform lesion segmentation based on the preprocessed perfusion image data set, and includes a cascaded first neural network module 300 and a second neural network module 400, the first neural network module 300 is used to generate perfusion parameters, and the second neural network module 400 is used to segment lesions according to the perfusion parameters.

[0051] The perfusion imaging system 100 and method 600 described in the present invention can be applied to various anatomical structures of various subjects, including but not limited to human patients and animals, and the anatomical structures are not limited to brain, liver, heart, lung, pancreas and bones. By way of example, the present invention is described below in conjunction with a brain CT perfusion imaging system. Those skilled in the art will appreciate that the present invention can also be applied to CT perfusion imaging of other anatomical structures, or other anatomical structures using other medical imaging devices for perfusion imaging.

[0052] The data acquisition module 201 acquires a perfusion image data set. For example, in brain CT perfusion imaging, the data acquisition module 201 acquires multi-phase CT perfusion image data. That is, after a rapid bolus injection of contrast agent into the patient's brain vein, a CT scan is performed on the patient to acquire CT perfusion image data at multiple time points at specific intervals. The CT perfusion image data form a CT perfusion image data set. Preferably, the CT image data set includes more than 10 phases of CT perfusion image data, such as 20 phases, 25 phases, 30 phases, 35 phases, and 40 phases. In CT perfusion imaging applied to other anatomical structures, multi-phase CT perfusion image data of the corresponding anatomical structure can also be acquired to form a CT perfusion image data set.

[0053] Figure 4 The preprocessing module 22 of the perfusion imaging system 200 according to an embodiment of the present invention is shown, and the preprocessing module 22 includes a data standardization module 22b, a data selection module 22c, and a label data generation module 22d. Optionally, in brain perfusion imaging, the preprocessing module 22 also includes a skull image information removal module 22a represented by a dotted box. The preprocessing module 22 preprocesses the acquired perfusion image data set so that the perfusion image data set meets certain requirements, so as to input the preprocessed perfusion image data set into the lesion segmentation module to obtain appropriate perfusion parameters and perform accurate lesion segmentation.

[0054] For example, in brain CT perfusion imaging, the preprocessing module 22 includes a skull image information elimination module 22a, which can eliminate the skull image information in the perfusion image set, and then in the subsequent image preprocessing and image segmentation, it can reduce the influence of the skull image information on the non-skull image information in the subsequent other preprocessing, perfusion parameter generation and lesion segmentation. For example, the skull image information elimination module 22a selects the clearest perfusion image data of the first phase in the aforementioned acquired CT perfusion image data set, and removes the skull image information of the perfusion image data of this phase by using a traditional method such as a threshold method or a template matching method, and at the same time uses the perfusion image data of this phase after the skull image information is removed as a mask, and then performs a dot multiplication operation on the perfusion image data of other phases in the perfusion image data set and the mask, thereby obtaining the perfusion image data set after the skull image information is eliminated. The selection of the clearest phase perfusion image data can be based on the CT value of each pixel in each phase of image data. For example, when the sum of the CT values ​​of all pixels in a certain phase of perfusion image data is the highest, or the sum of the CT values ​​of a certain proportion of pixels is the highest, or the average CT value of all pixels is the highest, the perfusion image data of that phase is selected as the clearest phase perfusion image data. It can be understood by those skilled in the art that the execution of the skull image information removal module 22a can be automatically executed by a preset program without human intervention. The skull image information removal module 22a is only used in brain CT perfusion imaging. When perfusion imaging is performed on other anatomical structures, the skull image information removal module 22a may not be needed, or a module with similar functions may be used to reduce the impact of unnecessary anatomical structure image information on other useful image information.

[0055] The data standardization module 22b is used to standardize the acquired perfusion image. During data standardization, each pixel value in the perfusion image data set is subtracted from the mean of the entire perfusion image data set, and then divided by the standard deviation of the entire perfusion image data set. Standardization is used to control the data distribution within the range of a mean of 0 and a standard deviation of 1, which is beneficial to accelerate the neural network training process and reduce the possibility of the neural network model falling into a local optimum. For example, in brain CT perfusion imaging, the data standardization module standardizes the perfusion image data set after the skull image information removal module 22a, specifically including calculating the mean and variance of the perfusion image data set, and then subtracting the mean from each pixel value and dividing it by the variance.

[0056] The data selection module 22c is used to select perfusion image data of a specific period. For example, as mentioned above, the brain CT perfusion image data set includes multi-period data, which may usually reach 30 to 40 periods. In order to reduce the amount of calculation of the lesion segmentation module 203, perfusion image data of a specific period can be selected. For example, in this embodiment, 10 periods of perfusion image data can be selected. Specifically, the inflow time and outflow time of the contrast agent in the arterial blood vessel can be determined by the arterial input function (AIF), and the specific time point between the inflow time and the outflow time, that is, the CT data corresponding to the specific period is selected. The inflow time refers to the initial point where the AIF curve starts to rise, and the outflow time refers to the time point when the AIF curve reaches the peak time point and reaches the first trough. If the period between the two moments is less than 10 periods, the perfusion image data is supplemented by zero matrix splicing, that is, the missing period is filled with a matrix with a value of 0, so that the overall perfusion image data can have 10 periods of data. If the period between the inflow and outflow times exceeds 10 periods, the data exceeding 10 periods are removed by truncation. The aforementioned selected 10-phase perfusion image data may form a 3D CT perfusion image data subset of, for example, 512*512*128.

[0057] The label data generation module 22d is used to generate label data for deep learning model training input into the lesion segmentation module 203. For example, in brain CT perfusion imaging, the label data includes a series of perfusion parameters such as cerebral blood volume (CBV), cerebral blood flow (CBF), mean contrast agent transit time (MTT), and contrast agent peak time (Tmax). The label data generation module 22d can calculate the label data using the traditional convolution method, maximum slope method, and deconvolution method for the aforementioned CT perfusion image data set. Due to the different levels of different cerebral perfusion parameters, the label data generation module 22d further processes the label data, specifically, obtains the maximum and minimum values ​​of each cerebral perfusion parameter in the entire data set, and then subtracts the minimum value from the label data, and then divides the label data by the difference between the maximum value and the minimum value, thereby limiting the label data to between 0-1, thereby generating label data for deep learning model training. In one embodiment, the label data may only include the contrast agent peak time (Tmax), which has a higher sensitivity in determining the ischemic penumbra area. In other embodiments, the label data may include a combination of contrast agent peak time (Tmax) and other cerebral perfusion parameters.

[0058] Figure 5 The lesion segmentation module 23 of the perfusion imaging system 200 according to an embodiment of the present invention is shown to be used for performing lesion segmentation based on preprocessed perfusion image data. The lesion segmentation module 23 includes a cascaded first neural network module 300 and a second neural network module 400.

[0059] like Figure 5As shown in the figure, the specific structure of the lesion segmentation module 23, the first neural network module 300 adopts the U-net network structure, and each dark rectangular box in the structural diagram corresponds to a multi-channel feature map (Feature Map), and the number above the dark rectangular box indicates the number of matrix channels; the blank rectangular box represents the copied feature map; the arrows represent different calculation operations. The U-net network structure includes a coding path (or search path, contracting path) 30a on the left and a decoding path (or expansive path) 30b on the right. The coding path 30a includes a first group of convolutional layers 31, a second group of convolutional layers 32, a third group of convolutional layers 33 and a fourth group of convolutional layers 34. The first group of convolutional layers 31 and the second group of convolutional layers 32 respectively include two convolutional layers with 1x1 convolutional kernels, and the third group of convolutional layers 33 and the fourth group of convolutional layers 34 respectively include two convolutional layers with 3x3 convolutional kernels. The output of each previous group of convolutional layers in the encoding path 30a is input to the next group of convolutional layers after downsampling. In this embodiment, the downsampling uses a pooling operation (max pool 2×2), which can use a maximum pooling operation or other suitable pooling operation methods. The decoding path 30b is symmetrically arranged with the encoding path 30a, and includes a fifth group of convolutional layers 35, a sixth group of convolutional layers 36, a seventh group of convolutional layers 37, and an eighth group of convolutional layers 38, wherein the seventh group of convolutional layers 37 and the eighth group of convolutional layers 38 respectively include convolutions of two 1x1 convolutional kernels, and the fifth group of convolutional layers 35 and the sixth group of convolutional layers 36 include convolutions of two 3x3 convolutional kernels. The output of each previous group of convolutional layers in the decoding path 30b is input to the next group of convolutional layers after upsampling, and the upsampling can use a linear interpolation operation or other suitable methods. The decoding path 30b is connected to the corresponding groups of convolution layers in the encoding path 30a through skip connections, that is, the input of each group of convolution layers in the decoding path 30b includes the feature map output of a group of convolution layers in the corresponding encoding path 30a and the feature map output of the previous group of convolution layers in the decoding path 30b after upsampling. The feature layer processed by the eighth group of convolution layers 38 in the decoding path 30b is input to the sigmoid mapping layer, and the predicted perfusion parameters are output. In the case of multiple perfusion parameters, multiple convolution branches can be set after the eighth group of convolution layers 38, and each convolution branch outputs a group of perfusion parameters respectively, thereby realizing the prediction of multiple perfusion parameters. In this embodiment, the first group of convolutional layers 31 and the second group of convolutional layers 32 of the encoding path 30a of the U-net network structure use 1x1 convolutional kernels, focusing on extracting the time series feature information of each pixel, and the third group of convolutional layers 33 and the fourth group of convolutional layers 34 use 3x3 convolutional kernels, focusing on extracting deeper semantic feature information. The feature information of different spatial dimensions improves the accuracy of the model prediction parameters.

[0060] like Figure 5 As shown, the second neural network module 400 adopts a Dense V-net structure, which includes a downsampling path (or compression path, left path) 40a and an upsampling path (or decompression path, right path) 40b. The downsampling branch 40a includes three stages of convolutional layers, that is, the downsampling branch 40a includes a first-stage convolutional layer 41, a second-stage convolutional layer 42, and a third-stage convolutional layer 43, and the convolutional layers of each stage use dense blocks (Dense Block) for convolution operations. The dense block includes several convolutional layers connected in series, and the input of each convolutional layer is formed by splicing together the feature maps output by all the convolutional layers before the layer. The output of the first-stage convolutional layer 41 is input to the second-stage convolutional layer 42 after downsampling, and the output of the second-stage convolutional layer 42 is input to the third-stage convolutional layer 43 after downsampling. In this embodiment, downsampling adopts a pooling operation (max pool 2×2), which can adopt a maximum pooling operation or other suitable pooling operation methods. The upsampling path 40b is symmetrical with the downsampling path 40a, and includes three stages of convolution layers, namely, the fourth stage convolution layer 44, the fifth stage convolution layer 45, and the sixth stage convolution layer 46. The output of the third stage convolution layer 43 is directly used as the input of the fourth stage convolution layer 44, and is output after sequentially undergoing 3x3 convolution operations and 4x upsampling; the output of the second stage convolution layer 45 is directly used as the input of the fifth stage convolution layer 45, and is output after sequentially undergoing 3x3 convolution operations and 2x upsampling; the output of the first stage convolution layer 44 is directly used as the input of the sixth stage convolution layer 46, and is output after undergoing 3x3 convolution operations. The outputs of the three stages of convolution layers 44, 45, and 46 of the upsampling path 40b are summed and passed through the softmax mapping layer to obtain the segmentation results predicted by the model, thereby realizing the segmentation of lesions in the perfusion image.

[0061] like Figure 5As shown, the first neural network 300 and the second neural network 400 of the lesion segmentation module 23 are connected in a cascade manner. For example, the perfusion parameters (perfusion parameter feature map) output by the first neural network 300 are concatenated with the feature map formed by downsampling the output of the first group of convolutional layers 31 as the input of the first stage convolutional layer 41 of the second neural network 400; the feature map formed by downsampling the output of the second group of convolutional layers 32 of the first neural network 300 and the feature map formed by downsampling the output of the first stage convolutional layer 41 of the second neural network 400 are concatenated as the input of the second stage convolutional layer 45 of the second neural network 400; the feature map formed by downsampling the output of the third group of convolutional layers 33 of the first neural network 300 and the feature map formed by downsampling the output of the second stage convolutional layer 45 of the second neural network 400 are concatenated as the input of the third stage convolutional layer 45 of the second neural network 400. It is understandable that in other embodiments, the first neural network 300 and the second neural network cascade 400 of the lesion segmentation module 23 may also be cascaded in other ways.

[0062] In the training phase, the perfusion image data and label data generated by the preprocessing module 22 are used for training the first neural network 300 and the second neural network 400. The training loss function includes two parts: MSE Loss based on regression and Dice loss based on segmentation. The first neural network 300 uses the mean square error (MSE Loss) function, and the second neural network uses the Dice coefficient difference (Dice Loss) function. The two losses are summed up as the loss of the overall model in each iteration, as shown in the formula:

[0063]

[0064] Among them, y i_predict represents the i-th perfusion parameter generated by the model, y i_label represents the i-th true perfusion parameter, n represents the predicted perfusion parameter category, TP, FN, and FP represent the true positive rate, false negative rate, and false positive rate in the segmentation results, respectively. For example, the training process uses the Adam optimizer, the learning rate is set to 0.0001, and the network loss converges after 50 iterations.

[0065] For example, in brain CT perfusion imaging, the first neural network module 300 is used to generate brain perfusion parameters, which include one or more parameters of cerebral blood volume (CBV), cerebral blood flow (CBF), mean contrast medium transit time (MTT), and contrast medium peak time (Tmax). The second neural network 400 performs lesion segmentation based on the aforementioned perfusion parameters and the input perfusion image to segment out the brain infarction area for diagnostic reference.

[0066] It can be understood that in the present invention, a cascaded first neural network and a second neural network are used, the first neural network generates perfusion parameters, and the second neural network performs lesion segmentation based on the perfusion parameters, which can realize automatic generation of perfusion parameters and lesion segmentation, can reduce the subjective differences introduced by the medical staff's empirical judgment, improve the accuracy of lesion segmentation judgment and reduce the time of lesion segmentation. In the above example, the lesion segmentation module 23 cascades the first neural network 300 using U-Net and the second neural network 400 using Dense V-net, which can automatically realize perfusion parameter generation and lesion segmentation at the same time, without the need for medical staff to intervene in the process. The first neural network 300 (U-Net network) inputs the features extracted from each group of convolutional layers of the coding path into the second neural network (Dense V-net network) 400 by skip connection, and combines the features extracted from different feature layers of the second neural network 400 to make the final lesion segmentation result more accurate.

[0067] Figure 6 An example block diagram of a perfusion imaging method according to an embodiment of the present invention is shown. The perfusion imaging method 600 includes a step 61 of acquiring a perfusion image dataset, a step 62 of preprocessing the perfusion image dataset, and a step 63 of segmenting a lesion.

[0068] In step 61, a perfusion image data set may be obtained from a perfusion image source, wherein the perfusion image source may include a perfusion image scanning device, such as CT, MRI, PET, PET-CT, PET-MR, SPECT, etc. The perfusion image source may also include a dedicated system for storing perfusion images, such as a picture archiving and communication system (PACS) or a computer cloud storage system. For example, in step 61, multi-phase CT perfusion image data is obtained. That is, after a rapid bolus injection of contrast agent into the patient's brain vein, a CT scan is performed on the patient to obtain CT perfusion image data at multiple time points at specific intervals. The CT perfusion image data form a CT perfusion image data set, and preferably, the CT image data set includes more than 10 phases of CT perfusion image data, such as 20 phases, 25 phases, 30 phases, 35 phases, and 40 phases. In CT perfusion imaging applied to other anatomical structures, multi-phase CT perfusion image data of the corresponding anatomical structure may also be obtained to form a CT perfusion image data set.

[0069] Figure 7 The step 62 of preprocessing the perfusion image data set is shown, and the step 62 includes a step 62b of normalizing the perfusion image data set, a step 62c of selecting perfusion image data, and a step 62d of generating label data. Optionally, in brain perfusion imaging, the step 62 of preprocessing the perfusion image may also include: Figure 7The step 62a of removing the skull image information is indicated by the dotted box in FIG. The pre-processed image data meets certain requirements, so as to further obtain suitable perfusion parameters and perform accurate lesion segmentation.

[0070] For example, in brain CT perfusion imaging, the step 62 of preprocessing the perfusion image may also include a step 62a of removing skull image information, which may remove skull image information from the CT perfusion image set, and then in subsequent image preprocessing and image segmentation, the influence of skull image information on non-skull image information in subsequent other preprocessing, perfusion parameter generation and lesion segmentation may be reduced. In step 62a, the clearest perfusion image data of one phase in the aforementioned perfusion image data set is selected, and the skull of the perfusion image data of this phase is removed by a traditional method such as a threshold method or a template matching method, and the perfusion image data of this phase after the skull image information is removed is used as a mask, and then the mask is multiplied with the perfusion image data of other phases to obtain a multi-phase perfusion image data set after the skull image information is removed. The selection of the clearest phase perfusion image data can be based on the CT value of each pixel in each phase of image data. For example, when the sum of the CT values ​​of all pixels in a certain phase of perfusion image data is the highest, or the sum of the CT values ​​of a certain proportion of pixels is the highest, or the average CT value of all pixels is the highest, the CT perfusion image data of that phase is selected as the clearest phase perfusion image data. It can be understood that the step of removing the skull image information can be automatically executed by a preset program without human intervention.

[0071] In step 62b, the perfusion image data set is standardized, and each pixel value in the perfusion image data set is subtracted from the mean of the entire perfusion image data set, and then divided by the standard deviation of the entire perfusion image data set. Standardization is used to control the data distribution within the range of a mean of 0 and a standard deviation of 1, which is beneficial to accelerate the neural network training process and reduce the possibility of the neural network model falling into a local optimum. For example, in brain CT perfusion imaging, the standardized perfusion image data set is to standardize the perfusion image data set after the skull image information is removed, specifically including calculating the mean and variance of the perfusion image data set, and then subtracting the mean from each pixel value and dividing it by the variance.

[0072] In step 62c, perfusion image data is selected to select perfusion image data of a specific period from the perfusion image data set. For example, as mentioned above, in brain CT perfusion imaging, the brain CT perfusion image data set includes multiple periods of data, which may usually reach 30 to 40 periods. In order to reduce the amount of calculation in step 62 of segmenting the lesion, CT perfusion image data of a specific period can be selected. For example, 10 periods of CT perfusion image data can be selected in this embodiment. The inflow time and outflow time of the contrast agent can be determined by the arterial input function (AIF), and the CT data corresponding to the period between the inflow and outflow time are selected. The inflow time refers to the initial point at which the AIF curve starts to rise, and the outflow time refers to the time point at which the AIF curve reaches the first trough after reaching the peak time point. If the period between the two moments is less than 10 periods, the CT perfusion image data is supplemented by zero matrix splicing, that is, the missing period is filled with a matrix with a value of 0, so that the overall CT perfusion image data can have 10 periods of data. If the number of periods between the inflow and outflow moments exceeds 10 periods, the data exceeding 10 periods are removed by truncation. The aforementioned selected 10-period CT perfusion image data can form a perfusion image data subset of, for example, 512*512*128.

[0073] In step 62d, label data is generated for generating label data required for deep learning model training in step 63 of lesion segmentation. For example, in brain CT perfusion imaging, the label data includes a series of perfusion parameters such as cerebral blood volume (CBV), cerebral blood flow (CBF), mean contrast agent transit time (MTT), and contrast agent peak time (Tmax). The generated label data can be calculated by the traditional maximum slope method and deconvolution method to obtain the label data. Since different cerebral perfusion parameters have different levels of quantity, generating label data includes further processing the label data, specifically, obtaining the maximum and minimum values ​​of each cerebral perfusion parameter in the entire data set, and then subtracting the minimum value from the label data, and then dividing the label data by the difference between the maximum value and the minimum value, thereby limiting the label data to between 0-1, thereby generating label data for deep learning model training. In one embodiment, the label data may include only the contrast agent peak time (Tmax), which has a higher sensitivity in determining the ischemic penumbra area. In other embodiments, the label data may include a combination of contrast agent peak time (Tmax) and other cerebral perfusion parameters.

[0074] In step 63, the lesions are segmented using Figure 5And the aforementioned cascaded first neural network module 300 and second neural network module 400. In the training stage, the perfusion image data set and label data generated by the aforementioned preprocessing are used for the training of the first neural network 300 and the second neural network 400. In the actual use stage, the trained first neural network 300 generates perfusion parameters, and the trained second neural network 400 performs image segmentation according to the perfusion parameters. For example, in brain perfusion imaging, brain perfusion parameters include one or more parameters of cerebral blood volume (CBV), cerebral blood flow (CBF), mean contrast agent transit time (MTT), and contrast agent peak time (Tmax). The second neural network module 400 segments the lesion based on the perfusion parameters and the preprocessed perfusion image data. In one embodiment, the label data may include only the contrast agent peak time (Tmax), which has a higher sensitivity in determining the ischemic penumbra area. In other embodiments, the label data may include a combination of the contrast agent peak time (Tmax) and other brain perfusion parameters.

[0075] It can be understood that in the present invention, a cascaded first neural network and a second neural network are used, the first neural network generates perfusion parameters, and the second neural network performs lesion segmentation based on the perfusion parameters, which can realize automatic generation of perfusion parameters and lesion segmentation, can reduce the subjective differences introduced by medical personnel's empirical judgment, improve the accuracy of lesion segmentation judgment and reduce the time of lesion segmentation. In the above example, the first neural network 300 using U-Net is cascaded with the second neural network 400 using DenseV-net, which can automatically realize perfusion parameter generation and lesion segmentation at the same time, without the need for medical personnel to intervene in the process. In addition, the first neural network 300 (U-Net network) inputs the features extracted from each group of convolutional layers of the coding path into the second neural network (Dense V-net network) 400 by skip connection, and the features extracted from the different feature layers of the second neural network 400 are combined to make the final lesion segmentation result more accurate.

[0076] Figure 8 An example of an electronic device 700 for executing a perfusion imaging method according to an embodiment of the present invention is shown. The electronic device 700 includes: one or more processors 71; a storage device 72 for storing one or more programs, when the one or more programs are executed by the one or more processors 71, the one or more processors 71 implement the method described herein. The processor is, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0077] Figure 8 The electronic device 700 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0078] like Figure 8 As shown, the electronic device 700 is in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: one or more processors 71, a storage device 72, and a bus 75 connecting different system components (including the storage device 72 and the processor 71).

[0079] Bus 75 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0080] The electronic device 700 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 700, including volatile and non-volatile media, removable and non-removable media.

[0081] The storage device 72 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 72a and / or cache memory 72c. The electronic device 700 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 72b may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 not shown, usually called a "hard drive"). Although Figure 8 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 75 via one or more data medium interfaces. The storage device 72 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0082] A program / utility 72d having a set (at least one) of program modules 72f may be stored, for example, in a storage device 72, such program modules 72f including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 72f generally perform the functions and / or methods of any embodiment described herein.

[0083] The electronic device 700 may also communicate with one or more external devices 76 (e.g., keyboards, pointing devices, displays 77, etc.), one or more devices that enable a user to interact with the electronic device 700, and / or any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 73. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 74. Figure 8 As shown, the network adapter 74 communicates with other modules of the electronic device 700 via the bus 75. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0084] The processor 71 executes various functional applications and data processing by running the programs stored in the storage device 72 .

[0085] According to an embodiment of the present invention, a computer readable medium is also provided, which has instructions thereon, and when the instructions are executed by a processor, the processor executes the steps of the method of the present invention. The computer readable medium may include, but is not limited to, a non-transient tangible arrangement of an article manufactured or formed by a machine or device, including a storage medium, such as: a hard disk; any other type of disk, including a floppy disk, an optical disk, a read-only compact disk memory (CD-ROM), a read-write compact disk memory (CD-RW), and a magneto-optical disk; a semiconductor device, such as a read-only memory (ROM), a random access memory (RAM) such as a dynamic random access memory (DRAM) and a static random access memory (SRAM), an erasable programmable read-only memory (EPROM), a flash memory, an electrically erasable programmable read-only memory (EEPROM); a phase change memory (PCM); a magnetic card or an optical card; or any other type of medium suitable for storing electronic instructions. The computer readable medium may be installed in a CT device, or in a separate control device or computer that remotely controls the CT device.

[0086] Fig. 9 FIG. 8 is a block diagram of an exemplary perfusion imaging system 800 according to an embodiment of the present invention. Fig. 9The perfusion imaging system 800 may include a medical imaging device 81 for performing imaging scans to generate medical images, a storage device 82 for storing medical images, and a medical imaging workstation 83 or a medical image cloud platform analysis system 84 that is communicatively connected to the storage device 82 and includes a processor 85. The processor 85 may be used to execute the aforementioned perfusion imaging method of the invention.

[0087] The medical imaging device 81 may be a CT device, an MRI device, a PET device, a SPECT device, or any other suitable imaging device. The storage device 82 may be located in the medical imaging device 81, in a server outside the medical imaging device 81, in an independent medical image storage system (such as PACS), and / or in a remote cloud storage system. The medical imaging workstation 83 may be set locally in the medical imaging device 81, that is, the medical imaging workstation 83 is set near the medical imaging device 81, and the two may be co-located in a scanning room, an imaging department, or the same hospital. The medical image cloud platform analysis system 84 may be located away from the medical imaging device 81, for example, in a cloud communicating with the medical imaging device 81. As an example, after a medical institution completes an imaging scan using the medical imaging device 81, the scanned data is stored in the storage device 82; the medical imaging workstation 83 may directly read the scanned data, and use the method of the present invention to perform image registration through its processor. As another example, the medical image cloud platform analysis system 84 may read the medical image in the storage device 82 through remote communication to provide "software as a service (SAAS)". SAAS can exist between hospitals, between hospitals and imaging centers, or between hospitals and third-party online diagnosis and treatment service providers.

[0088] The techniques described in the present invention may be implemented, at least in part, in hardware, software, firmware, or any combination thereof. For example, aspects of the techniques may be implemented in one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuits, and any combination of such components embodied in a programmer (such as a physician or patient programmer, stimulator, or other device). The terms "processor," "processing circuit," "controller," or "control module" may generally refer to any of the above logic circuits (alone or in combination with other logic circuits), or any other equivalent circuit (and alone or in combination with other digital or analog circuits).

[0089] Some exemplary embodiments of the present invention have been described above. However, it should be understood that various modifications may be made to the exemplary embodiments described above without departing from the spirit and scope of the present invention. For example, if the described techniques are performed in a different order and / or if the components in the described systems, architectures, devices, or circuits are combined in different ways and / or replaced or supplemented by other components or their equivalents, suitable results may be achieved, and accordingly, these modified other embodiments also fall within the scope of protection of the claims.

Claims

1. A perfusion imaging system for perfusion imaging, comprising: A data acquisition module, used for acquiring a perfusion image data set; A preprocessing module, used for preprocessing the perfusion image data set; A lesion segmentation module is used to perform lesion segmentation based on the preprocessed perfusion image data set, which includes a cascaded first neural network and a second neural network, the first neural network generates a perfusion parameter, and the second neural network performs lesion segmentation according to the perfusion parameter. The first neural network inputs the features extracted from each group of convolutional layers of the encoding path into the second neural network through skip connections.

2. The perfusion imaging system according to claim 1, wherein: The first neural network includes a U-net neural network.

3. The perfusion imaging system according to claim 2, wherein: The first neural network includes an encoding path having four groups of convolutional layers, wherein the first two groups of convolutional layers include convolutional layers with 1x1 convolutional kernels.

4. The perfusion imaging system according to claim 3, characterized in that: The second neural network includes a Dense V-net neural network.

5. The perfusion imaging system according to claim 4, characterized in that: The second neural network includes a downsampling path with three stages of convolutional layers, and the input of the first stage convolutional layer includes a concatenation of a downsampled feature map of the perfusion parameters generated by the first neural network and a downsampled feature map of the output of the first group of convolutional layers of the first neural network.

6. The perfusion imaging system according to claim 5, characterized in that: The inputs of the second-stage convolutional layer and the third-stage convolutional layer of the downsampling path of the second neural network respectively include the concatenation of the feature map formed by downsampling the output of the previous stage convolutional layer and the feature map formed by downsampling the output of a group of convolutional layers corresponding to the first neural network.

7. The perfusion imaging system according to claim 1, wherein: The preprocessing module includes a normalization module, which calculates the mean and variance of the perfusion image dataset, and subtracts the mean from each pixel value in the perfusion image dataset and divides the result by the variance.

8. The perfusion imaging system according to claim 1, wherein: The preprocessing module comprises a data selection module for selecting perfusion image data of a specific period from the perfusion image data set.

9. The perfusion imaging system according to claim 1, wherein: The perfusion image data set includes a brain perfusion image data set, and the preprocessing module also includes a skull image information removal module for removing skull part data in the brain perfusion image data set.

10. The perfusion imaging system according to claim 1, wherein: The perfusion image data set includes a cerebral perfusion image data set, and the perfusion parameter includes at least one of cerebral blood volume, cerebral blood flow, mean contrast agent transit time, and contrast agent peak time.

11. A perfusion imaging method, comprising: Acquire a perfusion image dataset; preprocessing the perfusion image dataset; Segmenting the lesion, comprising inputting the preprocessed perfusion image data set into a cascaded first neural network and a second neural network, the first neural network generating a perfusion parameter, and the second neural network performing lesion segmentation according to the perfusion parameter, The first neural network inputs the features extracted from each group of convolutional layers of the encoding path into the second neural network through skip connections.

12. The perfusion imaging method according to claim 11, characterized in that: The first neural network includes a U-net neural network.

13. The perfusion imaging method according to claim 12, wherein: The first neural network includes an encoding path having four groups of convolutional layers, wherein the first two groups of convolutional layers include convolutional layers with 1x1 convolutional kernels.

14. The perfusion imaging method according to claim 13, wherein: The second neural network includes a Dense V-net neural network.

15. The perfusion imaging method according to claim 14, characterized in that: The second neural network includes a downsampling path with three stages of convolutional layers, and the input of the first stage convolutional layer includes the perfusion parameters generated by the first neural network and the feature map output after the output of the first group of convolutional layers of the first neural network is downsampled.

16. The perfusion imaging method according to claim 15, characterized in that: The inputs of the second-stage convolutional layer and the third-stage convolutional layer of the downsampling path of the second neural network respectively include the concatenation of the feature map formed by downsampling the output of the previous stage neural network and the feature map formed by downsampling the output of a group of convolutional layers corresponding to the first neural network.

17. The perfusion imaging method according to claim 11, characterized in that: The perfusion image data set includes a brain perfusion image data set, and the preprocessing includes removing skull image information, which is used to remove skull part data in the brain perfusion image data set.

18. The perfusion imaging method according to claim 11, characterized in that: The perfusion image data set includes a cerebral perfusion image data set, and the perfusion parameter includes at least one of cerebral blood volume, cerebral blood flow, mean contrast agent transit time, and contrast agent peak time.

19. An electronic device comprising a processor for executing the perfusion imaging method according to any one of claims 11 to 18.

20. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the perfusion imaging method according to claims 11 to 18 is implemented.

Images