Prediction device, prediction program, and prediction method

Abstract

To provide a prediction device that outputs prediction information on a contrast effect by a contrast medium by a simple procedure.SOLUTION: A prediction device 32 that outputs prediction information on a contrast effect by a contrast medium includes: an acquisition unit 41A that acquires a pre-contrast image including at least a tissue image of a tissue of a subject imaged before injection of a contrast medium; and a prediction unit 41B that outputs prediction information by using a learned model DL that outputs effect information indicating a contrast effect by inputting the pre-contrast image.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a prediction device, a prediction program, and a prediction method for outputting prediction information regarding a contrast effect of a contrast agent. [Background technology]

[0002] Patent Document 1 discloses a trained model that estimates the injection amount and peak CT value of a contrast agent required for a main scan of a subject. A trained model generation device that generates this trained model processes a dataset into a training dataset. Here, the dataset includes data obtained when a scan in a test injection method (TI method) using X-rays and a main scan are performed. The dataset also includes information on the volume of organs through which a previously acquired contrast agent passes before reaching the subject's ROI, and other information on the subject, etc.

[0003] The trained model generating device then uses a training program to learn patterns of concentration change information of the contrast agent during the main scan from the training dataset, and generates a trained model. This trained model receives input data and outputs output data that estimates concentration change information of the contrast agent during the main scan. The input data also includes the injection amount and concentration change information of the contrast agent during the scan in the TI method. The input data also includes information on the volume of the organ, and other information on the subject. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-099409 A Summary of the Invention [Problem to be solved by the invention]

[0005] In clinical practice, a prediction device that outputs prediction information regarding the contrast effect of a contrast agent is required that does not require medical personnel to go through complicated procedures. Therefore, an object of the present invention is to provide a prediction device that outputs prediction information through simple procedures. [Means for solving the problem]

[0006] A prediction device according to one embodiment outputs predicted information regarding a contrast effect caused by a contrast agent, and includes an acquisition unit that acquires a pre-contrast image including at least a tissue image of a subject's tissue imaged before injection of the contrast agent, and a prediction unit that outputs the predicted information using a trained model that outputs effect information indicating the contrast effect by inputting the pre-contrast image.

[0007] In addition, a prediction program according to another embodiment is a prediction program that causes a computer to output predicted information regarding the contrast effect of a contrast agent, and causes the computer to acquire a pre-contrast image including at least a tissue image of a subject's tissue captured before injection of the contrast agent, and to output the predicted information using a trained model that outputs effect information indicating the contrast effect by inputting the pre-contrast image.

[0008] Furthermore, a prediction method according to another embodiment is a prediction method in which a computer outputs predicted information regarding the contrast effect of a contrast agent, in which the computer acquires a pre-contrast image including at least a tissue image of the tissue of a subject captured before injection of the contrast agent, and outputs the predicted information using a trained model that outputs effect information indicating the contrast effect by inputting the pre-contrast image. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram of an injection system and an imaging system. [Diagram 2] FIG. 1 is a schematic block diagram of a prediction device. [Diagram 3] FIG. 1 is an explanatory diagram of a trained model. [Figure 4]FIG. 1 is an explanatory diagram of neural network learning. [Diagram 5] 13 is a flowchart of input and output by a prediction unit. [Figure 6] FIG. [Figure 7] 7A and 7B are scatter plots of IDCE in the male subject group predicted by each model, where FIG. 7A corresponds to the BW model and FIG. 7B corresponds to the LBW model. [Figure 8] 8A and 8B are scatter plots of the IDCE in the male subject group predicted by each model, where FIG. 8A corresponds to the BSA model and FIG. 8B corresponds to the trained model. [Figure 9] 9A and 9B are scatter plots of IDCE in the female subject group predicted by each model, where FIG. 9A corresponds to the BW model and FIG. 9B corresponds to the LBW model. [Figure 10] 10A and 10B are scatter plots of the IDCE in the female subject group predicted by each model, where FIG. 10A corresponds to the BSA model and FIG. 10B corresponds to the trained model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, exemplary embodiments for carrying out the present invention will be described in detail with reference to the drawings. However, the dimensions, materials, shapes, and relative positions of components described in the following embodiments can be arbitrarily set and can be changed according to the configuration or various conditions of the device or method to which the present invention is applied. In addition, unless otherwise specified, the scope of the present invention is not limited to the embodiments specifically described below.

[0011] Unless otherwise specified, the term contrast agent includes both a contrast agent alone, such as iodine and gadolinium compounds, and a liquid medicine containing a contrast agent plus other solvents and additives. In the following description, the contrast agent alone may be referred to as a contrast component. In the following description, as size parameters representing body size, body weight may be referred to as "BW", lean body weight as "LBW", and body surface area as "BSA". The size parameters are calculated, but may also be measured or estimated, or may be average values ​​for a typical subject.

[0012] In the following, unless otherwise specified, the term pixel value includes the CT value of the contrast-imaging site, the sum or average of the CT values ​​of pixels included in the region of interest (ROI), or the SD value (standard deviation value) of the region of interest. Furthermore, the pixel value includes a value obtained by subtracting a value in a tissue not contrasted (for example, the CT value of the imaging site in a plain CT) from these values. The region of interest is set in advance, or the operator can select the region of interest.

[0013] [Embodiment] A medical system having an injection system 120 and an imaging system 130 will be described with reference to Fig. 1. The imaging system 130 of the medical system includes a control device 32, which is an example of a prediction device. In Fig. 1, the dotted lines connecting each component of the medical system indicate paths for transmitting and receiving data.

[0014] The injection system 120 includes an injection head 20, which is an example of an injection device that injects a contrast agent according to an injection protocol. For example, the injection protocol includes an injection amount, an injection time, and an injection rate. The injection system 120 also includes a console 23 that is connected to the injection head 20 by wire or wirelessly. The console 23 functions as an injection control device that controls the injection head 20. The injection system 120 also includes a stand 22 that holds the injection head 20.

[0015] Alternatively, injection system 120 may have, instead of console 23, an injection control device connected to injection head 20 and a display device (e.g., a tablet terminal or a touch panel display) separate from the injection control device. This display device is connected to the injection control device and displays the injection status of the medicinal liquid. In this case, injection head 20 and the injection control device can be configured integrally with stand 22. Furthermore, a hanging member can be provided instead of stand 22, and injection head 20 can be suspended from the ceiling via the hanging member.

[0016] The console 23 includes a touch panel 26 that functions as a display device and an input device. The touch panel 26 can display the injection protocol, the input state of the device, the setting state, the injection result, various information, and the like. The operator can also input medicinal solution information, the injection protocol, site information, and subject information, and the like, through the touch panel 26. Alternatively, the injection system 120 may include, instead of the touch panel 26, a display as a display device and an input device such as a keyboard or a numeric keypad as an input unit.

[0017] The injection head 20 is configured to mount a syringe filled with a medicinal liquid. The injection head 20 then injects the medicinal liquid filled in the syringe, such as physiological saline and various contrast agents, into the subject, who is the subject of the examination. To this end, the injection head 20 is provided with a drive mechanism that pushes out the medicinal liquid inside the syringe in accordance with an injection protocol. The injection head 20 also has a head display HD that displays the injection conditions, injection status, input status of the device, setting status, and various injection results. For example, the head display HD is a touch panel type display that can be operated by an operator.

[0018] Injection head 20 may also have a remote control device (e.g., a hand switch or foot switch) for remotely controlling injection head 20. This remote control device can remotely control injection head 20 to start or stop injection. Injection head 20 may also have a power source or battery. This power source or battery can be provided in either injection head 20 or the injection control device, or can be provided separately from these.

[0019] When the liquid medicine is to be injected, an accessory such as an extension tube is connected to the tip of the syringe mounted on injection head 20. Then, when preparation for injection is complete, the operator presses a check button. Alternatively, the operator may touch a check button displayed on touch panel 26. Thereafter, when injection begins, the liquid medicine pushed out from the syringe is injected into the subject via the extension tube.

[0020] In addition, the injection head 20 may be equipped with a pre-filled syringe having a data carrier such as an RFID chip, an IC tag, or a barcode, and various other syringes. The data carrier stores drug information related to the drug solution. When the injection head 20 is equipped with a syringe having a data carrier, the injection head 20 is equipped with a reading unit that reads the data carrier attached to the syringe. Furthermore, the injection head 20 may be configured to mount three or more syringes, or only one syringe.

[0021] The imaging system 130 includes a medical imaging device 30 that captures an image of a subject. The imaging device 30 is connected to the injection system 120 by wire or wirelessly. Examples of the imaging device 30 include various medical imaging devices such as an MRI (Magnetic Resonance Imaging) device, a CT (Computed Tomography) device, an angio imaging device, a PET (Positron Emission Tomography) device, a SPECT (Single Photon Emission Computed Tomography) device, a CT angio device, an MR angio device, an ultrasonic diagnostic device, and a blood vessel imaging device. In the following, an example in which the imaging device 30 is a CT device will be mainly described.

[0022] The imaging device 30 has an imaging section 31 that images a subject according to an imaging plan, and a control device 32 that controls the entire imaging device 30. As described later, the control device 32 also functions as a prediction device that outputs prediction information regarding the contrast effect of a contrast agent. For example, the imaging plan includes an imaging site, effective tube voltage, model name, manufacturer name, imaging time, tube voltage, imaging range, rotation speed, helical pitch, exposure time, dose, imaging method, and the like. The control device 32 controls the imaging section 31 to image the subject according to the imaging plan. The imaging device 30 also has a display 33.

[0023] The imaging unit 31 has a bed, an X-ray source that irradiates the subject with X-rays, and an X-ray detector that detects the X-rays that have passed through the subject. The imaging unit 31 irradiates the subject with X-rays and captures a perspective image of the subject by back-projecting the inside of the subject based on the X-rays that have passed through the subject.

[0024] The display 33 is connected to the control device 32 and displays the input state, setting state, imaging results, and various information of the device. Alternatively, the control device 32 and the display 33 may be configured integrally. Furthermore, the imaging device 30 has an interface 34 such as a keyboard that functions as an input unit. An operator can input information including drug solution information, injection protocol, tissue information, and subject information to the imaging device 30 from the interface 34. Furthermore, the display 33 may be a touch panel that functions as an input display device. In this case, the display 33 also functions as a substitute for the interface 34.

[0025] [Systems] 2, a control device 32, which is an example of a prediction device that outputs prediction information related to the contrast effect of a contrast agent, will be described. The control device 32 includes a control unit 41 and a storage unit 42. The storage unit 42 stores a trained model DL that outputs effect information indicating the contrast effect by inputting a pre-contrast image. Furthermore, the trained model DL has a neural network NN, which is a convolutional neural network.

[0026] Here, the pre-contrast image is an image captured before the injection of a contrast agent. The pre-contrast image includes at least a tissue image of the subject's tissue captured before the injection of a contrast agent. For example, the pre-contrast image includes an image captured by the imaging device 30 and an image obtained by reconstruction. As an example, the pre-contrast image is a positioning image (hereinafter also referred to as a scout image) obtained by imaging to determine the imaging range of the subject. Furthermore, the pre-contrast image may be an image obtained by performing image processing on the positioning image. For example, the image processing includes image quality adjustment processing such as brightness or saturation, enlargement processing, reduction processing, distortion correction processing, and trimming processing.

[0027] The contrast effect refers to an effect of amplifying the pixel value of the tissue in an image obtained by imaging the tissue to be examined by the contrast agent injected into the subject. Moreover, the effect information indicating the contrast effect is an effect that affects the pixel value of the tissue to be imaged, and is information indicating the effect of the contrast agent. As an example, the effect information is the amount of contrast agent used per pixel value. Specifically, the effect information is the amount of iodine used (mgI / HU) required to increase the pixel value by one unit (1HU). The amount of contrast agent used includes the amount of contrast agent containing a contrast component (mL) and the amount of only the contrast component (mgI). Therefore, the effect information may be the injection amount of contrast agent per pixel value (mL / HU). In the following, an example will be mainly described in which the amount of contrast agent used is the amount of only the contrast component, and the amount of contrast agent injected is the amount of a drug solution containing additives other than the contrast component.

[0028] Alternatively, the effect information may be a ratio to the average amount of contrast agent required to amplify the pixel value. As an example, the average amount of use can be obtained by calculating the amount of contrast agent required to increase the pixel value by one unit for each subject and averaging the amounts of use for a plurality of subjects. Specifically, when the average amount of iodine required to increase the pixel value by 1 HU is 40 mgI, the amount of iodine required to increase the pixel value of a specific subject's tissue by 1 HU as a predicted result is 36 mgI. In this case, the effect information is calculated as a ratio to the average amount of use, 90%=36 mgI÷40 mgI×100.

[0029] Contrast agent injection protocols may vary depending on the clinical site. In this regard, by outputting the ratio to the average usage amount, a trained model DL that can be used in a variety of clinical sites can be provided. In this case, the trained model DL is generated using supervised training data in which the ratio to the average usage amount is associated with the subject image.

[0030] The prediction information is information related to the contrast effect. For example, the prediction information includes effect information output by the trained model DL. As an example, the prediction information may be effect information, and in this case, the prediction information indicates the amount of contrast agent required to obtain a predetermined contrast effect. Specifically, the prediction device may output the amount of contrast agent used per pixel value as the prediction information. Furthermore, the prediction information may include information about the subject in addition to the effect information. As an example, the information about the subject is the subject's name, the subject ID, and the like. Furthermore, the prediction information may include examination information about the examination in addition to the effect information. As an example, the examination information is the examination ID, the examination date and time, and the like.

[0031] The prediction information may also indicate the injection amount (mL / HU) of the contrast agent required to increase the pixel value by one unit. Specifically, the injection amount of the contrast agent may be calculated and output based on the amount of contrast agent used per pixel value, the concentration of the contrast component, and the target value of the amplification degree. For example, the amount of contrast agent used per pixel value is 43.75 mgI / HU, the iodine concentration is 350 mgI / mL, and the target value of the amplification degree is 400 HU. In this case, the injection amount of the contrast agent is calculated and output as 50 mL=43.75 mgI / HU×400 HU÷350 mgI / mL. The target value of the amplification degree may be preset according to the case or the tissue of the test subject, or may be input by a medical professional.

[0032] Furthermore, the prediction information may indicate the pixel value or the amplification degree of the pixel value obtained when a predetermined amount of contrast agent is used or injected. Specifically, when the amount of contrast agent used per pixel value is 43.75 mgI / HU, when 50 mL of contrast agent with an iodine concentration of 350 mgI / mL is injected, the amplification degree of the pixel value of the contrast agent is calculated as 400HU=350 mgI / mL×50 mL÷43.75 mgI / HU. In the following, an example in which the prediction information and effect information are the amount of contrast agent used per pixel value will be mainly described.

[0033] [Trained model] Here, a trained model DL generated by deep learning, which is an example of machine learning, will be described with reference to Fig. 3 and Fig. 4. Fig. 3 is an explanatory diagram for explaining the trained model DL. Fig. 4 is an explanatory diagram for explaining the learning of the neural network NN.

[0034] In diagnostic imaging tests using contrast agents, appropriate contrast enhancement is important for accurate diagnosis. For example, in CT examinations using a CT scanner, the required amount of contrast agent is proportional to the body size. There is a high correlation between the required amount and body surface area (BSA), lean body weight (LBW), and body weight (BW). In addition, the effect of gradually amplifying the pixel values ​​of tissues is obtained from the time of injection of the contrast agent until imaging. Therefore, in addition to the body size, cardiac function such as cardiac output also affects the pixel values ​​in the captured image.

[0035] As an example of a model for predicting the amount of contrast agent used, a model generated by machine learning is considered. Specifically, an example of generating such a model will be described. To generate the model, machine learning is performed by dividing data of a predetermined number of subjects (e.g., 339 people) into a data set for cross-validation (e.g., 279 people) and a test data set (e.g., 60 people). In addition, images captured before and after the injection of the contrast agent are used to perform a cross-validation test. 2 The pixel values ​​in the circular ROI are recorded. Then, as a preparation for machine learning, the pixel values ​​of the image before the contrast agent injection are subtracted from the pixel values ​​of the image after the contrast agent injection to calculate the change in pixel value. After that, the effect of the subject's characteristics on the change in pixel value is evaluated. To do this, the amount of contrast agent used per pixel value (mgI / HU) for each subject is calculated.

[0036] In the machine learning of this example, six parameters (gender, age, height, weight, BMI, and cardiac output) are used. That is, in the supervised learning data used in the machine learning of this example, the six parameters are associated with the amount of contrast agent used. Then, regressors that apply linear regression, ridge regression, and lasso regression are created. Furthermore, the data set for cross-validation is divided into five, and the regressors are trained by applying cross-validation. In addition, in order to improve the prediction performance, ensemble learning is performed by calculating the average of the outputs of all regressors. This generates a model that outputs the amount of contrast agent used. However, the model generated in this way requires medical professionals to go through cumbersome input and measurement procedures because it uses six parameters.

[0037] On the other hand, the trained model DL according to this embodiment can output prediction information without requiring medical personnel to go through complicated procedures. A prediction device having the trained model DL outputs prediction information based on effect information indicating the contrast effect. To this end, a scout image, which is an example of a pre-contrast image, is input to the prediction device shown in FIG. 3. This scout image is captured before a CT examination, so no additional measurements such as weight measurement are required. Therefore, the scout image can be immediately used to predict the amount of contrast agent used during a CT examination.

[0038] The inventors of the present invention actually created a trained model DL using subject images of subjects. The subject images include tissue images captured before the injection of a contrast agent. First, as shown in FIG. 3, a data group of a scout image group G1 was prepared. Furthermore, a subject image group G2 used for learning was created from the scout image group G1. The subject group corresponding to the scout image group G1 was composed of 256 male subjects and 217 female subjects. Moreover, the imaging of the scout images and the imaging of the post-contrast images for diagnosis captured after the injection of the contrast agent were performed for all subjects under the following conditions.

[0039] The imaging device used was a 64-detector CT device ("LightSpeed ​​VCT" manufactured by GE Healthcare). Scout images and post-contrast images were taken by electrocardiogram-gated helical scanning in the forward axial direction. The imaging conditions were: rotation time 0.35 seconds, helical pitch 0.2 mm / rotation, beam collimation 40 mm, table travel distance 8.0 mm, scan field of view 50 cm, tube voltage 100 kVp, and tube current 400 mA to 770 mA. Image reconstruction was performed with a display field of view of 15 cm to 20 cm. The scout images and post-contrast images were saved in DICOM format. Since the gender of the subject affects the contrast effect, the scout images and post-contrast images were divided by gender.

[0040] The contrast agent injected into the subjects was Iomeron 350 (manufactured by Eisai Co., Ltd.), with an iodine content of 350 mgI / mL. A test bolus scan was performed to obtain the scan delay time. The amount of contrast agent injected was calculated by multiplying the total body weight (kg) of each subject by 0.6 ml. The injection time was 12 seconds, and 20 ml of saline was injected at the same injection rate as the contrast agent following the injection.

[0041] Then, as shown in FIG. 3, each scout image was trimmed to a size of 340 mm×530 mm. Specifically, the scout image was trimmed so as to maintain the horizontal width of the scout image relative to the subject of 340 mm. The scout image was also trimmed so as to remove the lower portion of the subject in the craniocaudal direction of the original image. This is because the tissue of the imaging target (e.g., the ascending aorta) is not captured in the lower portion, and there is little information related to the tissue.

[0042] As a result, the subject images in the subject image group G2 are obtained by trimming a predetermined range that does not include tissue images from the image captured before the injection of the contrast agent. Furthermore, the capture of a scout image is often performed using the subject's chin as a reference for the upper position. On the other hand, the size and length of the lower part of the subject's body included in the scout image varies depending on the medical staff in charge of the capture and the capture conditions. Therefore, a homogeneous subject image can be obtained by deleting the lower part of the scout image with the chin as the upper side.

[0043] The pre-contrast image may be trimmed to a predetermined size centered on the tissue of the imaging target. Furthermore, instead of trimming, the pre-contrast image may be reduced or enlarged to a predetermined size centered on the tissue of the imaging target. For example, the predetermined size is a size corresponding to the number of pixels of the pre-contrast image. As an example, when the number of pixels is 512 pixels x 512 pixels, the number of pixels is multiplied by the unit length applied to the subject image to set the predetermined size to 512 mm x 512 mm.

[0044] Also, the scout image may be trimmed so that the length and width are the same, and the subject image obtained by trimming may be used to generate the trained model DL. This makes it possible to prevent distortion of the scout image even when the subject image obtained by reducing or enlarging the scout image is used for training. Furthermore, the scout image that is not trimmed may be used as the subject image to generate the trained model DL.

[0045] Furthermore, brightness adjustment was performed on each scout image after the trimming. This brightness adjustment resulted in the subject images of the subject image group G2 being images that did not include pixels with brightness exceeding the upper limit or below the lower limit. Specifically, the brightness corresponding to the 98th percentile when all pixels of the scout images after the trimming were arranged in order from the lowest brightness pixel was set as the upper limit. Then, pixels with brightness exceeding the upper limit were changed to brightness corresponding to the 98th percentile. Furthermore, the brightness corresponding to the 1st percentile was set as the lower limit. Then, pixels with brightness below the lower limit were changed to brightness corresponding to the 1st percentile.

[0046] This prevents the learning performance from being degraded by pixels with abnormal brightness. Alternatively, the brightness adjustment may change pixels with brightness exceeding a predetermined upper limit to a brightness corresponding to the upper limit. Also, pixels with brightness below a predetermined lower limit to a brightness corresponding to the lower limit. As an example, in a brightness range of 0 to 255, the upper limit is 250 and the lower limit is 3.

[0047] The brightness adjustment may be performed before the trimming process. A scout image without brightness adjustment may be used as the subject image to generate the trained model DL. Images including pixels with brightness exceeding the upper limit or below the lower limit may be excluded from the training data. Images of subjects with metal embedded in their bodies may be excluded from the training data. This can prevent the learning performance from being degraded by metal artifacts.

[0048] After brightness adjustment, each scout image was downscaled to an 8-bit image of 224 × 224 pixels in PNG format to obtain each object image of object image group G2. Note that downscaled images cause distortion in the object images compared to the scout images. However, since the exact same distortion occurs in all object images, the effect of distortion on learning can be ignored.

[0049] In addition, before learning, an original model (specifically, EfficientNet-B0) was prepared. The original model has a convolutional neural network that has been trained in advance using a known dataset (specifically, ImageNet). Then, transfer learning was performed on the original model. Specifically, the fully connected layer of the original model was deleted, and four layers (a global average pooling layer, a batch normalization layer, a dropout layer, and a fully connected layer with an output size of 1) were added. In addition, an optimizer (specifically, Adam) was applied as the optimization function for learning. Furthermore, the mean squared error was used as the loss function. The learning rate was 1 × 10 -3 Alternatively, the Newton method or the like may be used for the optimization of learning.

[0050] The IDCE, which indicates the amount of contrast agent used per pixel value, was used as effect information output by the convolutional neural network of the trained model DL. IDCE was calculated by dividing the amount of iodine used (mgI) by the amplification of the pixel value (HU) (IDCE = amount of iodine used (mgI) ÷ amplification (HU)). This IDCE is the reciprocal of the change in pixel value per amount of contrast agent used. Here, IDCE was used because the change in pixel value per amount of contrast agent used is inversely proportional to size parameters such as BW, LBW, and BSA.

[0051] The amount used was calculated by multiplying the iodine concentration (mgI / mL) by the injection amount (mL) of the liquid containing the contrast component. For example, assume that the injection amount of the liquid is 50 mL and the iodine concentration is 350 mgI / mL. In this example, the amount of iodine used is calculated as 50 mL x 350 mgI / mL = 17500 mgI. If the amplification level is 400 HU, the IDCE is calculated as 17500 mgI ÷ 400 HU = 43.75 mgI / HU.

[0052] In addition, to calculate the amplification degree of the pixel value, the average pixel value (HU) of the ascending aorta was measured on the display 33. Specifically, a circular ROI was set in the ascending aorta in each of the scout image and the post-contrast image. The diameter of this ROI was 10 mm to 30 mm. Then, the amplification degree was calculated by subtracting the average pixel value in the ROI of the scout image from the average pixel value in the ROI of the post-contrast image.

[0053] In addition, since the gender of the subject affects the contrast effect, the prepared subject image group G2 was divided into groups by gender. Furthermore, 80% of the subject image group G2 was randomly assigned to the cross-validation dataset. Then, the remaining 20% ​​of the subject image group G2 was randomly assigned to the test dataset. As a result, image data of 204 male subjects was assigned to the cross-validation dataset, which is a training dataset. Then, image data of the remaining 52 male subjects was assigned to the test dataset. Furthermore, image data of 173 female subjects was assigned to the cross-validation dataset. Then, the remaining image group of 44 female subjects was assigned to the test dataset.

[0054] [Training for Convolutional Neural Networks] Thereafter, as shown in FIG. 3, the subject image group G2 was input to the input layer of the convolutional neural network during training of the convolutional neural network. The subject image group G2 was associated with the calculated IDCE and used as training data. The calculated IDCE was then used as training data for comparison with data output from the output layer of the convolutional neural network. The training data and training data are sometimes called supervised training data. As a result, the trained model DL is obtained by machine learning using supervised training data in which the subject image of the subject, including the tissue image captured before the injection of the contrast agent, is associated with the amplification degree of pixel values ​​based on the post-contrast image of the tissue captured after the injection of the contrast agent.

[0055] As shown in Figure 4, the K-fold cross-validation method was used to train the convolutional neural network. The number of divisions of the training dataset in the K-fold cross-validation method was set to "5". The number of training epochs for each of the five divided datasets was set to 750. The parameters that achieved the minimum value of the loss function during the iterative training were adopted as the parameters of the final trained model DL.

[0056] Specifically, taking the first dataset as an example, the cross-validation dataset (i.e., 80% of the subject image group G2) was divided into five to create the first dataset. Then, using the first dataset divided into five data groups, one of the five data groups was used as a test data group for validation. The remaining four data groups of the five data groups were used as training data groups for learning. Then, a fifth dataset was similarly created from the second dataset, and learning was performed so that each test data group was different.

[0057] As a result, the final trained model DL has five first to fifth split models corresponding to the five-split dataset. In the test, the test dataset (i.e., 20% of the subject image group G2) was input to each of the first to fifth split models, and IDCE was output. After the training was completed, the average value of the first to fifth output values, which are the IDCEs output from the first to fifth split models, was adopted as the IDCE output by the trained model DL. This enabled the effect of ensemble learning to be obtained.

[0058] In addition, when generating the trained model DL of this embodiment, the above-mentioned post-contrast image may be an image obtained by performing image processing such as brightness adjustment on a captured image. In addition, the amplification degree may be calculated without using the above-mentioned calculation method as long as it is information indicating the degree to which pixel values ​​are amplified by contrast. For example, the amplification degree may be calculated by subtracting an average innate pixel value of the tissue to be imaged from an average pixel value in the ROI of the post-contrast image. Furthermore, the amplification degree may be calculated by subtracting a peak pixel value in the ROI of the scout image from a peak pixel value in the ROI of the post-contrast image. In addition, the amplification degree may be an average pixel value or a peak pixel value in the ROI of the post-contrast image.

[0059] [Control Unit] Returning to FIG. 2, the control device 32 can receive information from an external storage device (not shown), such as a server, and can also transmit information to the server. This server can be, for example, a Radiology Information System (RIS), a Picture Archiving and Communication System (PACS), a Hospital Information System (HIS), an image inspection system, or an image creation workstation. The server can also store information about the imaging results including data such as a scout image and a post-contrast image transmitted from the imaging device 30. Note that an external image inspection system or an image creation workstation can also be used to operate the injection head 20 and the imaging device 30. Furthermore, a device (for example, a tablet terminal) that can remotely operate the injection head 20 may be installed inside or outside the hospital.

[0060] The control unit 41 is a processor that executes various arithmetic processing and operation control according to a predetermined program. This processor is configured as a computer that combines an internal memory required for the operation of the processor and other peripheral devices. For example, the processor is a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit). The processor controls the entire imaging system 130 based on a control program PG stored in the storage unit 42, and also centrally controls various processes. The control unit 41 may have an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0061] In addition, the control unit 41 may perform control according to a control program PG stored in a portable recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), a CF (Compact Flash) card, a micro SD card, and a USB (Universal Serial Bus) memory, or in an external storage medium such as a server connected to the Internet or a LAN.

[0062] Furthermore, the control unit 41 has an acquisition unit 41A and a prediction unit 41B. Each unit is a logical device realized by a combination of the hardware resources of the control unit 41 and a control program PG as a software resource. The storage unit 42 is a computer-readable non-transitory recording medium. The storage unit 42 also includes a RAM (Random Access Memory) which is a system work memory for the processor of the control unit 41 to operate, as well as a ROM (Read Only Memory) for storing programs and system software, and storage devices such as an HDD (Hard Disc Drive) and an SSD (Solid State Drive).

[0063] The storage unit 42 also stores a control program PG as an example of a prediction program. The control program PG causes a computer to predict the contrast effect of a contrast agent. Specifically, the control program PG causes the control unit 41 as a computer to function as an acquisition unit 41A and a prediction unit 41B. The control program PG can also be stored in a computer-readable non-transitory recording medium other than the storage unit 42.

[0064] The control unit 41 may function as a generating device for the trained model DL. In this case, the storage unit 42 may further store training data and teacher data. As an example, the training data and teacher data are used to generate trained parameters PM by performing deep learning on the neural network NN. The trained parameters PM include weighting coefficients indicating the connection weights between the nodes.

[0065] The acquisition unit 41A acquires a pre-contrast image including at least a tissue image of the subject's tissue captured before injection of a contrast agent. As an example, the acquisition unit 41A acquires a scout image as a pre-contrast image from the imaging device 30. Alternatively, the acquisition unit 41A may acquire a scout image from a server, or may acquire a scout image input by a medical professional. Note that, if image processing has been applied to the pre-contrast image acquired by the acquisition unit 41A, the control unit 41 may perform the image processing.

[0066] Furthermore, when the control unit 41 functions as a generating device for the trained model DL, the acquisition unit 41A may acquire pre-contrast images for learning. In this case, the control unit 41 may generate learning data by performing image processing on the pre-contrast images acquired by the acquisition unit 41A. Furthermore, the acquisition unit 41A may acquire post-contrast images to calculate IDCE as teacher data. In this case, the control unit 41 may calculate IDCE based on the acquired pre-contrast images and post-contrast images.

[0067] The prediction unit 41B outputs prediction information using the trained model DL. The trained model DL outputs effect information indicating the contrast effect by inputting the acquired pre-contrast image. For this purpose, the storage unit 42 stores trained parameters PM related to the neural network NN. Note that the neural network NN may be a neural network other than a convolutional neural network.

[0068] As an example, the prediction unit 41B inputs a scout image captured during a CT examination into the input layer of the neural network NN of the trained model DL. Then, the trained model DL performs a calculation based on the trained parameters PM on the scout image input into the input layer. Furthermore, the trained model DL outputs IDCE from the output layer. Then, the prediction unit 41B outputs the IDCE output by the trained model DL as prediction information. This allows a medical professional to calculate the amount of contrast agent to be injected to contrast the tissue to be imaged based on the output IDCE.

[0069] The trained model DL may be configured to output effect information by inputting other information in addition to the pre-contrast image. For example, the trained model DL may output effect information by further inputting subject information related to the subject. In this case, the trained model DL is generated using supervised training data in which the subject information is associated with the subject image.

[0070] As an example, the subject information is the gender of the subject. The subject information may include at least one of body weight, lean body mass, subcutaneous fat thickness, race, and heart size in addition to or instead of gender. The subject information may be automatically acquired by the acquisition unit 41A by subjecting the pre-contrast image to image analysis. The acquisition unit 41A may also acquire subject information input by a medical professional. Furthermore, the acquisition unit 41A may acquire the subject information from the imaging system 130, the injection system 120, or a server.

[0071] Furthermore, the trained model DL may be configured to output effect information by inputting an injection protocol of the contrast agent in addition to the pre-contrast image. In this case, the trained model DL is generated using supervised learning data in which the injection protocol is associated with the subject image. As an example, the injection protocol includes at least one of an injection amount, an injection rate, and an injection time of the contrast agent. For example, the acquisition unit 41A acquires the injection protocol from the imaging system 130, the injection system 120, or a server. Alternatively, the acquisition unit 41A may acquire the injection protocol input by a medical professional.

[0072] The trained model DL may be configured to output effect information by inputting imaging conditions in addition to the pre-contrast image. In this case, the trained model DL is generated using supervised learning data in which imaging conditions are associated with the subject image. As an example, the imaging conditions include at least one of information identifying the imaging device 30, such as the device name, tube voltage, tube current, and number of pixels. For example, the acquisition unit 41A acquires the imaging conditions from the imaging system 130, the injection system 120, or a server. Alternatively, the acquisition unit 41A may acquire the imaging conditions input by a medical professional.

[0073] [flow] With reference to FIG. 5, the input and output by the prediction unit 41B will be described. When a scout image is captured, the acquisition unit 41A acquires the scout image as a pre-contrast image (S101). Then, the prediction unit 41B inputs the acquired scout image to an input layer of the neural network NN of the trained model DL (S102). Then, the trained model DL outputs IDCE as effect information (S103). Furthermore, the prediction unit 41B outputs the output IDCE as predicted information (S104). For example, the prediction unit 41B causes the display 33 to display the predicted information. In this way, the control device 32 as a prediction device outputs IDCE as predicted information regarding the contrast effect of the contrast agent.

[0074] [Evaluation of pre-trained model DL] The inventors of the present invention evaluated the generated trained model DL. Specifically, a model in which the output IDCE is proportional to the body size parameter was adopted as a comparative model for comparing the performance with the trained model DL. Specifically, a BW model, an LBW model, and a BSA model were used as comparative models for predicting IDCE from BW, LBW, and BSA. The IDCE output by each model is proportional to a proportionality coefficient C determined by the least squares method using the training dataset used to train the trained model DL. BW , C LBW , and C BSA The weight and height of the subject used in the following calculations were measured before the scout image was taken.

[0075] Specifically, in the BW model, IDCE=C BW The IDCE is calculated and output by the formula × BW. In the LBW model, IDCE = C LBW IDCE is calculated and output by the formula: IDCE=C × LBW. Here, LBW is calculated using James's definition formula. Specifically, the LBW of a male subject is calculated using formula 1 in FIG. 6. Also, the LBW of a female subject is calculated using formula 2 in FIG. 6. In addition, in formulas 1 to 3 in FIG. 6, "W" is body weight (kg) and "H" is height (cm). In the BSA model, IDCE=C BSA The IDCE is calculated and output by the formula: ×BSA, where BSA is calculated using formula 3 in FIG.

[0076] Then, the trained model DL generated in the above-mentioned manner was compared with the BW model, the LBW model, and the BSA model. Specifically, the test data set used for training the trained model DL was used to compare the Pearson correlation coefficient (r) between the actual IDCE and the IDCE output from each model. Note that Figures 7 to 10 are scatter plots between the actual value of the calculated IDCE and the output IDCE. In each figure, the horizontal axis represents the actual value, and the vertical axis represents the output value.

[0077] As shown in Figs. 7 to 10, the comparison of the trained model DL and the comparison model was performed separately for men and women. The correlation coefficient between the output IDCE and the actual IDCE was r = 0.539 for the BW model in the male subject group shown in Fig. 7A. The correlation coefficient for the LBW model in the male subject group shown in Fig. 7B was r = 0.563. The correlation coefficient for the BSA model in the male subject group shown in Fig. 8A was r = 0.587. The correlation coefficient for the trained model DL in the male subject group shown in Fig. 8B was r = 0.607.

[0078] Furthermore, in the female subject group shown in FIG. 9A, the correlation coefficient in the case of the BW model was r=0.290. In addition, in the female subject group shown in FIG. 9B, the correlation coefficient in the case of the LBW model was r=0.352. In addition, in the female subject group shown in FIG. 10A, the correlation coefficient in the case of the BSA model was r=0.349. And, in the female subject group shown in FIG. 10B, the correlation coefficient in the case of the trained model DL was r=0.412.

[0079] In addition, we performed a Bland-Altman analysis of the output IDCE and the actual IDCE using each model. As a result, for the trained model DL, the tolerance range between the output value and the actual value was narrow, with the average difference for the male subject group being -0.1 ± 8.0, and the tolerance range being -15.9 to 16.3. The average difference for the female subject group was 0.2 ± 5.5, and the tolerance range being -10.5 to 10.9. On the other hand, for the BW model, the average difference for the male subject group was -0.4 ± 8.5, and the tolerance range being -17.0 to 16.3. The average difference for the female subject group was -1.4 ± 6.2, and the tolerance range being -13.6 to 10.7.

[0080] Furthermore, for the LBW model, the mean difference for the male subject group was -0.4 ± 8.4, with a tolerance range of -16.8 to 16.0. For the female subject group, the mean difference was -0.7 ± 5.7, with a tolerance range of -11.8 to 10.3. For the BSA model, the mean difference for the male subject group was -0.7 ± 8.5, with a tolerance range of -17.3 to 16.0. For the female subject group, the mean difference was -0.6 ± 5.6, with a tolerance range of -11.6 to 10.5.

[0081] Thus, compared to the other three models, the trained model DL provided a higher correlation coefficient and an average difference closest to zero for both the male subject group and the female subject group. These results demonstrate that the trained model DL of this embodiment is more stable than the other three models.

[0082] The trained model DL, which inputs a pre-contrast image and outputs effect information, can predict the contrast effect with higher accuracy than the BW model, the LBW model, and the BSA model. The reason for the high prediction accuracy is that the pre-contrast image contains information about the subject. For example, the pre-contrast image contains information about the body size and the heart. Therefore, by machine learning using a dataset including the pre-contrast image, it is possible to generate a trained model DL that learns features directly from the pre-contrast image and predicts the contrast effect with high accuracy.

[0083] Specifically, the pre-contrast image includes information on the size of the heart and the thickness of the ventricular wall. For example, when the size of the heart is large, the cardiac function is reduced, which negatively affects the contrast effect. Also, when the body size of the subject is large, the amount of contrast agent required is increased. Furthermore, the pre-contrast image includes information on subcutaneous fat. When the thickness of the subcutaneous fat is large, blood does not flow through the subcutaneous fat, so the amount of contrast agent delivered to the tissue increases. As a result, the contrast effect is positively affected, and the amount of contrast agent required is reduced. Furthermore, the pre-contrast image includes information on the age of the subject. For example, the costal cartilage is calcified with age, so the pre-contrast image shows that the subject is elderly. And when the subject is elderly, the contrast effect is negatively affected due to the decline in cardiac function and hardening of blood vessels.

[0084] According to the trained model DL of the present embodiment described above, a prediction device that outputs prediction information in a simple procedure can be provided. Specifically, there is no need to add a complicated procedure for inputting many variables such as the weight of the subject. Furthermore, by using the positioning image used during the examination, no additional measurements are required to calculate the IDCE. Therefore, it is possible to improve the throughput in the clinical field and reduce human errors. Furthermore, the IDCE can be determined even when subject information such as weight or height is not easily available due to an emergency. Furthermore, according to the prediction device of the present embodiment, the IDCE can be predicted more accurately than before.

[0085] Although the present invention has been described above with reference to each embodiment, the present invention is not limited to the above embodiment. Inventions modified without violating the present invention and inventions equivalent to the present invention are also included in the present invention. In addition, each embodiment and each modified form, and technical means included in each embodiment or each modified form can be appropriately combined without violating the present invention.

[0086] For example, although the imaging system 130 includes the prediction device in the above example, the injection system 120 may include the prediction device. As an example, the console 23 may function as the prediction device instead of or in addition to the control device 32. In this case, the prediction device of the injection system 120 may obtain a pre-contrast image from a server or the imaging system 130.

[0087] Furthermore, instead of or in addition to the control device 32, a computer device functioning as a prediction device may be provided in the medical system. Moreover, instead of or in addition to displaying the prediction information, the prediction device may present the prediction information to a medical professional by voice. Moreover, instead of or in addition to displaying the prediction information, the prediction device may transmit the prediction information to an external device.

[0088] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0089] (Appendix 1) A prediction device that outputs prediction information regarding a contrast effect of a contrast agent, an acquisition unit that acquires a pre-contrast image including at least a tissue image of a tissue of a subject captured before injection of the contrast agent; a prediction unit that outputs the prediction information using a trained model that outputs effect information indicating the contrast effect by inputting the pre-contrast image.

[0090] (Appendix 2) The prediction device according to claim 1, wherein the pre-contrast image is a positioning image obtained by imaging for determining an imaging range of the subject.

[0091] (Appendix 3) The prediction device according to claim 1 or 2, wherein the trained model further outputs the effect information by inputting subject information related to the subject.

[0092] (Appendix 4) The prediction device according to claim 3, wherein the subject information is the gender of the subject.

[0093] (Appendix 5) The prediction device according to any one of appendix 1 to 4, wherein the trained model further outputs the effect information by inputting an injection protocol of the contrast agent.

[0094] (Appendix 6) The prediction device described in Appendix 5, wherein the injection protocol includes at least one of an injection amount, an injection rate, and an injection time of the contrast agent.

[0095] (Appendix 7) the effect information being an amount of the contrast agent used per pixel value, or a ratio of the amount of the contrast agent required to amplify a pixel value to an average amount of the contrast agent used, The prediction device according to any one of claims 1 to 6, wherein the prediction information includes the effect information.

[0096] (Appendix 8) The prediction device according to any one of Appendix 1 to 7, wherein the trained model is obtained by machine learning using supervised learning data in which a subject image of the subject including the tissue image captured before injection of the contrast agent is associated with an amplification degree of pixel values ​​based on a post-contrast image of the tissue captured after injection of the contrast agent.

[0097] (Appendix 9) The prediction device according to claim 8, wherein the subject image is obtained by trimming a predetermined range that does not include the tissue image from an image captured before injection of the contrast agent.

[0098] (Appendix 10) The prediction device according to claim 8 or 9, wherein the object image is an image that does not contain any pixels having a brightness exceeding an upper limit value or less than a lower limit value.

[0099] (Appendix 11) A prediction program for causing a computer to output prediction information regarding a contrast effect of a contrast agent, The computer includes: acquiring a pre-contrast image including at least a tissue image of the subject's tissue imaged before the injection of the contrast agent; A prediction program that outputs the prediction information using a trained model that outputs effect information indicating the contrast effect by inputting the pre-contrast image.

[0100] (Appendix 12) A prediction method in which a computer outputs predicted information regarding a contrast effect of a contrast agent, the method comprising: The computer includes: obtaining a pre-contrast image including at least a tissue image of the subject's tissue captured before the injection of the contrast agent; A prediction method comprising: outputting the prediction information using a trained model that outputs effect information indicating the contrast effect by inputting the pre-contrast image. [Explanation of symbols]

[0101] 32: Control device (prediction device) 41: Control unit (computer) 41A: Acquisition section 41B: Prediction Department DL: Pre-trained model PG: Control program (prediction program)

Claims

1. A prediction device that outputs prediction information regarding a contrast effect of a contrast agent, an acquisition unit that acquires a pre-contrast image including at least a tissue image of a tissue of a subject captured before injection of the contrast agent; a prediction unit that outputs the prediction information using a trained model that outputs effect information indicating the contrast effect by inputting the pre-contrast image.

2. The prediction device according to claim 1 , wherein the pre-contrast image is a positioning image obtained by imaging for determining an imaging range of the subject.

3. The prediction device according to claim 2 , wherein the trained model further inputs subject information relating to the subject, and outputs the effect information.

4. The prediction device according to claim 3 , wherein the subject information is a gender of the subject.

5. The prediction device according to claim 1 , wherein the trained model further outputs the effect information by inputting an injection protocol of the contrast agent.

6. The prediction device according to claim 5 , wherein the injection protocol includes at least one of an injection amount, an injection rate, and an injection time of the contrast agent.

7. the effect information is a usage amount of the contrast agent per pixel value, or a ratio of the usage amount of the contrast agent required to amplify a pixel value to an average usage amount of the contrast agent; The prediction device according to claim 1 , wherein the prediction information includes the effect information.

8. 5. The prediction device according to claim 1, wherein the trained model is obtained by machine learning using supervised learning data in which a subject image of the subject including the tissue image captured before the injection of the contrast agent is associated with an amplification degree of pixel values ​​based on a post-contrast image of the tissue captured after the injection of the contrast agent.

9. The prediction device according to claim 8 , wherein the subject image is obtained by trimming a predetermined range that does not include the tissue image from an image captured before the injection of the contrast agent.

10. The prediction device according to claim 8 , wherein the object image is an image that does not include pixels having a brightness exceeding an upper limit value or a brightness less than a lower limit value.

11. A prediction program for causing a computer to output prediction information regarding a contrast effect of a contrast agent, The computer includes: acquiring a pre-contrast image including at least a tissue image of the subject's tissue imaged before the injection of the contrast agent; A prediction program that outputs the prediction information using a trained model that outputs effect information indicating the contrast effect by inputting the pre-contrast image.

12. A prediction method in which a computer outputs predicted information regarding a contrast effect of a contrast agent, the method comprising: The computer includes: obtaining a pre-contrast image including at least a tissue image of the subject's tissue captured before the injection of the contrast agent; A prediction method comprising: outputting the prediction information using a trained model that outputs effect information indicating the contrast effect by inputting the pre-contrast image.

Images