Program, information processing method, and information processing device

Abstract

A program according to one aspect causes a computer to execute a process for acquiring a plurality of brain CT images that are sliced, and inputting the acquired plurality of CT images into a learning model (171) that outputs information pertaining to a plurality of bleeding regions when a plurality of CT images are input, thereby outputting information pertaining to bleeding regions. In this way, it becomes possible to output information pertaining to a plurality of bleeding regions in CT images.

Description

Program, information processing method and information processing device

[0001] The present invention relates to a program, an information processing method, and an information processing device.

[0002] In recent years, there has been active development of technology for detecting brain diseases based on medical images of a patient's brain. For example, Patent Literature 1 discloses a stroke diagnosis and treatment support device that estimates the type of stroke using predetermined image information related to the stroke of a patient and test finding information indicating the test findings of the patient.

[0003] Patent No. 7315657

[0004] However, the technique disclosed in Patent Document 1 has a problem in that it is not possible to output information relating to multiple bleeding regions in a CT (Computed Tomography) image.

[0005] One aspect of the present invention is to provide a program or the like that is capable of outputting information about a plurality of bleeding regions in a CT image.

[0006] A program according to one aspect acquires multiple sliced ​​CT images of the brain, and causes a computer to execute a process of inputting the acquired multiple CT images into a learning model that outputs information about multiple bleeding areas when multiple CT images are input, thereby outputting information about bleeding areas.

[0007] In one aspect, it is possible to output information about multiple bleeding regions in a CT image.

[0008] FIG. 1 is a block diagram showing an example of the configuration of a computer. FIG. 1 is an explanatory diagram explaining a bleeding region output model. FIG. 2 is a flowchart showing the processing procedure when generating a U-Net. FIG. 3 is a flowchart showing the processing procedure when generating a transformer. FIG. 4 is a flowchart showing the processing procedure when outputting bleeding position information using a U-Net. FIG. 5 is a flowchart showing the processing procedure when outputting information about a bleeding region using a transformer. FIG. 6 is a block diagram showing an example of the configuration of a computer in embodiment 2. FIG. 7 is an explanatory diagram explaining a hematoma growth output model. FIG. 8 is a flowchart showing the processing procedure when generating a hematoma growth output model. FIG. 9 is a flowchart showing the processing procedure when outputting information about hematoma growth.

[0009] The present invention will be described in detail below with reference to the drawings showing embodiments thereof.

[0010] (Embodiment 1) Embodiment 1 relates to a form in which information about multiple hemorrhagic regions is output using artificial intelligence (AI) based on multiple sliced ​​CT images of the brain. Brain CT images are images obtained by taking tomographic images of the head using X-rays. By using brain CT images, diseases such as intracerebral hemorrhage or cerebral infarction can be diagnosed.

[0011] The acute intracerebral hemorrhage system of this embodiment includes an information processing device 1. The information processing device 1 processes, stores, and transmits / receives various types of information. The information processing device 1 is, for example, a server device, a personal computer, or a general-purpose tablet PC (personal computer). Note that the information processing device 1 may also be an information processing device such as a smartphone, a mobile phone, a wearable device such as an Apple Watch (registered trademark), a wearable camera, or a tablet. In this embodiment, the information processing device 1 is a personal computer, and for simplicity, will be referred to as a computer 1 below.

[0012] Because intracerebral hemorrhage can cause serious aftereffects and, in some cases, lead to death, it is necessary to quickly determine the appropriate treatment.However, it is not easy for doctors without experience in stroke treatment to determine the location or amount of cerebral hemorrhage and determine the treatment plan, including whether surgery is appropriate.

[0013] The computer 1 of this embodiment uses a learning model that outputs information about multiple bleeding areas when multiple CT images are input, based on multiple sliced ​​CT images of the brain, and automatically outputs information about multiple bleeding areas that serves as the basis for determining a treatment plan for intracerebral hemorrhage.

[0014] 1 is a block diagram showing an example of the configuration of a computer 1. The computer 1 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, a display unit 15, a reading unit 16, and a large-capacity storage unit 17. Each component is connected by a bus B.

[0015] The control unit 11 includes a processing unit such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and a GPU (Graphics Processing Unit), and performs various information processing, control processing, etc. related to the computer 1 by reading and executing a control program 1P (program product) stored in the memory unit 12.

[0016] It should be noted that the control program 1P can be deployed to run on a single computer, or on multiple computers located at one site, or distributed across multiple sites and interconnected by a communications network.

[0017] 1, the control unit 11 is described as a single processor, but it may be a multi-processor. The control unit 11 may execute various information processing or control processes by the same processor within the computer 1, or may execute various information processing or control processes by different processors within the computer 1.

[0018] The storage unit 12 includes memory elements such as RAM (Random Access Memory) and ROM (Read Only Memory), and stores the control program 1P or data required for the control unit 11 to execute processing. The storage unit 12 also temporarily stores data required for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing processing related to communication.

[0019] The input unit 14 may be a keyboard, a mouse, or a touch panel integrated with the display unit 15. The display unit 15 is a liquid crystal display, an organic EL (electroluminescence) display, or the like, and displays various information according to instructions from the control unit 11.

[0020] The reading unit 16 reads a portable storage medium 1a including a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 16 and store it in the mass storage unit 17. Alternatively, the control unit 11 may download the control program 1P from another computer via a network or the like and store it in the mass storage unit 17. Alternatively, the control unit 11 may read the control program 1P from the semiconductor memory 1b.

[0021] The mass storage unit 17 includes a recording medium such as a hard disk drive (HDD) or a solid state drive (SSD), etc. The mass storage unit 17 includes a bleeding region output model (learning model) 171.

[0022] The hemorrhage region output model 171 is an output device (estimator) that outputs information about multiple hemorrhage regions based on multiple sliced ​​CT images of the patient's brain, and is a trained model generated by deep learning. In this embodiment, multiple CT images obtained by performing tomography of the head of the same patient using X-rays are input to the hemorrhage region output model 171.

[0023] In this embodiment, the storage unit 12 and the large-capacity storage unit 17 may be configured as an integrated storage device. The large-capacity storage unit 17 may also be configured with a plurality of storage devices. Furthermore, the large-capacity storage unit 17 may also be an external storage device connected to the computer 1.

[0024] The computer 1 may execute various information processing and control processing by itself or may execute them in a distributed manner across multiple computers. The computer 1 may also be implemented by multiple virtual machines installed in a single computer, or may be implemented using a cloud server.

[0025] Next, a process for outputting information about a plurality of hemorrhage regions will be described. The computer 1 acquires a plurality of sliced ​​CT images of the patient's brain. The CT images may be stored in advance in the storage unit 12 or the mass storage unit 17 of the computer 1, or may be stored in an external information processing device. The computer 1 inputs the acquired plurality of CT images (input data) to the hemorrhage region output model 171 and outputs information about the hemorrhage regions (output data).

[0026] 2 is an explanatory diagram illustrating the hemorrhage region output model 171. The hemorrhage region output model 171 is used as a program module that is part of artificial intelligence software. The hemorrhage region output model 171 is an output device in which a neural network has been constructed, which receives multiple sliced ​​CT images of the brain as input and outputs information about multiple hemorrhage regions. The information about hemorrhage regions will be described later.

[0027] The hemorrhage region output model 171 is a model that combines a U-Net 91 that segments hemorrhage from the cerebral parenchyma, intraventricular hemorrhage, and edema around the hemorrhage, and a Transformer model 92 (hereinafter referred to as Transformer 92).

[0028] First, we will explain U-Net91. U-Net91 is a neural network model constructed using U-Net (Convolutional Networks for Biomedical Image Segmentation), which is designed for CT image segmentation. U-Net91 is an output device (classifier) ​​that has a pre-constructed neural network that receives a brain CT image as input and outputs bleeding location information, including the location of cerebral parenchymal hemorrhage, the location of intraventricular hemorrhage, and the location of edema around the bleeding location, in the CT image.

[0029] In this embodiment, the U-Net 91 is used for classification processing to classify each pixel of a brain CT image into classes, thereby classifying bleeding location information including the location of cerebral parenchymal hemorrhage, the location of intraventricular hemorrhage, and the location of edema around the bleeding location in the CT image. The U-Net 91 is configured, for example, with an encoder including a convolutional layer and a pooling layer that perform downsampling, and a decoder including a convolutional layer and a pooling layer that perform upsampling.

[0030] The computer 1 acquires a large number of CT images of the brain as training data. The training data is a combination of CT images of the brain and annotation masks (annotated CT images) corresponding to the CT images. The training data can be used to train the U-Net 91 in an image segmentation task (a task of predicting to which class each pixel belongs). The training data may also be data created separately and manually. For example, a segmentation mask in which an expert (e.g., a radiologist) manually draws the boundary of a bleeding area on a CT image may be created as training data.

[0031] The computer 1 uses training data to perform a learning process for the U-Net 91. Specifically, the computer 1 inputs CT images included in the training data into the encoder of the U-Net 91. The computer 1 extracts features (feature maps) of the CT images by gradually reducing the image size of the input CT images through the encoder of the U-Net 91 (e.g., from 512 × 512 to 16 × 16) and performing multiple types of filtering for each layer with different image sizes. By extracting features for each layer with different image sizes, global and local features of the bleeding area in the CT image can be extracted.

[0032] The computer 1 inputs the feature quantities extracted by the encoder of the U-Net 91 to the decoder of the U-Net 91. Based on the feature quantities input from the encoder of the U-Net 91, the computer 1 learns (trains) parameter information of the convolution layer using backpropagation or the like so that the output (prediction) of the decoder of the U-Net 91 approaches the CT image included in the training data. The parameter information includes the weight or bias of the filter (kernel), etc.

[0033] Specifically, the computer 1 performs skip connection processing to connect the feature map of the encoder to the feature map of the decoder at each layer. Based on the features input to the decoder, the computer 1 performs, for example, deconvolution processing in the decoder to reconstruct (predict) a segmentation map (probability map) of the same size as the input CT image.

[0034] The computer 1 uses a loss function to calculate the loss (error) between the reconstructed segmentation map and the segmentation map included in the training data. The computer 1 uses the calculated loss to perform backpropagation and update the parameter information of the U-Net 91.

[0035] In this way, the computer 1 performs the above processing on each CT image included in the training data to generate the U-Net 91. As a result, for example, the computer 1 can use the training data to learn the U-Net 91, and thereby construct a model that can output bleeding location information (brain parenchymal bleeding location, intraventricular hemorrhage location, edema location around the bleeding location, etc.) in the CT image when the CT image is input.

[0036] When the computer 1 acquires multiple sliced ​​CT images of the patient's brain, it inputs the acquired multiple CT images to the encoder of the U-Net 91. The computer 1 extracts features of each CT image through the encoder of the U-Net 91. The computer 1 inputs the features extracted by the encoder of the U-Net 91 to the decoder of the U-Net 91.

[0037] The computer 1 reconstructs a segmentation map using the decoder of the U-Net 91 based on the feature values ​​input to the decoder. The computer 1 assigns a label to each pixel that constitutes the CT image using semantic segmentation. That is, the computer 1 assigns a label (bleeding location information) to each pixel via the decoder of the U-Net 91, classifying each pixel into bleeding from the cerebral parenchyma, intraventricular hemorrhage, and edema surrounding the hemorrhage.

[0038] The computer 1 outputs the segmented CT image from U-Net 91 as output data based on bleeding location information that classifies bleeding from the cerebral parenchyma, intraventricular hemorrhage, and edema around the hemorrhage, with labels assigned to each pixel of the CT image.

[0039] As shown in the figure, for the segmented CT image, bleeding location information that is "bleeding from the cerebral parenchyma" is shown in green (left-downward hatching), bleeding location information that is "intraventricular bleeding" is shown in red (cross-hatching), and bleeding location information that is "edema around the bleeding" is shown in blue (right-downward hatching).

[0040] In this embodiment, an example has been described in which hemorrhage from the cerebral parenchyma, intraventricular hemorrhage, and peri-hemorrhagic edema are classified using the U-Net 91. However, the present invention is not limited to this. For example, in addition to hemorrhage from the cerebral parenchyma, intraventricular hemorrhage, and peri-hemorrhagic edema, subdural hematoma, subarachnoid hemorrhage, ependymal hemorrhage, etc. may also be classified.

[0041] The above-mentioned process for acquiring the features of CT images is not limited to U-net, and learning models constructed using U-Net++, SegNet, FCN (Fully Convolutional Network), RNN (Recurrent Neural Network), CNN (Convolutional Neural Network), R-CNN, Fast R-CNN, or Faster R-CNN may also be used.

[0042] Note that the features of CT images are not limited to learning models constructed by machine learning, and may be extracted using local feature extraction methods such as A-KAZE (Accelerated KAZE), SIFT (Scale Invariant Feature Transform), SURF (Speeded-Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), or HOG (Histograms of Oriented Gradients).

[0043] Furthermore, the volume of the hematoma (hematoma volume) can be calculated based on bleeding position information of the CT image. In this embodiment, the hematoma volume is the total hematoma volume (ICH volume) of all slices, but this is not limiting. For example, the hematoma volume may be calculated for each CT image.

[0044] Specifically, the computer 1 counts the number of voxels included in the area showing bleeding from the brain parenchyma based on the bleeding location information of each CT image. A voxel is the smallest unit constituting three-dimensional image data and represents a volume element in the form of a rectangular parallelepiped. The computer 1 calculates the volume of one voxel. The volume of a voxel is calculated from the length, width, and height (slice thickness) of the voxel.

[0045] The computer 1 calculates the volume of the hematoma in each CT image based on the number of counted voxels and the calculated volume of one voxel using the following formula (1): Volume of hematoma (ml) = Total number of voxels × Volume of one voxel (mm ３ ) x 10 -３ …(1)

[0046] That is, the volume of the hematoma in each CT image is the sum of the volumes of bleeding from the cerebral parenchyma. Then, based on the calculated volume of the hematoma in each CT image, the computer 1 calculates the total volume of the hematoma in all slices (the sum of the volumes of the hematoma in each CT image).

[0047] Next, the transformer 92 will be described. The transformer 92 is a model that outputs a value indicating the possibility of hemorrhage in each brain region. The transformer 92 is a deep learning model constructed by learning the relationship between each token in the CT image using a self-attention mechanism based on training data.

[0048] The training data input to the transformer 92 is a combination of data that matches the features of each CT image, which is a sequence image (sequence data) extracted by the encoder of U-Net 91, and the slice thickness of each CT image with the bleeding area (e.g., subcortex, putamen, thalamus, brainstem, or cerebellum) corresponding to the correct sequence image.

[0049] Sequence images are a series of CT images of slices of the same patient's brain arranged in order, ordered along a progression of time or space (slices through the brain), with each CT image having a relationship to the CT images before and after it.

[0050] The training data may be data that has been created manually. For example, a specialist (e.g., a radiologist) may manually annotate bleeding regions on CT images, and the annotated bleeding regions may be labeled (correct answers) to create training data.

[0051] The transformer 92 includes an encoder 921 and a decoder 922. The learning process of the transformer 92 is performed by learning (updating) parameter information included in the transformer 92 by backpropagation (error backpropagation method) so as to minimize loss, for example, using a gradient obtained from a loss function that calculates the difference between output data and correct data based on a large amount of training data.

[0052] The computer 1 performs the above learning process to generate the transformer 92. As a result, for example, the computer 1 can use the training data to train the transformer 92, thereby constructing a model that can output information about multiple bleeding regions when the feature amounts and slice thicknesses of each CT image are input to the transformer 92.

[0053] The information about the bleeding region in the brain parenchyma includes a value (probability value) indicating the possibility of bleeding in each bleeding region of the cerebral subcortex, putamen, thalamus, brainstem, and cerebellum, or the presence or absence of bleeding in each bleeding region (e.g., 0: no bleeding, 1: bleeding). Note that the bleeding region is not limited to the cerebral subcortex, putamen, thalamus, brainstem, and cerebellum, but may include two or more bleeding regions among multiple bleeding regions. Note that the bleeding region may also include ventricles and the like other than the brain parenchyma.

[0054] When the computer 1 acquires the feature quantities of a series of CT images extracted by the encoder of the U-Net 91, it inputs the feature quantities of each acquired CT image and the slice thickness to a transformer 92, which outputs a value indicating the possibility of bleeding in each bleeding region. The slice thickness is the thickness of the tomographic image obtained in the CT examination, i.e., a numerical value (e.g., 5 mm or 8 mm) indicating the thickness of the brain tomography.

[0055] Specifically, the computer 1 inputs the features of each CT image patch (token) of a predetermined size (e.g., 16x16) extracted by the encoder of the U-Net 91 and the slice thickness of each CT image into the transformer 92, and vectorizes the features and slice thickness of each CT image patch.

[0056] The computer 1 inputs the vectorized feature vectors of each CT image to the encoder 921 of the transformer 92. Alternatively, the computer 1 may input the position vectors of the feature amounts of each CT image. The computer 1 extracts features of the feature vectors using the encoder 921 and inputs the extracted features to the decoder 922.

[0057] The computer 1 utilizes a self-attention mechanism and an encoder-decoder attention mechanism to aggregate related information based on the input features in a decoder 922. Based on the information aggregated in the decoder 922, the computer 1 calculates a value (probability value) indicating the possibility of bleeding in each bleeding region in the CT image, including the cerebral subcortex, putamen, thalamus, brainstem, and cerebellum.

[0058] The computer 1 normalizes the calculated probability values ​​by applying an activation function (e.g., a sigmoid function) via the decoder 922. The computer 1 outputs the normalized probability values ​​indicating the possibility of bleeding in each hemorrhage region. As shown in the figure, the probability values ​​for the hemorrhage regions "subcortical region," "putamen," "thalamus," "brainstem," and "cerebellum" for the multiple sliced ​​CT images are "0.62," "0.99," "0.89," "0.32," and "0.00," respectively.

[0059] Although the slice thickness is used in the above-described process, this is not limiting. For example, the computer 1 may input only the feature quantities of the patches of the CT image extracted by the encoder of the U-Net 91 to the transformer 92. In this case, the computer 1 vectorizes the feature quantities of the patches. The computer 1 inputs the vectorized feature vectors to the encoder 921. As in the above-described process, the computer 1 calculates a value indicating the possibility of bleeding in each bleeding region using the encoder 921 and the decoder 922.

[0060] The transformer 92 may be configured using a neural network such as LSTM (Long Short-Term Memory) instead of a transformer model.

[0061] In the above-described processing, the computer 1 trains the U-Net 91 and the transformer 92 included in the bleeding region output model 171 separately, but this is not limited to this, and the U-Net 91 and the transformer 92 may also be trained simultaneously.

[0062] The computer 1 displays on the screen the information about the bleeding region output from the bleeding region output model 171 and the calculated volume of the hematoma.

[0063] 3 is a flowchart showing the processing steps for generating the U-Net 91. The control unit 11 of the computer 1 acquires training data, for example, in which CT images of the brain are associated with annotation masks for the CT images, from the storage unit 12 or the mass storage unit 17 (step S101). The control unit 11 inputs the CT images included in the acquired training data to the encoder of the U-Net 91 (step S102). The control unit 11 extracts feature quantities from each CT image through the encoder of the U-Net 91 (step S103).

[0064] The control unit 11 inputs the features extracted by the encoder of the U-Net 91 to the decoder of the U-Net 91 (step S104). Based on the input features, the control unit 11 learns parameter information of the convolution layer using backpropagation or the like so that the output of the decoder of the U-Net 91 approaches the CT image included in the training data (step S105). The control unit 11 generates the U-Net 91 by performing a learning process in the decoder of the U-Net 91 (step S106). The control unit 11 then ends the process.

[0065] 4 is a flowchart showing the processing steps for generating the transformer 92. The control unit 11 of the computer 1 acquires, for example, training data from the storage unit 12 or the mass storage unit 17, which includes the feature values ​​of each CT image, which is a sequence image extracted by the encoder of the U-Net 91, and the slice thickness of each CT image, corresponding to the bleeding region corresponding to the sequence image (step S111). The control unit 11 inputs the feature values ​​and slice thickness of each CT image included in the acquired training data to the encoder 921 of the transformer 92 (step S112).

[0066] Based on the input training data, the control unit 11 uses the gradient obtained from a loss function that calculates the difference between output data and correct data via the encoder 921 and decoder 922 to learn parameter information included in the transformer 92 by backpropagation so as to minimize loss (step S113). The control unit 11 generates the transformer 92 by performing a learning process (step S114). The control unit 11 then terminates the process.

[0067] 5 is a flowchart showing the processing procedure for outputting bleeding location information using U-Net 91. The control unit 11 of the computer 1 acquires multiple sliced ​​CT images of the patient's brain from the storage unit 12 or the mass storage unit 17 (step S121). The control unit 11 may also acquire multiple CT images from an external information processing device via the communication unit 13.

[0068] The control unit 11 inputs the acquired multiple CT images to the encoder of the U-Net 91 (step S122). The control unit 11 extracts features of each CT image through the encoder of the U-Net 91 (step S123). The control unit 11 inputs the extracted features of each CT image to the decoder of the U-Net 91 (step S124). The control unit 11 performs semantic segmentation based on the features input to the decoder of the U-Net 91 (step S125).

[0069] The control unit 11 outputs the bleeding location information, in which each pixel of the CT image is labeled and classified into bleeding from the cerebral parenchyma, intraventricular hemorrhage, and edema around the hemorrhage, from the U-Net 91 (step S126). For example, the control unit 11 outputs a segmented CT image indicating the bleeding location information.

[0070] The control unit 11 calculates the volume of the hematoma based on the output bleeding location information (step S127). Specifically, the control unit 11 counts the number of voxels included in the area indicating bleeding from the cerebral parenchyma, intraventricular hemorrhage, and edema around the hemorrhage based on the bleeding location information in each CT image. The control unit 11 calculates the volume of one voxel (length x width x slice thickness). Based on the counted number of voxels and the calculated volume of one voxel, the control unit 11 calculates the volume of the hematoma in each CT image using the above-mentioned formula (1). The control unit 11 calculates the sum of the calculated hematoma volumes in each CT image as the total hematoma volume.

[0071] The control unit 11 displays the bleeding position information output from the U-Net 91 and the calculated volume of the hematoma on the display unit 15 (step S128). The control unit 11 then ends the process.

[0072] 6 is a flowchart showing the processing procedure for outputting information about a bleeding region using the transformer 92. The control unit 11 of the computer 1 inputs the feature amounts of each CT image, which is a sequence image extracted by the encoder of the U-Net 91, and the slice thickness of each CT image to the transformer 92 (step S131). The control unit 11 then vectorizes the feature amounts and slice thickness of each CT image (step S132).

[0073] The control unit 11 inputs the vectorized feature vectors of each CT image to the encoder 921 of the transformer 92 (step S133). The control unit 11 extracts features of the feature vectors using the encoder 921 (step S134) and inputs the extracted features to the decoder 922 (step S135).

[0074] The control unit 11 uses the self-attention mechanism and the encoder-decoder attention mechanism to calculate a probability value indicating the possibility of bleeding in each bleeding region in the CT image, including the cerebral subcortex, putamen, thalamus, brainstem, and cerebellum (step S136). The control unit 11 displays information about the bleeding region, including the calculated probability value indicating the possibility of bleeding in each bleeding region, on the display unit 15 (step S137). The control unit 11 then ends the processing.

[0075] According to this embodiment, it is possible to automatically output information about the bleeding region, which serves as the basis for determining a treatment plan for intracerebral hemorrhage, using a bleeding region output model 171 based on multiple sliced ​​CT images.

[0076] (Embodiment 2) Embodiment 2 relates to a form in which information regarding the growth of a hematoma within a predetermined time period is output using artificial intelligence based on information regarding the bleeding region, feature quantities of the bleeding region, and clinical information of the patient. Note that a description of content that overlaps with embodiment 1 will be omitted.

[0077] 7 is a block diagram showing an example of the configuration of the computer 1 according to the second embodiment. The same reference numerals are used to designate the same components as those in FIG. 1, and the description thereof will be omitted. The large-capacity storage unit 17 includes a hematoma growth output model (second learning model) 172.

[0078] The hematoma growth output model 172 is an output device (estimator) that outputs information about the growth of the hematoma within a specified period of time (e.g., 24 or 48 hours) based on information about the bleeding area, features of the bleeding area, and clinical information about the patient, and is a trained model generated by machine learning.

[0079] For example, in the case of intracerebral hemorrhage for which surgical treatment was not performed (conservative treatment was selected) by a doctor with no experience in stroke treatment, the presence or absence of subsequent hematoma growth influences the treatment policy. In this embodiment, the computer 1 uses the hematoma growth output model 172 to output information regarding hematoma growth within a predetermined time period.

[0080] The computer 1 acquires a segmentation image corresponding to the CT image output from the decoder of U-Net 91 included in the hemorrhage region output model 171. The computer 1 calculates the distance from the surface of the cerebral cortex to the hematoma (ICH Distance from Surface Cortical) from the acquired segmentation image.

[0081] In the process of calculating the distance from the surface of the cerebral cortex to the hematoma, for example, the computer 1 identifies the surface of the cerebral cortex and bleeding position information based on the segmentation image. The computer 1 then calculates the distance between pixels based on the identified surface of the cerebral cortex and bleeding position information, thereby obtaining the shortest distance from the surface of the cerebral cortex to the hematoma.

[0082] The computer 1 determines whether or not surgery is appropriate based on the identified bleeding location information, the calculated distance from the surface of the cerebral cortex to the hematoma, the total volume of the hematoma, or the length of the hematoma, etc. The indication for surgery is reference information that indicates the appropriateness of surgery in accordance with the guidelines.

[0083] Specifically, the computer 1 compares the identified bleeding location information, the calculated distance from the cerebral cortical surface to the hematoma, the total hematoma volume, or the hematoma length with predetermined stroke treatment guidelines (e.g., guidelines from Japan or other countries) to determine whether or not surgery is indicated. For example, the computer 1 may determine whether or not surgery is indicated if the maximum diameter of the cerebral hemorrhage in the "cerebellum" is equal to or greater than a predetermined threshold (e.g., 3 cm). Alternatively, the computer 1 may determine whether or not surgery is indicated if the total hematoma volume in the "subcortical" hemorrhage is equal to or greater than a predetermined volume (e.g., 30 ml) or if the distance from the cerebral cortical surface to the hematoma is equal to or less than a predetermined distance (e.g., 1 cm). The determination of whether or not surgery is indicated may be made by a physician or other professional.

[0084] If surgery is not indicated, the computer 1 uses the hematoma growth output model 172 to output information regarding the growth of the hematoma within a specified time period as output data based on input data including information about the bleeding area, features of the bleeding area, and clinical information about the patient.

[0085] 8 is an explanatory diagram illustrating the hematoma growth output model 172. The hematoma growth output model 172 is used as a program module that is part of artificial intelligence software. The hematoma growth output model 172 is a pre-constructed output device that receives information about the bleeding region, feature quantities of the bleeding region, and clinical information about the patient as input, and outputs information about the growth of the hematoma within a predetermined time period.

[0086] The information about the bleeding region is output from the bleeding region output model 171 and includes the presence or absence of bleeding (e.g., 0: no bleeding, 1: bleeding) in each bleeding region (subcortical region, putamen, thalamus, brainstem, and cerebellum). Note that the information about the bleeding region may include a probability value indicating the possibility of bleeding in each bleeding region, or a level indicating the growth of the hematoma.

[0087] The feature amount of the hemorrhage region includes, for example, the presence or absence of intraventricular hemorrhage (Hematoma Subtype Semantic Segmentation) or radiomics feature amount. The presence or absence of intraventricular hemorrhage is obtained from hemorrhage location information output from the U-Net 91 included in the hemorrhage region output model 171 based on multiple sliced ​​CT images. Specifically, if the computer 1 has not identified the intraventricular hemorrhage location based on the hemorrhage location information, it determines that there is no intraventricular hemorrhage. Alternatively, if the computer 1 has identified the intraventricular hemorrhage location, it determines that there is intraventricular hemorrhage. The presence or absence of intraventricular hemorrhage may be indicated, for example, as "no bleeding: 0, bleeding: 1."

[0088] Radiomic features are a large number of quantitative features extracted from medical images such as CT images, and include, for example, the total intracerebral hemorrhage volume of all slices, major axis length, statistics (pixel value statistics), texture, etc.

[0089] The total hematoma volume of all slices is obtained by the calculation process for the total hematoma volume in embodiment 1. The length of the hematoma is the longest diameter of the hematoma. For example, from the segmentation images corresponding to the multiple CT images output from the decoder of U-Net 91, computer 1 identifies the segmentation image that is the widest slice (cross-section) based on the vertical and horizontal diameters of the hemorrhage region. Computer 1 obtains the length of the hematoma by calculating the longest diameter of the hemorrhage region shown in the identified segmentation image.

[0090] Statistics are features that represent the distribution of pixel intensities in an image, and include the mean, median, standard deviation, skewness, and kurtosis. Texture is a feature that represents the pattern or relationship of pixel intensities in an image, i.e., the spatial distribution of pixel values. Texture includes Gray Level Co-occurrence Matrix (GLCM), Gray Level Run Length Matrix (GLRLM), Gray Level Size Zone Matrix (GLSZM), and the like.

[0091] In this embodiment, the feature quantities of the hemorrhagic region will be exemplified by using the presence or absence of intraventricular hemorrhage, the length of the hematoma, and the total volume of the hematoma.

[0092] The patient's clinical information includes, for example, the patient's age, sex, medication information, time from onset to hospitalization, symptoms, data on treatment, body temperature, blood pressure, treatment plan or treatment progress, etc.

[0093] The hematoma growth output model 172 is generated (constructed) by learning the features of a large amount of training data collected from medical institutions, for example, using LightGBM based on a Gradient Boosting Decision Tree (GBDT).

[0094] The training data is a combination of information about the hemorrhagic region, features of the hemorrhagic region, and clinical information about the patient, associated with the presence or absence of hematoma growth within a predetermined time period (e.g., 24 or 48 hours) (e.g., 0: no growth, 1: growth). Specifically, the training data is a combination of clinical information about the patient, the presence or absence of bleeding in each hemorrhagic region (subcortical region, putamen, thalamus, brainstem, and cerebellum), the presence or absence of intraventricular hemorrhage, the length of the hematoma, and the total hematoma volume, and the presence or absence of hematoma growth within a predetermined time period (e.g., 24 hours). The training data is collected from a large amount of actual clinical data about patients obtained in clinical settings. Note that the length of the hematoma is not required in this embodiment.

[0095] The computer 1 generates one weak classifier (decision tree) using training data, and then sequentially generates the next weak classifier based on the error (more precisely, the residual) between the predicted value and the actual measurement value using the generated weak classifier. The computer 1 sequentially generates weak classifiers by referring to the gradient of a loss function defined by the error between the predicted value and the actual measurement value, so as to take into account the learning result of the previous weak classifier, and generates a final classifier, i.e., a hematoma growth output model 172.

[0096] In addition to the above-described learning methods, other ensemble learning methods may be used, such as bagging, which generates multiple weak classifiers in parallel. The hematoma growth output model 172 may also be generated using a learning method other than ensemble learning. For example, the computer 1 may generate the hematoma growth output model 172 using a random forest method. Specifically, the computer 1 randomly selects feature items to be used for classification (classification) at non-terminal nodes based on learning data sampled from training data, thereby creating multiple decision trees with low correlation, and then generates the hematoma growth output model 172 using the multiple decision trees.

[0097] Note that dimension reduction may be applied to the input data input to the hematoma growth output model 172 .

[0098] When the computer 1 acquires information about the hemorrhagic region, the presence or absence of intraventricular hemorrhage, the feature values ​​of the hemorrhagic region including the length of the hematoma and the total hematoma volume, and the patient's clinical information, the computer 1 inputs the acquired information about the hemorrhagic region, the feature values ​​of the hemorrhagic region, and the patient's clinical information into the generated hematoma growth output model 172, and outputs a probability value of hematoma growth within a predetermined time. As shown in the figure, the hematoma growth output model 172 outputs a prediction result of "Probability value of hematoma growth <within the next 24 hours>: 0.87."

[0099] In this embodiment, the hematoma growth output model 172 is described as being LightGBM, but in addition to LightGBM, boosting methods such as XGBoost or CatBoost may also be used. The hematoma growth output model 172 is not limited to a decision tree, and may be a trained model constructed with any learning algorithm other than a decision tree, such as an SVM (Support Vector Machine), a Bayesian network, a regression tree, or a neural network (e.g., CNN or R-CNN).

[0100] 9 is a flowchart showing the processing steps for generating the hematoma growth output model 172. The control unit 11 of the computer 1 acquires, from the storage unit 12 or the mass storage unit 17, training data in which information about the bleeding region, feature quantities of the bleeding region, and clinical information about the patient are associated with the presence or absence of hematoma growth within a predetermined time period (e.g., 24 hours) (e.g., 0: no growth, 1: growth) (step S141).

[0101] The control unit 11 uses the acquired training data to perform a learning process for the hematoma growth output model 172 (step S142). Specifically, the control unit 11 generates one weak classifier using the training data, and then sequentially generates the next weak classifier based on the error between the predicted value by the generated weak classifier and the actual measurement value. The control unit 11 sequentially generates weak classifiers by referring to the gradient of the loss function defined by the error between the predicted value and the actual measurement value, and generates a final classifier.

[0102] The control unit 11 generates the final classifier as the hematoma growth output model 172 (step S143), and then ends the process.

[0103] 10 is a flowchart showing the processing procedure for outputting information about hematoma growth. The control unit 11 of the computer 1 acquires a segmentation image corresponding to the CT image output from the decoder of U-Net 91 included in the hemorrhage region output model 171 (step S151). Based on the acquired segmentation image, the control unit 11 identifies the surface of the cerebral cortex and bleeding location information (step S152).

[0104] The control unit 11 calculates the distance (shortest distance) from the surface of the cerebral cortex to the hematoma based on the identified surface of the cerebral cortex and the bleeding location information (step S153).The control unit 11 compares the identified bleeding location information, the calculated distance from the surface of the cerebral cortex to the hematoma, the total hematoma volume, or the length of the hematoma with the indications for surgery based on predetermined stroke guidelines, and determines whether surgery is indicated (step S154).

[0105] If the control unit 11 determines that surgery is indicated (YES in step S154), it ends the process. If the control unit 11 determines that surgery is not indicated (NO in step S154), it acquires information about the hemorrhagic region obtained by the process in Fig. 6 (step S155). The control unit 11 acquires feature quantities of the hemorrhagic region, including the presence or absence of intraventricular hemorrhage, the total hematoma volume, the length of the hematoma, pixel value statistics, texture, etc. (step S156).

[0106] Specifically, the control unit 11 acquires the presence or absence of intraventricular hemorrhage (for example, 0: no bleeding, 1: bleeding) included in the bleeding location information output from the U-Net 91. The control unit 11 acquires the total hematoma volume obtained by the processing of step S127 in FIG.

[0107] The control unit 11 identifies the segmentation image that is the widest slice based on the vertical and horizontal diameters of the bleeding region from the segmentation images corresponding to the multiple CT images output from the decoder of the U-Net 91. The computer 1 obtains the length of the hematoma by calculating the longest diameter of the bleeding region shown in the identified segmentation image.

[0108] Based on the segmentation image, the control unit 11 extracts pixel value statistics, texture, etc., by machine learning such as U-Net91, SegNet, or CNN. Note that the pixel value statistics, texture, etc. may also be extracted using a local feature extraction method such as A-KAZE, SIFT, or SURF.

[0109] The control unit 11 acquires the patient's clinical information (such as age, sex, medication information, symptoms, or progress of treatment) from the storage unit 12 or the mass storage unit 17 (step S157). The control unit 11 may also acquire the patient's clinical information from an external information processing device.

[0110] The control unit 11 inputs the acquired information about the bleeding region, the feature values ​​of the bleeding region, and the patient's clinical information into the hematoma growth output model 172 (step S158), and outputs information about hematoma growth within a predetermined time period (e.g., 24 or 48 hours) (step S159). The control unit 11 displays the information about hematoma growth output from the hematoma growth output model 172 on the display unit 15 (step S160). The control unit 11 then terminates the process.

[0111] According to this embodiment, it is possible to output information regarding the growth of a hematoma within a predetermined time period using the hematoma growth output model 172 .

[0112] According to this embodiment, by predicting hematoma growth using the hematoma growth output model 172, doctors with no experience in stroke treatment can quickly present appropriate treatment plans and improve the prognosis of patients with intracerebral hemorrhage.

[0113] Although the above-described embodiments have been described using examples of brain CT images, the present invention is not limited thereto. For example, the learning model in embodiment 1 may be an output device in which a neural network is constructed, which inputs multiple sliced ​​lung CT images and outputs information about diseased lung regions (e.g., lung fields, pulmonary arteries, blood vessels, or pleura). In this case, the computer 1 inputs multiple lung CT images into the learning model, thereby outputting information about diseased lung regions.

[0114] Alternatively, the learning model in embodiment 1 may be an output device in which a neural network has been constructed that receives a plurality of sliced ​​CT images of the stomach as input and outputs information related to diseased regions of the stomach (e.g., the stomach wall, the internal tissue of the stomach, or surrounding tissues). In this case, the computer 1 inputs a plurality of CT images of the stomach into the learning model, and outputs information related to diseased regions of the stomach.

[0115] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0116] The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used.

[0117] DESCRIPTION OF SYMBOLS 1 Information processing device (computer) 11 Control unit 12 Memory unit 13 Communication unit 14 Input unit 15 Display unit 16 Reading unit 17 Large-capacity memory unit 171 Bleeding region output model (learning model) 91 U-Net 92 Transformer model (transformer) 921 Encoder 922 Decoder 172 Hematoma growth output model (second learning model) 1a Portable storage medium 1b Semiconductor memory 1P Control program

Claims

1. A program that causes a computer to acquire multiple sliced CT (Computed Tomography) images of the brain, input the acquired multiple CT images into a learning model that outputs information about multiple bleeding areas when multiple CT images are input, and then output information about bleeding areas.

2. The program according to claim 1, wherein the learning model outputs values indicating the possibility of bleeding in each region including the subcortical region, putamen, thalamus, brainstem, and cerebellum as information about the bleeding region.

3. The program according to claim 1 or 2, wherein the learning model includes a transformer model, extracts features from each CT image, and inputs the extracted features from each CT image into the transformer model, thereby outputting a value indicating the possibility of bleeding in each bleeding region.

4. The program according to claim 3, wherein the extracted feature values of each CT image and slice thickness are input to the transformer model, and a value indicating the possibility of bleeding in each bleeding region is output.

5. The program according to claim 4, wherein the learning model includes a U-Net model that, when a CT image is input, outputs bleeding location information including the location of cerebral parenchymal hemorrhage, the location of intraventricular hemorrhage, and the location of edema around the bleeding location in the CT image, and the features of the CT image output from the encoder of the U-Net model are input to the transformer model.

6. The program according to claim 5, wherein the volume of the hematoma is determined based on bleeding position information of each CT image.

7. A program as described in claim 1 or 2, which outputs information regarding the growth of a hematoma within a predetermined time period when information regarding the bleeding area, features of the bleeding area, and clinical information of a patient is input into a second learning model that outputs information regarding the growth of a hematoma within a predetermined time period.

8. The program described in claim 7, which outputs the probability of hematoma growth within 24 hours by inputting information about the bleeding area, features of the bleeding area, and clinical information about the patient into the second learning model.

9. The program according to claim 7, wherein the feature of the bleeding region is a radiomics feature including the length of the blood mass.

10. An information processing method that acquires multiple sliced CT images of the brain, inputs the acquired multiple CT images into a learning model that outputs information about multiple bleeding areas when multiple CT images are input, and outputs information about bleeding areas.

11. An information processing device having a control unit, wherein the control unit acquires multiple sliced CT images of the brain, and outputs information regarding multiple bleeding areas by inputting the acquired multiple CT images into a learning model that outputs information regarding multiple bleeding areas when multiple CT images are input.

Images