Medical information processing device, medical information processing method, and medical information processing program

Abstract

To start post-processing earlier. [Solution] The medical information processing device according to the embodiment comprises a first acquisition unit, a second acquisition unit, an aggregation unit, and a determination unit. The first acquisition unit sequentially acquires medical information. The second acquisition unit acquires the likelihood for each classification result of the medical information sequentially acquired by the first acquisition unit in a trained model for classifying classification targets included in the medical information. The aggregation unit aggregates the likelihoods acquired by the second acquisition unit. The determination unit determines whether or not to perform post-processing based on the aggregation result by the aggregation unit.

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a medical information processing apparatus, a medical information processing method, and a medical information processing program.

Background Art

[0002] Conventionally, a medical information processing apparatus such as an X-ray CT apparatus generates CT image data by performing reconstruction processing on data obtained by scanning a subject. Then, the X-ray CT apparatus transmits the generated CT image data to an apparatus such as a PACS (Picture Archiving and Communication System).

[0003] However, the medical information processing apparatus takes a lot of time for the reconstruction processing and also takes a lot of time for transmitting the CT image data. Therefore, the analysis apparatus takes a long time until it starts the analysis processing on the CT image data. Also, medical staff take a long time until they start reading the CT image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to accelerate the timing of starting post-processing. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. It is also possible to position other problems corresponding to the respective effects of each configuration shown in the embodiments described later as other problems.

Means for Solving the Problems

[0006] The medical information processing device according to the embodiment comprises a first acquisition unit, a second acquisition unit, an aggregation unit, and a determination unit. The first acquisition unit sequentially acquires medical information. The second acquisition unit acquires the likelihood for each classification result of the medical information sequentially acquired by the first acquisition unit in a trained model for classifying classification targets included in the medical information. The aggregation unit aggregates the likelihoods acquired by the second acquisition unit. The determination unit determines whether or not to perform post-processing based on the aggregation results by the aggregation unit. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows an example of a medical information processing system according to the first embodiment. [Figure 2] Figure 2 shows an example of the configuration of an X-ray CT apparatus according to the first embodiment. [Figure 3] Figure 3 shows an example of how to obtain likelihood using a classification model. [Figure 4] Figure 4 shows an example of an accumulation method for accumulating likelihoods for each classification target. [Figure 5] Figure 5 shows an example of an operation input image. [Figure 6] Figure 6 shows an example of an early activation process performed by an X-ray CT apparatus according to the first embodiment. [Figure 7] Figure 7 shows an example of a classification target display transition with a high likelihood. [Modes for carrying out the invention]

[0008] The medical information processing apparatus, medical information processing method, and medical information processing program according to this embodiment will be described below with reference to the drawings. In the following embodiments, parts with the same reference numerals perform similar operations, and redundant explanations will be omitted as appropriate.

[0009] (First Embodiment) Figure 1 shows an example of a medical information processing system 1 according to the first embodiment. The medical information processing system 1 includes an X-ray CT (Computed Tomography) device 10, a medical image storage device 20, and an operation terminal 30. The X-ray CT device 10, the medical image storage device 20, and the operation terminal 30 are connected to each other via a network 40. The medical information processing system 1 shown in Figure 1 has one X-ray CT device 10, one medical image storage device 20, and one operation terminal 30, but the number of each can be changed as desired. Furthermore, the medical information processing system 1 may have devices or systems not shown in Figure 1.

[0010] The X-ray CT scanner 10 takes images of the subject P (see Figure 2) and generates CT image data.

[0011] The medical image storage device 20 stores medical image data generated by the X-ray CT scanner 10. For example, the medical image storage device 20 can be implemented using computer equipment such as a server or workstation, such as a PACS (Picture Archiving and Communication System).

[0012] The operating terminal 30 is a device operated by medical professionals. For example, the operating terminal 30 can be implemented using computer equipment such as a personal computer or a workstation.

[0013] Next, the X-ray CT scanner 10 will be described in detail.

[0014] Figure 2 shows an example of the configuration of an X-ray CT apparatus 10 according to the first embodiment. The X-ray CT apparatus 10 includes a cradle 110, a patient bed 130, and a console 140.

[0015] Here, in FIG. 2, the rotation axis of the rotation frame 113 or the longitudinal direction of the top plate 133 of the bed device 130 in the non-tilted state is defined as the Z-axis direction. Also, the axial direction orthogonal to the Z-axis direction and horizontal with respect to the floor surface is defined as the X-axis direction. Further, the axial direction orthogonal to the Z-axis direction and the X-axis direction and perpendicular to the floor surface is defined as the Y-axis direction. Note that FIG. 2 shows the gantry device 110 drawn from multiple directions for the purpose of explanation, and shows the case where the X-ray CT apparatus 10 has one gantry device 110.

[0016] The gantry device 110 includes an X-ray tube 111, an X-ray detector 112, a rotation frame 113, an X-ray high voltage device 114, a control device 115, a wedge 116, a collimator 117, and a DAS (Data Acquisition System) 118. Note that the gantry device 110 is also referred to as a gantry.

[0017] The X-ray tube 111 is a vacuum tube having a cathode (filament) that generates thermoelectrons and an anode (target) that generates X-rays upon receiving the collision of thermoelectrons. The X-ray tube 111 generates X-rays to irradiate the subject P by irradiating thermoelectrons from the cathode toward the anode by applying a high voltage from the X-ray high voltage device 114. That is, the X-ray tube 111 generates X-rays corresponding to the tube voltage and tube current applied from the X-ray high voltage device 114. For example, the X-ray tube 111 includes a rotating anode type X-ray tube that generates X-rays by irradiating thermoelectrons to the rotating anode.

[0018] The X-ray detector 112 detects the X-rays irradiated from the X-ray tube 111 and passing through the subject P, and outputs a signal corresponding to the detected X-ray dose to the DAS 118. The X-ray detector 112 has, for example, a plurality of detector element arrays in which a plurality of detector elements are arranged in the channel direction (channel direction) along an arc centered on the focal point of the X-ray tube 111. The X-ray detector 112 has, for example, a structure in which a plurality of detector element arrays in which a plurality of detector elements are arranged in the channel direction are arranged in the column direction (slice direction, row direction).

[0019] For example, the X-ray detector 112 is an indirect conversion type detector having a grid, a scintillator array, and an optical sensor array. The scintillator array has a plurality of scintillators. Each scintillator has a scintillator crystal that outputs light in an amount of photons corresponding to the incident X-ray dose. The grid is disposed on the X-ray incident side surface of the scintillator array and has an X-ray shielding plate that absorbs scattered X-rays. Note that the grid may also be called a collimator (one-dimensional collimator or two-dimensional collimator). The optical sensor array has a function of converting the amount of light from the scintillator into an electrical signal, and has, for example, optical sensors such as photodiodes. Note that the X-ray detector 112 may be a direct conversion type detector having a semiconductor element that converts the incident X-ray into an electrical signal.

[0020] The rotating frame 113 is an annular frame that oppositely supports the X-ray tube 111 and the X-ray detector 112 and rotates the X-ray tube 111 and the X-ray detector 112 by the control device 115. For example, the rotating frame 113 is a casting made of aluminum. Note that in addition to the X-ray tube 111 and the X-ray detector 112, the rotating frame 113 can further support an X-ray high voltage device 114, a wedge 116, a collimator 117, a DAS 118, and the like. Further, the rotating frame 113 can further support various configurations not shown in FIG. 2. The various configurations supported by the rotating frame 113 will be described later. Note that the rotating frame 113 is also referred to as a rotating base, a rotating body, or the like. Also, in the gantry device 110, the rotating frame 113 and the portions that rotate and move together with the rotating frame 113 are also referred to as a rotating part.

[0021] The X-ray high-voltage device 114 has an electrical circuit including a transformer and a rectifier, and includes a high-voltage generator that generates a high voltage to be applied to the X-ray tube 111, and an X-ray control device that controls the output voltage according to the X-rays generated by the X-ray tube 111. In other words, the X-ray high-voltage device 114 controls the tube voltage and tube current applied to the X-ray tube 111. The high-voltage generator may be of the transformer type or the inverter type. The X-ray high-voltage device 114 may be mounted on the rotating frame 113 or on a fixed frame (not shown).

[0022] The control device 115 includes a processing circuit with a CPU (Central Processing Unit), etc., and a drive mechanism such as a motor and actuator. The control device 115 receives input signals from the input interface 143 and controls the operation of the frame device 110 and the bed device 130. For example, the control device 115 controls the rotation of the rotating frame 113, the tilt of the frame device 110, and the operation of the bed device 130 and the top plate 133. The control device 115 may be installed on the frame device 110 or on the console device 140.

[0023] The wedge 116 is a filter used to adjust the amount of X-rays irradiated from the X-ray tube 111. Specifically, the wedge 116 is a filter that transmits and attenuates the X-rays irradiated from the X-ray tube 111 so that the distribution of X-rays irradiated from the X-ray tube 111 to the subject P becomes a predetermined distribution. For example, the wedge 116 is a wedge filter or a bow-tie filter, which is a filter made of aluminum or the like processed to have a predetermined target angle and thickness.

[0024] The collimator 117 is a lead plate or the like used to narrow the irradiation range of X-rays that have passed through the wedge 116, and a slit is formed by combining multiple lead plates or the like. The collimator 117 is sometimes called an X-ray diaphragm. In Figure 2, the wedge 116 is shown to be placed between the X-ray tube 111 and the collimator 117, but the collimator 117 may also be placed between the X-ray tube 111 and the wedge 116. In this case, the wedge 116 transmits and attenuates the X-rays that are irradiated from the X-ray tube 111 and whose irradiation range has been limited by the collimator 117.

[0025] The DAS118 collects X-ray signals detected by each detection element of the X-ray detector 112. For example, the DAS118 has an amplifier that amplifies the electrical signals output from each detection element and an A / D converter that converts the electrical signals into digital signals, thereby generating detection data.

[0026] The data generated by DAS118 is transmitted via optical communication from a transmitter having a light-emitting diode (LED) on the rotating frame 113 to a receiver having a photodiode located on the non-rotating part of the mounting device 110 (e.g., a fixed frame, which is not shown in Figure 2), and then transferred to the console device 140. Here, the non-rotating part is, for example, a fixed frame that rotatably supports the rotating frame 113. Note that the method of transmitting data from the rotating frame 113 to the non-rotating part of the mounting device 110 is not limited to optical communication; any non-contact data transmission method or a contact-type data transmission method may be used.

[0027] The examination bed apparatus 130 is a device for placing and moving the subject P to be photographed, and comprises a base 131, an examination bed drive device 132, a tabletop 133, and a support frame 134. The base 131 is a housing that supports the support frame 134 so that it can move vertically. The examination bed drive device 132 is a drive mechanism that moves the tabletop 133 on which the subject P is placed in the direction of the long axis of the tabletop 133, and includes a motor and actuator, etc. The tabletop 133, which is provided on the upper surface of the support frame 134, is a plate on which the subject P is placed. In addition to moving the tabletop 133, the examination bed drive device 132 may also move the support frame 134 in the direction of the long axis of the tabletop 133.

[0028] The console device 140 includes a memory 141, a display 142, an input interface 143, a network interface 144, and a processing circuit 145. Although the console device 140 is described separately from the mounting device 110, the mounting device 110 may include the console device 140 or some of its components.

[0029] Memory 141 can be implemented using, for example, semiconductor memory elements such as RAM (Random Access Memory) or flash memory, a hard disk, or an optical disc. Memory 141 stores, for example, projection data or CT image data. Alternatively, memory 141 can store programs for circuits included in the X-ray CT scanner 10 to perform various functions. Memory 141 may also be implemented using a group of servers (cloud) connected to the X-ray CT scanner 10 via a network 40.

[0030] The display 142 displays various types of information. For example, the display 142 may display various images generated by the processing circuit 145, or it may display a GUI (Graphical User Interface) to receive various operations from the operator. For example, the display 142 may be an LCD display or a CRT (Cathode Ray Tube) display. The display 142 may be a desktop type, or it may be composed of the console device 140 main unit and a tablet terminal or the like that can communicate wirelessly.

[0031] The input interface 143 can be implemented using a mouse, keyboard, trackball, switch, button, joystick, touchpad for input operations by touching the operating surface, touchscreen with integrated display screen and touchpad, non-contact input circuit using optical sensor, audio input circuit, etc. The input interface 143 may also be provided on the mounting device 110. Furthermore, the input interface 143 may consist of the console device 140 main unit and a tablet terminal or the like that can communicate wirelessly. Moreover, the input interface 143 is not limited to those equipped with physical operating components such as a mouse or keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the console device 140 and outputs these electrical signals to a processing circuit 145 is also included as an example of the input interface 143.

[0032] The NW interface 144 is connected to the processing circuit 145 and controls the transmission and communication of various data between the device connected via the network 40. For example, the NW interface 144 can be implemented by a network card, network adapter, NIC (Network Interface Controller), etc.

[0033] The processing circuit 145 controls the operation of the entire X-ray CT apparatus 10. For example, the processing circuit 145 performs system control function 4501, preprocessing function 4502, medical information acquisition function 4503, likelihood acquisition function 4504, likelihood aggregation function 4505, visualization function 4506, text conversion function 4507, operation determination function 4508, reconstruction processing function 4509, transmission function 4510, notification function 4511, and display control function 4512. For example, the system control function 4501, preprocessing function 4502, medical information acquisition function 4503, likelihood acquisition function 4504, likelihood aggregation function 4505, visualization function 4506, text conversion function 4507, operation determination function 4508, reconstruction processing function 4509, transmission function 4510, notification function 4511, and display control function 4512, which are components of the processing circuit 145 shown in Figure 2, each of the processing functions performed by these functions is recorded in memory 141 in the form of a program that can be executed by a computer. The processing circuit 145 is, for example, a processor, and it reads each program from memory 141 and executes it to realize the function corresponding to each program that has been read. In other words, the processing circuit 145 in the state in which each program has been read will have each of the functions shown in the processing circuit 145 of Figure 2.

[0034] In Figure 2, the processing functions performed by the system control function 4501, preprocessing function 4502, medical information acquisition function 4503, likelihood acquisition function 4504, likelihood aggregation function 4505, visualization function 4506, text conversion function 4507, operation determination function 4508, reconstruction processing function 4509, transmission function 4510, notification function 4511, and display control function 4512 are described as being realized by a single processor. However, the processing circuit 145 may be configured by combining multiple independent processors, and each processor may realize the functions by executing a program. Also, in Figure 2, a single memory 141 is described as storing the program corresponding to each processing function. However, multiple memories 141 may be distributed and arranged, and the processing circuit 145 may read the corresponding program from the individual memories 141.

[0035] In the above explanation, the term "processor" refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), an Application Specific Integrated Circuit (ASIC), or a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). The processor functions by reading and executing a program stored in memory 141. Alternatively, instead of storing the program in memory 141, the processor may be configured to directly incorporate the program into its circuitry. In this case, the processor functions by reading and executing the program incorporated into the circuitry.

[0036] The system control function 4501 controls various functions of the processing circuit 145 based on input operations received from the operator via the input interface 143. The system control function 4501 is also an example of a control unit.

[0037] The preprocessing function 4502 generates data by applying preprocessing such as logarithmic transformation, offset correction, inter-channel sensitivity correction, and beam hardening correction to the detection data output from the DAS118. Note that the data before preprocessing (detection data) and the data after preprocessing are sometimes collectively referred to as projection data. Furthermore, the preprocessing function 4502 is an example of a preprocessing unit.

[0038] The medical information acquisition function 4503 sequentially acquires medical information. The medical information acquisition function 4503 is an example of the first acquisition unit. More specifically, the medical information acquisition function 4503 acquires projection data sequentially acquired by the preprocessing function 4502. That is, the medical information acquisition function 4503 acquires medical information such as detection data sequentially output by the DAS118 and projection data sequentially generated by the preprocessing function 4502. For example, the medical information is slice images included in one series of images taken of subject P.

[0039] The likelihood acquisition function 4504 acquires the likelihood for each classification result of the medical information sequentially acquired by the medical information acquisition function 4503 in the classification model M1. The likelihood acquisition function 4504 is an example of a second acquisition unit. The classification targets are, for example, diseases and injuries.

[0040] Figure 3 shows an example of how likelihood is obtained using classification model M1. Classification model M1 is a pre-trained model that classifies the items to be classified in medical information. In other words, classification model M1 is a pre-trained model that classifies the diseases and injuries included in medical information.

[0041] Furthermore, classification model M1 is not limited to classifying one classification target included in the medical information, but may classify multiple classification targets. In addition, likelihood acquisition function 4504 may acquire the likelihood for each classification result of the medical information for multiple classification models M1, not just one classification model M1.

[0042] The likelihood aggregation function 4505 aggregates the likelihoods corresponding to the medical information obtained by the likelihood acquisition function 4504. The likelihood aggregation function 4505 is an example of an aggregation unit. More specifically, the likelihood aggregation function 4505 sequentially collects and calculates the likelihood of each classification target of the medical information sequentially acquired by the likelihood acquisition function 4504.

[0043] For example, the likelihood aggregation function 4505 accumulates the likelihood corresponding to the medical information acquired sequentially for each classification target. Figure 4 shows an example of an accumulation method for accumulating likelihood for each classification target. As shown in Figure 4, the likelihood aggregation function 4505 accumulates the likelihood corresponding to the medical information acquired by the likelihood acquisition function 4504 for each classification target. The likelihood threshold line T1 shown in Figure 4 is the criterion for determining whether or not the relevant classification target is estimated to be included in subject P. The X-ray CT device 10 estimates that the relevant classification target is included in subject P if the accumulated value exceeds the likelihood threshold.

[0044] The visualization function 4506 visualizes the regions that form the basis for the classification of the items to be classified by the classification model M1. For example, the visualization function 4506 generates a heat map showing the influence of each region of medical information that influenced the classification of the items to be classified by the classification model M1. The visualization function 4506 is an example of the second generation unit. For example, the visualization function 4506 generates a heat map showing the influence of each region containing one or more pixels included in the medical information.

[0045] For example, visualization function 4506 generates a heatmap showing the influence of each area of ​​medical information based on the likelihood of each area of ​​medical information, using LIME (Local Interpretable Model-agnostic Explanations) and Grad-CAM (Gradient-weighted Class Activation Mapping).

[0046] The text generation function 4507 generates text indicating classification targets estimated to be included in the medical information, based on the aggregation results from the likelihood aggregation function 4505. The text generation function 4507 is an example of the first generation unit. For example, the text generation function 4507 generates text indicating the location where the classification target exists in the medical information and the type of classification target, based on the heatmap generated by the visualization function 4506. That is, the text generation function 4507 generates text indicating the details of the injury or illness. For example, the details of the injury or illness include information such as the size and location of a tumor, the progression of the disease such as the stenosis rate, and the details of the injury such as fractures or trauma. The text may also contain constituent information such as organs included in the medical information. Constituent information includes information such as the region and volume of the organ, organ regions such as the Quinaud classification of the liver and lung segments, anatomical landmarks within the organ, vascular course, and bone region.

[0047] More specifically, the text conversion function 4507 identifies locations where the impact level is higher than a threshold using a heatmap. The text conversion function 4507 estimates that classification content exists at the identified locations. Furthermore, the text conversion function 4507 obtains which classification target's impact level the heatmap represents. Based on this, the text conversion function 4507 generates text indicating the location where the classification target exists in the medical information and the type of classification target. In other words, the text conversion function 4507 generates text indicating the location where the disease or injury exists in the medical information and the type of disease or injury. For example, the text conversion function 4507 generates text such as "A tumor is present in the posterior segment of the liver."

[0048] The operation determination function 4508 determines whether or not to perform post-processing based on the aggregation results from the likelihood aggregation function 4505. The operation determination function 4508 is an example of a determination unit. More specifically, the operation determination function 4508 determines whether or not to perform post-processing based on the likelihood of each medical information for each classification target aggregated by the likelihood aggregation function 4505. For example, the operation determination function 4508 determines to perform post-processing if the cumulative value of the likelihood corresponding to each medical information for each classification target, accumulated by the likelihood aggregation function 4505, is equal to or greater than the likelihood threshold.

[0049] Post-processing refers to processes performed after the acquisition of medical information. For example, medical information includes detection data sequentially output by DAS118 and projection data sequentially generated by the pre-processing function 4502. For example, post-processing may include reconstruction processes using methods such as filtered back projection and iterative reconstruction, the transmission of CT image data generated by the reconstruction process to the medical image storage device 20, and the notification that subject P contains a subject for classification.

[0050] As shown in Figure 4, the likelihood aggregation function 4505 accumulates the likelihood of each medical information for each classification target. The operation determination function 4508 determines to execute post-processing if the likelihood of any classification target exceeds the likelihood threshold.

[0051] Furthermore, the likelihood calculation function 4505 allows setting an arbitrary value as the likelihood threshold. For example, the likelihood calculation function 4505 may set an arbitrary value output from the trained model as the likelihood threshold. More specifically, the likelihood calculation function 4505 accepts the operation of specifying one or more guidelines. The likelihood calculation function 4505 inputs the specified one or more guidelines to the trained model. The trained model extracts a value to be set as the likelihood threshold from the input one or more guidelines. The likelihood calculation function 4505 may then set the value extracted by the trained model as the likelihood threshold. Note that the likelihood calculation function 4505 may input not only guidelines, but also articles presented at academic conferences, documents created within the hospital, or papers to the trained model.

[0052] The reconstruction processing function 4509 generates CT image data by performing reconstruction processing on the projection data generated by the preprocessing function 4502, using methods such as filtered back projection and iterative reconstruction. The reconstruction processing function 4509 is also an example of a reconstruction processing unit. Based on input operations received from the operator via the input interface 143, the reconstruction processing function 4509 converts the CT image data generated by the reconstruction processing function 4509 into tomographic image data of an arbitrary cross-section or 3D image data using known methods. Note that the reconstruction processing function 4509 may directly generate the 3D image data.

[0053] More specifically, the reconstruction processing function 4509, when the operation determination function 4508 determines that post-processing should be performed, performs reconstruction processing on the projection data, which is medical information acquired by the medical information acquisition function 4503 up to the time of determination, to generate CT image data. The reconstruction processing function 4509 is an example of a reconstruction processing unit.

[0054] Furthermore, the reconstruction processing function 4509 may, when the scan of subject P is completed, perform reconstruction processing on projection data, which is medical information acquired after the judgment, to generate CT image data. In this case, the reconstruction processing function 4509 may also perform reconstruction processing on projection data, which is medical information acquired before and after the judgment, to generate CT image data.

[0055] The transmission function 4510 controls the NW interface 144 to transmit the CT image data generated by the reconstruction processing function 4509 to the medical image storage device 20. When the operation determination function 4508 determines that post-processing should be performed, the transmission function 4510 transmits the CT image data generated from the medical information acquired by the medical information acquisition function 4503 up to the time of determination to the medical image storage device 20. In addition, when the operation determination function 4508 determines that post-processing should be performed, the transmission function 4510 may transmit not only CT image data but also medical information acquired by the medical information acquisition function 4503 up to the time of determination. The transmission function 4510 is an example of a transmission unit.

[0056] The notification function 4511 notifies the operation terminal 30 that there are classification targets with a likelihood above the likelihood threshold when the operation determination function 4508 determines that post-processing should be performed.

[0057] For example, the notification function 4511 notifies the operating terminal 30 using text generated by the text conversion function 4507. The notification function 4511 is an example of a notification unit. More specifically, the notification function 4511 notifies the operating terminal 30 of text indicating the location of the disease or injury and the type of disease or injury.

[0058] The display control function 4512 displays various images on the display 142. For example, the display control function 4512 displays the operation input image G1 on the display 142.

[0059] Figure 5 shows an example of an operation input image G1. The operation input image G1 is an image that displays information about the classification target included in medical information. The operation input image G1 has a cumulative probability area G11, a summary area G12, a setting switching area G13, a medical information display area G14, and a probability transition area G15.

[0060] The cumulative probability area G11 is an area that displays a bar graph showing the cumulative likelihood of each medical information for each classification target. In other words, the cumulative probability area G11 is an area that displays a graph of the likelihood of each medical information for each classification target, which has been accumulated by the likelihood aggregation function 4505. The display control function 4512 displays a bar graph in which the likelihood for the medical information is accumulated for each classification target, which is sequentially acquired by the medical information acquisition function 4503 and aggregated by the likelihood aggregation function 4505. The display control function 4512 is an example of a display control unit. Furthermore, the cumulative probability area G11 has a likelihood threshold line T1 that indicates the likelihood threshold that serves as the criterion for the operation determination function 4508 to decide whether or not to perform post-processing. In other words, the display control function 4512 displays the likelihood threshold line T1, which indicates the threshold that serves as the criterion for deciding whether or not to perform post-processing, in the bar graph.

[0061] Summary area G12 is a region that displays a summary of the image diagnosis based on the cumulative likelihood of each medical information. For example, summary area G12 has fields for displaying the detected disease / injury, likelihood threshold, size threshold, guidelines, stack count, and description.

[0062] Detected illnesses and injuries are those classified above the likelihood threshold. In other words, detected illnesses and injuries are diseases or injuries estimated to be included in subject P. The likelihood threshold is the threshold used by the operation judgment function 4508 to decide whether or not to perform post-processing. The size threshold is the threshold for the size of the classification target. If the size of the classification target is above the size threshold, a heatmap generated by the visualization function 4506 is superimposed in the medical information display area G14. The guideline is the guideline used to extract the likelihood threshold. The stack count is information indicating the number of medical information items accumulated in the cumulative probability area G11. The description is information indicating the analysis results for classification targets above the likelihood threshold. The description may be generated by generative AI (Artificial Intelligence) or entered by a medical professional.

[0063] Note that summary area G12 may or may not contain any of this information, or it may contain other information.

[0064] The setting switching area G13 is the area where settings in the operation input image G1 are entered. More specifically, the setting switching area G13 has checkboxes to switch whether or not to apply a size threshold and checkboxes to switch whether or not to apply past data.

[0065] The display control function 4512 overlays a heatmap generated by the visualization function 4506 onto the medical information when the checkbox for applying a size threshold is checked. For example, the display control function 4512 overlays a heatmap generated by the visualization function 4506 onto classification targets with a size greater than or equal to the size threshold in the medical information display area G14.

[0066] The medical information display area G14 is an area for displaying medical information. The medical information display area G14 has a first cross-sectional area G141, a second cross-sectional area G142, a third cross-sectional area G143, and a fourth cross-sectional area G144. When a likelihood is selected in the bar graph of the cumulative probability area G11, the display control function 4512 displays the medical information corresponding to the selected likelihood in the medical information display area G14. More specifically, when a likelihood is selected, the display control function 4512 displays the medical information, i.e., a medical image, corresponding to the selected likelihood in at least one of the first cross-sectional area G141, the second cross-sectional area G142, the third cross-sectional area G143, and the fourth cross-sectional area G144.

[0067] The first cross-sectional region G141 is the region that displays the coronal cross-sectional image of subject P included in the medical information. The first cross-sectional region G141 also has a first difference region image G1411 that shows the region where there is a difference between the medical information from the previous scan of subject P and the current medical information. In other words, the first difference region image G1411 is an image that shows the region that changes over time.

[0068] The second cross-sectional region G142 is the region that displays the sagittal cross-sectional image of subject P included in the medical information. The second cross-sectional region G142 also has a second heatmap image H12 and a second difference region image G1421. The second heatmap image H12 is a heatmap generated by the visualization function 4506 and is a heatmap that shows the likelihood of classification targets included in the medical information. The second difference region image G1421 is an image that shows the region where there is a difference between the medical information from the previous scan of subject P and the current medical information. In other words, the second difference region image G1421 is an image that shows the region where there is a change over time.

[0069] The third cross-sectional region G143 is a region that displays an image of the subject P included in the medical information, rendered by volume rendering. The third cross-sectional region G143 also contains the third heatmap image H13 and the third difference region image G1431. The third heatmap image H13 is a heatmap generated by the visualization function 4506, and is a heatmap that shows the likelihood of classification of the subject included in the medical information. The third difference region image G1431 is an image that shows the region where there is a difference between the medical information from the previous scan of subject P and the current medical information. In other words, the third difference region image G1431 is an image that shows the region that changes over time.

[0070] The fourth cross-sectional region G144 is a region that displays the axial cross-sectional image of subject P included in the medical information. The fourth cross-sectional region G144 also contains the fourth heatmap image H14. The fourth heatmap image H14 is a heatmap generated by the visualization function 4506, and is a heatmap that shows the likelihood of classification targets included in the medical information.

[0071] The probability transition region G15 is a graph showing the transition in the likelihood of classification targets included in the medical information of subject P. In the graph of the probability transition region G15, the horizontal axis represents the axial direction, and the vertical axis represents the likelihood of classification targets included in each medical information. In other words, the graph of the probability transition region G15 shows the likelihood of classification targets at each slice position in the axial direction. The probability transition region G15 also has axial display lines G151 that indicate the position of the axial cross-sectional image shown in the fourth cross-sectional region G144.

[0072] Next, we will describe the early operation process performed by the X-ray CT scanner 10.

[0073] Figure 6 shows an example of an early activation process performed by the X-ray CT apparatus 10 according to the first embodiment.

[0074] The medical information acquisition function 4503 acquires medical information sequentially (step S1).

[0075] The likelihood acquisition function 4504 uses the classification model M1 to acquire the likelihood of classification targets that are estimated to be included in the medical information acquired sequentially by the medical information acquisition function 4503 (step S2).

[0076] The likelihood aggregation function 4505 aggregates the likelihood of classification targets that are estimated to be included in the medical information (step S3).

[0077] The activation determination function 4508 determines whether or not to activate post-processing based on the aggregated likelihood (step S4). If the aggregated likelihood is less than the likelihood threshold (step S4; No), the medical information acquisition function 4503 sequentially acquires medical information in step S1.

[0078] If the aggregated likelihood is greater than or equal to the likelihood threshold (Step S4; Yes), the reconstruction processing function 4509 performs reconstruction processing on the medical information acquired up to the present time by the medical information acquisition function 4503 as post-processing (Step S5). As a result, the reconstruction processing function 4509 generates CT image data.

[0079] The transmission function 4510 transmits the generated CT image data to the medical image storage device 20 (step S6).

[0080] The notification function 4511 notifies the operating terminal 30 that there are classification targets with a likelihood above the likelihood threshold (step S7).

[0081] Based on the above, the medical information processing system 1 performs feedback processing.

[0082] As described above, the X-ray CT scanner 10 according to the first embodiment sequentially acquires medical information. The X-ray CT scanner 10 also aggregates the likelihood of each classification result for the sequentially acquired medical information in the classification model M1. Based on the aggregated results, the X-ray CT scanner 10 determines whether or not to perform post-processing. In this way, the X-ray CT scanner 10 performs post-processing such as reconstruction processing and transmission of medical information when the classification result, i.e., when there is a high probability that the subject P has an injury or illness, based on the sequentially acquired medical information. Therefore, the X-ray CT scanner 10 can start post-processing earlier. Furthermore, analysis processing can be performed earlier. In addition, medical professionals can interpret the images earlier.

[0083] (Variation 1) The medical information was explained as pre-reconstruction information such as detection data sequentially output by DAS118 and projection data sequentially generated by the pre-processing function 4502. However, the medical information may also be CT image data generated by the reconstruction process.

[0084] In this case, the medical information acquisition function 4503 acquires each slice image of the CT image data sequentially generated by the reconstruction processing function 4509. The likelihood acquisition function 4504 acquires the likelihood that each slice image is estimated to contain a classification target, based on the classification model M1. The likelihood aggregation function 4505 aggregates the likelihoods of the medical information acquired by the likelihood acquisition function 4504. The transmission function 4510 transmits one or more slice images acquired up to the present by the medical information acquisition function 4503 to the medical image storage device 20 when the operation determination function 4508 determines that post-processing should be performed. This allows medical professionals to interpret one or more slice images.

[0085] (Modification 2) The classification targets may include multiple conditions within a single medical image, such as invasive cancer. Furthermore, in the case of invasive cancer, the classification may be further subdivided by region. For example, the classification may be subdivided by region, such as invasive cancer of the posterior segment of the liver.

[0086] (Variation 3) The display control function 4512 may display classification targets with high likelihood on the display 142. Figure 7 shows an example of the display transition of classification targets with high likelihood. The likelihood aggregation function 4505 accumulates the likelihood of the medical information acquired sequentially by the medical information acquisition function 4503 for each classification target. The display control function 4512 displays each classification target according to the likelihood ranking of each classification target. This allows healthcare professionals to predict which classification targets are included in subject P.

[0087] (Modification 4) The likelihood aggregation function 4505 is described as accumulating the likelihood of medical information obtained by the likelihood acquisition function 4504 for each classification target. The likelihood aggregation function 4505 may not only accumulate likelihoods, but may also calculate the average of multiple likelihoods, the median of multiple likelihoods, or other values. The operation determination function 4508 then determines to perform post-processing if the average or median of multiple likelihoods is greater than or equal to the likelihood threshold.

[0088] (Variation 5) In the first embodiment, the X-ray CT apparatus 10 has a system control function 4501, a preprocessing function 4502, a medical information acquisition function 4503, a likelihood acquisition function 4504, a likelihood aggregation function 4505, a visualization function 4506, a text conversion function 4507, an operation determination function 4508, a reconstruction processing function 4509, a transmission function 4510, a notification function 4511, and a display control function 4512. All or part of these functions are not limited to the X-ray CT apparatus 10, but may also be provided by the medical image storage device 20, the operation terminal 30, or other information processing devices.

[0089] According to at least one embodiment described above, the timing of initiating post-processing can be brought forward.

[0090] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0091] 1. Medical Information Processing System 10 X-ray CT (Computed Tomography) device 20 Medical image storage device 30 Operating terminals 4501 System control function 4502 Preprocessing function 4503 Medical information acquisition function 4504 Likelihood acquisition function 4505 Likelihood Calculation Function 4506 Visualization function 4507 Text conversion function 4508 Operation detection function 4509 Reconstruction processing function 4510 Transmission function 4511 Notification function 4512 Display control function M1 Classification Model T1 Likelihood Threshold Line G1 Operation Input Image G11 Cumulative Probability Region G12 Summary Area G13 Setting Switching Area G14 Medical information display area G15 Probability Transition Area P Subject

Claims

1. The first acquisition unit sequentially acquires medical information, In a trained model for classifying classification targets included in medical information, a second acquisition unit acquires the likelihood for each classification result of the medical information sequentially acquired by the first acquisition unit, A summarization unit that aggregates the likelihoods obtained by the second acquisition unit, A determination unit determines whether or not to perform post-processing based on the aggregation results from the aggregation unit, A medical information processing device equipped with [a specific feature].

2. The aggregation unit accumulates the likelihoods for the medical information acquired sequentially by the first acquisition unit for each of the classification targets. The determination unit determines that post-processing should be performed if the cumulative value accumulated by the aggregation unit is equal to or greater than a threshold. The medical information processing device according to claim 1.

3. A first generation unit generates text indicating the disease or injury content estimated to be included in the medical information based on the aggregation results by the aggregation unit, The system further comprises a notification unit that uses the text generated by the first generation unit to provide notification. The medical information processing device according to claim 1.

4. The system further includes a display control unit that displays a graph showing the cumulative likelihood of the medical information acquired sequentially by the first acquisition unit for each classification target. The medical information processing device according to claim 1.

5. The display control unit displays the medical information corresponding to the selected likelihood when the likelihood stacked in the graph is selected. The medical information processing device according to claim 4.

6. The system further comprises a second generation unit that generates a heat map showing the degree of influence of each area of ​​the medical information that influenced the classification of the target to be classified by the trained model, The display control unit displays the heat map superimposed on the medical information. The medical information processing device according to claim 4.

7. The display control unit overlays and displays the heatmap for classification targets that are larger than or equal to a size threshold related to the size of the classification target. The medical information processing device according to claim 6.

8. The display control unit displays a threshold value on the graph that serves as a criterion for deciding whether or not to perform the post-processing. The medical information processing device according to claim 4.

9. If the determination unit determines that the post-processing should be performed, the system further comprises a reconstruction processing unit that performs reconstruction processing using the medical information acquired by the first acquisition unit. A medical information processing device according to any one of claims 1 to 8.

10. If the determination unit determines that the post-processing should be performed, the system further includes a transmission unit that transmits the medical information acquired by the first acquisition unit. A medical information processing device according to any one of claims 1 to 8.

11. Medical information will be acquired sequentially. In a trained model that classifies classification targets included in medical information, the likelihood of each classification result of the medical information acquired sequentially is obtained. The obtained likelihoods are aggregated, Based on the aggregated results, a decision is made as to whether or not to perform post-processing. A medical information processing method that includes the following.

12. On the computer, Medical information will be acquired sequentially. In a trained model that classifies classification targets included in medical information, the likelihood of each classification result of the medical information acquired sequentially is obtained. The obtained likelihoods are aggregated, Based on the aggregated results, a decision is made as to whether or not to perform post-processing. A medical information processing program designed to perform a task.

Images