Medical image processing apparatus, method, and program

Abstract

The goal is to easily obtain analysis results even when medical images do not meet the conditions for analysis. [Solution] The medical image processing apparatus according to the embodiment comprises an acquisition unit, a determination unit, a decision unit, a processing unit, and an analysis unit. The acquisition unit acquires a first medical image. The determination unit determines whether the first medical image satisfies the analyzable conditions related to the image characteristics required for the planned analysis processing, and if it is determined that the first medical image does not satisfy the analyzable conditions, it outputs log information representing the content of the determination. Based on the log information, the decision unit determines the processing to be applied to the first medical image in order to satisfy the analyzable conditions. The processing unit applies processing to the first medical image to generate a second medical image. The analysis unit applies analysis processing to the second medical image.

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a medical image processing apparatus, method, and program.

Background Art

[0002] There is a medical image processing apparatus for performing automatic analysis that is communicably connected to a medical imaging apparatus. The medical image processing apparatus receives a medical image generated by the medical imaging apparatus, performs analysis processing on the received medical image, and transmits the analysis result to a PACS (Picture Archiving and Communication System) or the like. Here, the medical image processing apparatus determines whether the medical image satisfies the analyzable conditions regarding the image characteristics that the image to be analyzed should have, using the input image conditions such as DICOM tags and image information of the received medical image. And when it is determined that the analyzable conditions are not satisfied, the medical image processing apparatus outputs an error and aborts the analysis processing. In this case, the user cannot obtain the analysis result and cannot obtain the information necessary for diagnosis.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0004] One of the problems that the embodiments disclosed in this specification and drawings aim to solve is to easily obtain analysis results even when medical images do not meet the conditions for analysis. However, the problems that the embodiments disclosed in this specification and drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0005] The medical image processing apparatus according to the embodiment comprises an acquisition unit, a determination unit, a decision unit, a processing unit, and an analysis unit. The acquisition unit acquires a first medical image. The determination unit determines whether the first medical image satisfies the analyzable conditions related to the image characteristics required for the planned analysis processing. If it is determined that the first medical image does not satisfy the analyzable conditions, it outputs log information representing the content of the determination. Based on the log information, the decision unit determines the processing to be applied to the first medical image in order to satisfy the analyzable conditions. The processing unit applies the processing to the first medical image to generate a second medical image. The analysis unit applies the analysis to the second medical image. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 shows an example of the configuration of a medical image processing system. [Figure 2] Figure 2 shows an example of the configuration of a medical image processing device. [Figure 3] Figure 3 shows the flow of automated analysis processing by a medical image processing system. [Figure 4] Figure 4 shows an example of analyzable conditions and input image conditions. [Figure 5] Figure 5 is a diagram illustrating the correspondence between the conditions for analysis, the input image conditions, and the processing details of the image processing. [Figure 6] Figure 6 shows an example of the analysis results display screen. [Figure 7] Figure 7 shows an example of the analysis results display screen. [Figure 8]Figure 8 shows an example of a confirmation window. [Figure 9] Figure 9 shows the first implementation configuration of the judgment process, processing decision process (log analysis), processing process, and analysis process. [Figure 10] Figure 10 shows a second implementation configuration of the judgment process, processing decision process (log analysis), processing process, and analysis process. [Figure 11] Figure 11 schematically shows the changes in judgment processing, log analysis, processing, and analysis processing across multiple analysis applications. [Figure 12] Figure 12 schematically illustrates the process of creating and operating an application decision model. [Modes for carrying out the invention]

[0007] The medical image processing apparatus, method, and program according to this embodiment will be described in detail below with reference to the drawings.

[0008] Figure 1 shows an example configuration of a medical image processing system 100. The medical image processing system 100 is a computer network system that performs medical image acquisition, automatic analysis of medical images, and provision of analysis results. As shown in Figure 1, the medical image processing system 100 includes a medical imaging device 200, a medical image processing device 300, and a medical image display device 400. The medical imaging device 200, the medical image processing device 300, and the medical image display device 400 are connected to any network so that they can communicate with each other. The network refers to any information and communication network that utilizes telecommunications technology. The network may include wireless / wired LANs such as hospital backbone LANs (Local Area Networks), the Internet network, telephone communication lines, fiber optic communication networks, cable communication networks, satellite communication networks, etc.

[0009] The medical imaging device 200 performs medical imaging on a subject, such as a patient, and collects medical images of the subject. The medical imaging device 200 is also called a modality device. Specifically, the medical imaging device 200 may be a single modality device such as a magnetic resonance imaging (MRI) device, an X-ray computed tomography (X-ray CT) device, an X-ray diagnostic device, an ultrasound diagnostic device, a PET (Positron Emission Tomography) device, and a SPECT (Single Photon Emission CT) device, or it may be a composite modality device such as a PET / CT device, a SPECT / CT device, a PET / MRI device, or a SPECT / MRI device. The medical images collected by the medical imaging device 200 are transmitted to the medical image processing device 300. The medical image processing device 300 is a computer that automatically performs analysis processing on the medical images. The medical image processing device 300 is also called an automatic analysis device. The analysis results from the analysis process are transmitted to the medical image display device 400. The medical image display device 400 is a computer that displays the analysis results. The medical image display device 400 is implemented, for example, by a computer included in a PACS (Picture Archiving and Communication System), which is a medical image storage system. The medical image processing device 300 may also acquire the medical images to be analyzed from the PACS.

[0010] Figure 2 shows an example configuration of a medical image processing device 300. As shown in Figure 2, the medical image processing device 300 includes a processing circuit 31, a storage device 32, a display device 33, an input interface 34, and a communication interface 35. The processing circuit 31, storage device 32, display device 33, input interface 34, and communication interface 35 are connected to each other via a bus so that they can communicate with one another.

[0011] The processing circuit 31 has a processor such as a CPU (Central Processing Unit). By starting various programs installed in the storage device 32, the processor realizes the acquisition function 311, judgment function 312, decision function 313, processing function 314, analysis function 315, communication control function 316, and display control function 317. Each of the functions 311 to 317 is not limited to being realized by a single processing circuit. A processing circuit may be configured by combining multiple independent processors, and each of the functions 311 to 317 may be realized by each processor executing a program.

[0012] The acquisition function 311 acquires various types of information. As an example, the acquisition function 311 acquires the first medical image of a subject collected by the medical imaging device 200. Specifically, the acquisition function 311 acquires the first medical image received from the medical imaging device 200 via the communication interface 35. As another example, the acquisition function 311 acquires the first medical image of a subject from the PACS server. Medical images collected by the medical imaging device 200 are stored in a searchable format on the PACS server. The acquisition function 311 is an example of an acquisition unit.

[0013] The determination function 312 determines whether the first medical image acquired by the acquisition function 311 satisfies the analyzable conditions related to the image characteristics required for the planned analysis process. Specifically, the image characteristics include parameters related to the characteristics of the medical image, such as the image type, whether or not it includes the anatomical region to be analyzed, image size, image orientation, slice thickness, and whether or not it is a contrast-enhanced or non-contrast image. Hereinafter, the parameters related to image characteristics will be referred to as image characteristic parameters. Image characteristics may be defined by numerical values ​​or symbols of these image characteristic parameters, or by text or strings that qualitatively express the image characteristic parameters. If it is determined that the first medical image does not satisfy the analyzable conditions, the determination function 312 outputs log information representing the content of the determination. If it is determined that the first medical image satisfies the analyzable conditions, the determination function 312 may or may not output log information representing the content of the determination. The determination function 312 is an example of a determination unit.

[0014] The determination function 313 determines the processing to be performed on the first medical image to satisfy the analyzable conditions based on the log information output by the determination function 312. As an example, the determination function 313 analyzes the log information to identify the analyzable conditions and the input image conditions, and determines the processing based on the identified analyzable conditions and input image conditions. The input image conditions mean the image characteristics of the first medical image. The determination function 313 applies rules and / or artificial intelligence to the identified analyzable conditions and input image conditions to determine the processing. The processing means image processing for changing the image characteristic parameters, more specifically, image processing for changing the parameter values of the image characteristic parameters to satisfy the analyzable conditions. The determination function 313 is an example of a determination unit.

[0015] The processing function 314 performs the processing determined by the determination function 313 on the first medical image to generate a second medical image. The processing function 314 is an example of a processing unit.

[0016] The analysis function 315 performs an analysis process on the medical image and outputs an analysis result. As an example, the analysis function 315 performs an analysis process on the second medical image generated by the processing function 314. The analysis process according to this embodiment is realized by software such as an application and includes various processes according to the analysis purpose. Examples of the analysis process include a process for detecting the position of a specific lesion such as a tumor, a process for identifying an arbitrary anatomical site, a process for measuring the volume of an arbitrary anatomical site or a lesion site, a process for measuring the oxygen partial pressure, and any other arbitrary process. The analysis function 315 is an example of an analysis unit.

[0017] The communication control function 316 transmits and receives various data to and from the medical imaging device 200 and the medical image display device 400 via the communication interface 35. For example, if the determination function 312 determines that the first medical image does not meet the conditions for analysis, the communication control function 316 sends a command to the medical imaging device 200 or the medical image display device 400 to confirm whether or not to allow processing of the first medical image. The communication control function 316 receives a response from the medical imaging device 200 or the medical image display device 400 regarding whether or not to allow processing of the first medical image. The communication control function 316 is an example of a communication control unit.

[0018] The display control function 317 displays various data on the display device 33 provided by the device 300 or on a medical image display device 400 connected via a network. For example, the display control function 317 displays the analysis results of the analysis process performed by the analysis function 315, the first medical image, and the second medical image. In this case, the display control function 317 may also display, in addition to the analysis results of the analysis process and the second medical image, a string of characters and / or a schematic diagram representing the processing details of the processing applied to the first medical image. As another example, the display control function 317 may add a label to the second medical image indicating that it was generated by processing and display it thereon. As yet another example, the display control function 317 may visually highlight the difference between the first medical image and the second medical image.

[0019] The storage device 32 is a storage device that stores various types of data, such as ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit memory. In addition to the above storage devices, the storage device 32 may also be a drive device that reads and writes various types of information to and from portable storage media such as CDs, DVDs, and flash memory, or semiconductor memory elements. The storage device 32 may also be located in another computer connected to the medical image processing device 300 via a network.

[0020] The display device 33 displays various data. The display device 33 can be a liquid crystal display (LCD), a CRT (Cathode Ray Tube) display, an organic electroluminescent display (OELD), a plasma display, or any other display as appropriate. Alternatively, the display device 33 may be a projector. Furthermore, the display device 33 may be an input device located on another computer connected via a network or the like. In this case, the medical image display device 400 is an example of the display device 33.

[0021] The input interface 34 receives various input operations from the user, converts the received input operations into electrical signals, and outputs them to the processing circuit 31. Specifically, the input interface 34 is connected to input devices such as a mouse, keyboard, trackball, switch, button, joystick, touchpad, and touch panel display. The input interface 34 outputs electrical signals to the processing circuit 31 corresponding to the input operations to the input device. The input device connected to the input interface 34 may also be an input device provided on another computer connected via a network or the like. The input interface 34 may also be a voice recognition device that converts voice signals collected by a microphone into instruction signals.

[0022] The communication interface 35 includes a NIC (Network Interface Card) or the like that enables communication connections with external devices such as a medical imaging device 200 and a medical image display device 400 via a network.

[0023] The following describes an example of the operation of the medical image processing system 100 according to this embodiment.

[0024] Figure 3 shows the flow of automatic analysis processing by the medical image processing system 100. For the purposes of the following explanation, the medical imaging device 200 will be assumed to be an MRI device.

[0025] As shown in Figure 3, first, the medical imaging device 200 performs one or more medical imaging scans on the subject according to a pre-planned imaging protocol (step S1). For example, in an MRI scan, the imaging protocol may include calibration scans, T1-weighted imaging, T2-weighted imaging, FLAIR (Fluid Attenuated Inverse Recovery) imaging, diffusion-weighted imaging, and other medical imaging scans. Various MR imaging scans are performed according to the imaging protocol.

[0026] As an example, the medical imaging device 200 applies a static magnetic field via a static magnetic field magnet and, according to the pulse sequence of various MR imaging, repeatedly applies a gradient magnetic field via a gradient magnetic field coil and an RF pulse via a transmitting coil. The application of the RF pulse causes the subject to emit an MR signal. The emitted MR signal is received via a receiving coil. The received MR signal is subjected to signal processing such as A / D conversion by the receiving circuit. The MR signal after A / D conversion is called k-space data. k-space data is an example of raw data.

[0027] Once step S1 is performed, the medical imaging device 200 performs image reconstruction processing on the collected raw data to generate a medical image (step S2). The image reconstruction processing can be performed using the Fourier transform method, iterative reconstruction method, iterative reconstruction method using machine learning, or any other arbitrary image reconstruction method. Medical images are generated sequentially for each MR imaging. The generated medical images are sequentially stored in the storage device 32. Among the medical images generated sequentially by the implementation of the imaging protocol, the medical image used for analysis processing is referred to as the first medical image.

[0028] When step S2 is performed, the medical imaging device 200 transmits the first medical image to the medical image processing device 300 (step S3). The medical image processing device 300 receives the first medical image. Upon receiving the first medical image, the processing circuit 31 reads the medical image processing program from the storage device 32 and executes it, and then performs the various processes in steps S4 to S10 below.

[0029] First, the processing circuit 31 acquires a first medical image using the acquisition function 311 (step S4). After step S4 is performed, the processing circuit 31 uses the determination function 312 to perform a determination process to determine whether the first medical image satisfies the conditions for analysis of the planned analysis process (step S5). For the determination process, it is possible to use either a method that uses the DICOM tag information of the first medical image or a method that uses pre-processing for the analysis process.

[0030] This section describes how to use DICOM tag information for first medical images. DICOM tag information is metadata for medical images that conforms to the DICOM standard and stores attribute information of the medical image. DICOM tag information has a tag number representing the type of attribute information and a value (hereinafter referred to as the attribute value) associated with that attribute information in the medical image. Attribute information includes patient name, patient ID, time of medical image acquisition, and image characteristic parameters such as image type, image size, image orientation, slice thickness, and contrast / non-contrast image type. Attribute information related to image characteristic parameters among the attribute information included in the DICOM tag information corresponds to the input image conditions.

[0031] The processing circuit 31 determines whether the first medical image satisfies the analyzable conditions based on the DICOM tag information attached to the first medical image. Specifically, first, the processing circuit 31 reads out the analyzable conditions for the planned analysis process. The analyzable conditions are predetermined according to the planned analysis process. The analyzable conditions are described by values, ranges, and conditional expressions for various image characteristic parameters that the medical image must satisfy. Next, the processing circuit 31 reads out the attribute values ​​of the tag numbers corresponding to the image characteristic parameters of the analyzable conditions from the DICOM tag information attached to the first medical image as input image conditions. Then, the processing circuit 31 determines whether the input image conditions satisfy the analyzable conditions.

[0032] Figure 4 shows an example of analyzable conditions and input image conditions. Figure 4 illustrates analyzable conditions and input image conditions for five types of image characteristic parameters. For example, the first image characteristic parameter is "image type," the image type for analyzable conditions is CT image, and the image type for input image conditions is MR image. In this case, the input image condition "MR image" does not match the analyzable condition "CT image," so it is determined that the input image condition does not satisfy the analyzable conditions. As another example, the second image characteristic parameter is "image size," the image size for analyzable conditions is 512 × 512 pixels or larger, and the image type for input image conditions is 256 × 256 pixels. In this case, the input image condition "256 × 256 pixels" is not included in the analyzable condition "512 × 512 pixels or larger," so it is determined that the input image condition does not satisfy the analyzable conditions. As yet another example, the third image characteristic parameter is "image orientation," the image orientation for analyzable conditions is axial, and the image type for input image conditions is axial. In this case, the input image condition "axial" matches the analyzable condition "axial," and it is determined that the input image condition does not satisfy the analyzable condition. As another example, the fourth image characteristic parameter is "slice thickness," the analyzable condition for slice thickness is 3 mm or less, and the input image condition for image type is 1 mm. In this case, the input image condition "1 mm" is included in the analyzable condition "3 mm or less," so the input image condition is included in the analyzable condition, and it is determined that the input image condition does not satisfy the analyzable condition.

[0033] Next, we will explain how to use preprocessing for analysis. Generally, preprocessing is performed prior to analysis. For example, the analysis process for detecting the location of a lung tumor (hereinafter referred to as the lung tumor location detection process) requires that the medical image includes the entire lung field region. That is, the inclusion of the entire lung field region in the medical image is set as a condition for analysis. Therefore, preprocessing is performed to extract the lung field region from the medical image. If the entire lung field region is extracted from the medical image during preprocessing, the subsequent lung tumor location detection process is performed. If only a part of the lung field region is extracted from the medical image, or if no lung field region is extracted at all, the subsequent lung tumor location detection process is not performed.

[0034] The processing circuit 31 then determines whether the first medical image satisfies the conditions for analysis based on the processing results of the pre-processing applied to the first medical image. For example, if the analysis process is lung tumor location detection, the processing circuit 31 determines that the first medical image satisfies the conditions for analysis if the entire lung field region is extracted from the medical image in the pre-processing. On the other hand, the processing circuit 31 determines that the first medical image does not satisfy the conditions for analysis if the entire lung field region is not extracted from the medical image in the pre-processing.

[0035] In both the method using DICOM tag information and the method using preprocessing, if the processing circuit 31 determines that the first medical image does not meet the analyzable conditions, it outputs log information representing the determination. The output of log information stops the start of the analysis process. The log information includes a string representing the reason why the first medical image was determined not to meet the analyzable conditions.

[0036] If it is determined in step S5 that the conditions for analysis are met (step S6: YES), the processing circuit 31 performs analysis on the first medical image using the analysis function 315 (step S9). The processing circuit 31 uses the display control function 317 to display the first medical image acquired in step S4 and the analysis results output in step S9 on the medical image display device 400. Specifically, the processing circuit 31 uses the display control function 317 to generate data for a display screen including the first medical image and the analysis results, and uses the communication control function 316 to transmit the generated data to the medical image display device 400 (step S10). When step S10 is performed, the medical image display device 400 displays the received display screen including the first medical image and the analysis results (step S11). The physician, who is the user of the medical image display device 400, makes a diagnosis of the subject by referring to the first medical image and the analysis results.

[0037] If it is determined in step S5 that the analyzable conditions are not met (step S6: NO), the processing circuit 31 determines the processing to be applied to the first medical image based on the log information using the decision function 313 (step S7). In step S7, the processing circuit 31, as an example, analyzes the log information to identify the analyzable conditions and input image conditions, and determines the processing based on the identified analyzable conditions and input image conditions. The decision function 313 determines the processing by applying rule-based and / or artificial intelligence to the identified analyzable conditions and input image conditions. The processing refers to image processing that changes image characteristic parameters, or more specifically, image processing that changes the parameter values ​​of image characteristic parameters to satisfy the analyzable conditions. This will be explained in detail below. In the following explanation, the log information is written in English, but this embodiment is also applicable to Japanese, Chinese, or any other language.

[0038] As an example, consider the case where the first log information contains the following two strings. "2024-04-03 10:20:30 ERROR: The protocol is not compliant with the requirements" "2024-04-03 10:20:30 ERROR: input data's modality=MR, but this application supports CT"

[0039] The first log information is an example of log information when the determination process in step S5 is performed using DICOM tag information. The first string indicates that the first medical image does not meet the analyzable conditions. The second string indicates that the analyzable conditions for the analysis process require the image type to be a CT image, but the input image conditions for the first medical image mean that the image type is an MR image. In other words, the second string includes both the analyzable conditions and the input image conditions.

[0040] As another example, consider the case where the following second log information is output. "2024-04-08 12:20:30 ERROR: no whole lung region"

[0041] The second log information is an example of log information when the determination process in step S5 is performed using a method that uses preprocessing for the analysis process. This log information indicates that the analyzable condition for the analysis process requires that the first medical image includes the entire lung field region, but the input image condition for the first medical image means that the first medical image does not include the entire lung field region. In other words, the second log information includes the analyzable condition, and the input image condition can be inferred from this analyzable condition.

[0042] The processing circuit 31 analyzes the log information to identify the analyzable conditions and / or input image conditions. The difference between the analyzable conditions and the input image conditions corresponds to the reason why the analyzable conditions were not met. The processing circuit 31 can identify the analyzable conditions and input image conditions from the log information using rule-based methods or artificial intelligence. In the rule-based method, a table is pre-stored in the storage device 32 that links the analyzable conditions and / or input image conditions with the corresponding strings output as log information. The processing circuit 31 inputs the log information into this table to identify the analyzable conditions and / or input image conditions corresponding to specific strings contained in the log information. In the artificial intelligence method, the processing circuit 31 uses artificial intelligence to predict the input image conditions from the strings corresponding to the analyzable conditions contained in the log information. For the artificial intelligence, a large-scale language model applying natural language processing (NLP) can be used.

[0043] For example, the first log information includes both the analyzable conditions and the input image conditions, so it is possible to use this table to identify the analyzable condition "image type = CT image" and the input image condition "image type = MR image". The second log information includes only the analyzable condition "includes the entire lung field region", so it is possible to use this table to identify the input image condition "part of the lung field region is missing" from the analyzable condition "includes the entire lung field region".

[0044] Once the analyzable conditions and input image conditions are identified, the processing circuit 31 identifies the processing content of the image processing according to the types of image characteristic parameters that differ between the identified analyzable conditions and input image conditions. For example, a model (hereinafter referred to as the processing content prediction model) that associates the types of image characteristic parameters that differ between the analyzable conditions and input image conditions with the processing content of the image processing corresponding to those types of image characteristic parameters is pre-stored in the storage device 32. The processing content prediction model may be a look-up table (LUT) that takes the analyzable conditions and input image conditions as input and outputs the processing content of the image processing, or it may be a machine learning model that takes the analyzable conditions and input image conditions as input and outputs the processing content of the image processing. The processing circuit 31 applies the identified analyzable conditions and input image conditions to the processing content prediction model to determine the processing content of the image processing.

[0045] Figure 5 is a diagram showing the correspondence between the analyzable conditions, input image conditions, and processing content. Numbers 1 to 5 in Figure 5 correspond to numbers 1 to 5 in Figure 4, respectively. As shown in the first example, when the image characteristic parameters that differ between the analyzable conditions and the input image conditions are image types, the processing content is to "convert from the image type of the input image conditions to an image type that satisfies the analyzable conditions using a Generative Adversarial Network (GAN)." For example, if the image type of the analyzable conditions is a CT image and the image type of the input image conditions is an MR image, the processing content is to "convert from an MR image to a CT image using a GAN."

[0046] As shown in the second example, if the image characteristic parameter that differs between the analyzable conditions and the input image conditions is the image size, the processing steps are to "convert the image from the size of the input image conditions to an image size that satisfies the analyzable conditions using zero padding." For example, if the image size of the analyzable conditions is 512×512 pixels and the image type of the input image conditions is 256×256 pixels, the processing steps are to "convert to 512×512 pixels using zero padding."

[0047] As shown in the third example, when the image characteristic parameter that differs between the analyzable conditions and the input image conditions is the image direction, the processing steps are to "convert from the image direction of the input image conditions to an image direction that satisfies the analyzable conditions using MPR (Multi-Planar Reconstruction)." For example, if the image direction of the analyzable conditions is axial and the image type of the input image conditions is sagittal, the processing steps are to "convert to an axial image using MPR processing."

[0048] As shown in the fourth example, when the image characteristic parameter that differs between the analyzable conditions and the input image conditions is the slice thickness, the processing steps are to "use a re-slice process to convert from the slice thickness of the input image conditions to a slice thickness that satisfies the analyzable conditions." For example, if the slice thickness of the analyzable conditions is 1 mm or less and the slice thickness of the input image conditions is 5 mm, the processing steps are to "use a re-slice process to convert it to 1 mm."

[0049] As shown in the fifth example, when the difference in image characteristic parameters between the analyzable conditions and the input image conditions is whether the image is contrast-enhanced or non-contrast, there are two types of processing steps. The first type is to "create a non-contrast image by utilizing the differences in features between contrast-enhanced and non-contrast images that exist in the database." The second type is to "create a non-contrast image using deep learning." For example, if the analyzable condition is that the image is non-contrast, and the input image condition is that the image is contrast-enhanced, then the processing step is to "create a non-contrast image using deep learning."

[0050] As shown in the sixth example, if the image characteristic parameters that differ between the analyzable conditions and the input image conditions are within the range of an anatomical region, the processing content is to "use an atlas to fill in the missing anatomical region." For example, if the range of the anatomical region in the analyzable conditions is "including the entire lung area," and the range of the anatomical region in the input image conditions is "a part of the lung area is missing," the processing content is to "use an atlas based on age, height, weight, sex, etc. to fill in the missing lung area."

[0051] The method for determining the processing content of the processing is not limited to the above. For example, the processing circuit 31 may use an artificial intelligence chatbot to determine the processing content of the processing based on the analyzable conditions and input image conditions. In this case, the processing circuit 31 can interactively add conditions related to the processing content of the processing based on the analyzable conditions and input image conditions to determine the processing content of the processing that better suits the user's needs.

[0052] When step S7 is performed, the processing circuit 31 uses the processing function 314 to apply the processing determined in step S7 to the first medical image to generate a second medical image (step S8). For example, in the first example of Figure 5, the processing circuit 31 applies a GAN to the first medical image, which is a CT image, to generate a second medical image, which is an MR image.

[0053] If step S8 is performed, the processing circuit 31 uses the analysis function 315 to perform the planned analysis on the second medical image generated in step S8 (step S9). The analysis results are output. For example, if the analysis is a tumor location detection process, the tumor location and / or tumor probability are output. The tumor probability is output for each pixel of the second medical image. The tumor probability represents the probability that the pixel is in a tumor region. The tumor location represents the location of a pixel where the tumor probability is above a threshold.

[0054] When step S9 is performed, the processing circuit 31, using the display control function 317, displays the first medical image acquired in step S4, the second medical image generated in step S8, and the analysis results output in step S9 on the medical image display device 400. Specifically, the processing circuit 31, using the display control function 317, generates data for a display screen including the first medical image, the second medical image, and the analysis results, and transmits the generated display screen data to the medical image display device 400 using the communication control function 316 (step S10). Layout conditions such as screen division, image display magnification, and image order of the display screen can be set as parameters of the DICOM hanging protocol. The medical image display device 400 receives the data for the display screen including the first medical image, the second medical image, and the analysis results. When step S10 is performed, the medical image display device 400 displays the received display screen (hereinafter referred to as the analysis results display screen) (step S11). The layout of the analysis results display screen can be arbitrarily set. For example, the medical image display device 400 displays the first medical image, the second medical image, and the analysis results side by side on a single screen. The physician, who is the user of the medical image display device 400, makes a diagnosis for the subject by referring to the first medical image, the second medical image, and the analysis results.

[0055] Figure 6 shows an example of the analysis result display screen I1. The analysis result display screen I1 shown in Figure 6 represents the display screen for the first case in Figure 5. The analysis process for this case is assumed to be image processing that extracts an image region suspected of being a tumor from the second medical image, which is a CT image. As shown in Figure 6, the analysis result display screen I1 has display areas I11, I12, I13, and I14. Display area I11 displays the first medical image, which is an MR image. Display area I11 may also display a description of the first medical image, such as "Original MR image". Display area I12 displays the second medical image, which is a CT image. Display area I12 may also display a description of the second medical image, such as "Created CT image". This description is an example of a label indicating that the second medical image was generated by processing. Display area I13 displays a schematic diagram that schematically represents the processing content of the processing. For example, as shown in Figure 6, a schematic diagram is displayed in which a reduced image of the first medical image and a reduced image of the second medical image are connected by an arrow. Note that display area I13 may also display a description of the processing method, such as "Processing Method: GAN". Display area I14 displays the analysis results. For example, a mark I141 representing the location of a suspected brain tumor is overlaid on the second medical image, aligned to its position. Mark I141 may be displayed with a color value corresponding to the tumor probability.

[0056] Figure 7 shows an example of the analysis result display screen I2. The analysis result display screen I2 shown in Figure 7 represents the display screen for the sixth case in Figure 5. The analysis process for this case is an image processing that extracts an image region suspected of being a tumor from the second medical image, which is a lung field image. As shown in Figure 7, the analysis result display screen I2 has display areas I21, I22, I23 and I24. Display area I21 displays the first medical image, which is a medical image including the lung field region I211 in which a part of the area is missing. Display area I21 may also display a description of the first medical image, such as "Original lung image". Display area I22 displays the second medical image, which is a medical image including the entire lung field region in which the missing portion I222 of the lung field region has been restored. The processing circuit 31 may visually distinguish and display the original image region that has not been processed (hereinafter referred to as the unprocessed region) I221 and the image region that has been restored by the processing (hereinafter referred to as the restored region) I222. For example, a semi-transparent gray image area may be overlaid on the restoration area I222. This visually highlights the restoration area I222. The restoration area I222 is an example of the difference between the first and second medical images in the second medical image. The display area I22 may also display a description of the second medical image, such as "processed lung image." This description is an example of a label indicating that the second medical image was generated through processing. The display of the restoration area I222 can be arbitrarily switched via the input interface 34.

[0057] Display area I23 displays a schematic diagram that schematically represents the processing content of the processing. For example, as shown in Figure 7, a schematic diagram is displayed that connects the lung image of the first medical image and the lung image of the second medical image with an arrow. Display area I23 may also display a description of the processing content, such as "Processing method: Atlas". Display area I24 displays the analysis results. For example, a mark I243 representing the location of a suspected lung tumor is aligned and overlaid on the second medical image. Mark I243 may be displayed with a color value corresponding to the tumor probability. The processing circuit 31 does not have to display the restoration area I242. In this case, the processing circuit 31 can simply align and overlay the mark I243 on the unprocessed area I241. The display of the restoration area I242 can be arbitrarily switched via the input interface 34.

[0058] As shown in Figures 6 and 7, the processing circuit 31 presents the first medical image (the medical image before processing), the second medical image (the medical image after processing), and the analysis results of the analysis process on the second medical image to the physician, who is the user of the medical image display device 400, all on a single screen. This clearly shows the user that the analysis process was performed on the second medical image, thereby reducing user confusion and enabling the presentation of information usable for diagnosis. Furthermore, by displaying the processing details of the processing applied to the first medical image alongside the first medical image, the second medical image, and the analysis results, the processing details can also be clearly presented to the user.

[0059] Once step S10 is performed, the automatic analysis process by the medical image processing system 100 is completed.

[0060] The automated analysis process shown in Figure 3 above is just one example; various processes can be added, deleted, and / or modified.

[0061] As an example, in step S4, the medical image processing device 300 acquires a first medical image from the medical imaging device 200. However, this embodiment is not limited to this, and the first medical image may also be acquired from a PACS server.

[0062] As another example, if the medical image processing device 300 determines in step S6 that the analyzable conditions are not met (S6: NO), it sequentially executes the processing decision process (S7), processing process (S8), and analysis process (S9). However, this embodiment is not limited to this. For example, if it is determined in step S6 that the first medical image does not meet the analyzable conditions and processing is decided in step S7, the processing circuit 31 transmits a command to the medical imaging device 200 or PACS server via the communication control function 316 to confirm whether or not to allow processing of the first medical image. In response to the command, the medical imaging device 200 or PACS server displays a window (hereinafter referred to as the confirmation window) on a display device to confirm whether or not to allow processing of the first medical image.

[0063] Figure 8 shows an example of the confirmation window I3. As shown in Figure 8, the confirmation window I3 displays a message to confirm whether or not to allow processing of the first medical image, such as, "Regarding the examination for patient ID: ○○, the conditions of △△ are not met in order to perform analysis with ××. Shall we process and perform analysis using the ☆☆ method?" For example, if the confirmation window I3 is displayed on the medical imaging device 200, the user of the medical imaging device 200, such as a radiologist or physician, will decide whether or not to allow processing of the first medical image.

[0064] As shown in Figure 8, the confirmation window I3 displays a GUI (Graphical User Interface) button I31 to allow processing and a GUI button I32 to deny processing. If the user presses the GUI button I31 via the input interface 34, the medical imaging device 200 or PACS server sends a signal to the medical image processing device 300 indicating that processing of the first medical image is permitted. Upon receiving this signal, the processing circuit 31 executes processing on the first medical image using the processing function 314. If the user presses the GUI button I32 via the input interface 34, the medical imaging device 200 or PACS server sends a signal to the medical image processing device 300 indicating that processing of the first medical image is not permitted. Upon receiving this signal, the processing circuit 31 stops processing after the processing (S8). In this way, by requiring the user to decide whether or not to allow the processing, it becomes possible to avoid unnecessary generation of second medical images, analysis of second medical images, and unnecessary data transfer of second medical images to the medical image display device 400.

[0065] Next, we will describe the implementation forms of the judgment process by the judgment function 312, the processing process determination process by the decision function 313, the processing process by the processing function 314, and the analysis process by the analysis function 315.

[0066] Figure 9 shows the first implementation configuration of the judgment process P1, processing process determination process (log analysis) P2, processing process P3, and analysis process P4. As mentioned above, the processing process determination process involves the analysis of log information, so it is referred to as "log analysis." As shown in Figure 9, in the first implementation configuration, in addition to the analysis application that executes analysis process P4, a dedicated application that executes log analysis P2 is provided. The analysis application executes analysis process P4, as well as judgment process P1 and processing process P3. The dedicated application executes log analysis P2.

[0067] When the first medical image is acquired, the processing circuit 31 reads the analysis application and executes the determination process P1, as shown in Figure 9, to determine whether the first medical image meets the conditions for analysis of the planned analysis process. If the first medical image is determined to meet the conditions for analysis (OK), the processing circuit 31 executes the analysis process P4 in the analysis application. If the first medical image is determined to not meet the conditions for analysis (NG), the processing circuit 31 stops the analysis application, reads the dedicated application, executes the log analysis P2, and determines the processing process to be performed. Then, the processing circuit 31 restarts the analysis application and sequentially executes the processing process P3 and analysis process P4 of the analysis application.

[0068] Figure 10 shows a second implementation form of the judgment process P1, processing process determination process (log analysis) P2, processing process P3, and analysis process P4. In the second implementation form, the analysis application contains all of the judgment process P1, log analysis P2, processing process P3, and analysis process P4. When the first medical image is acquired, the processing circuit 31 reads the analysis application as shown in Figure 10 and executes the judgment process P1 to determine whether the first medical image meets the conditions for analysis. If it is determined that the first medical image meets the conditions for analysis (OK), the processing circuit 31 executes the analysis process P4 in the analysis application. If it is determined that the first medical image does not meet the conditions for analysis (NG), the processing circuit 31 sequentially executes the log analysis P2, processing process P3, and analysis process P4 in the analysis application.

[0069] In the first implementation configuration, a dedicated application for log analysis P2 is provided separately from the analysis application, thus reducing the effort and memory capacity required to implement log analysis P2 for each analysis application. Furthermore, even if the vendors of the dedicated application and the analysis application are different, log analysis P2 can be easily executed by simply installing the dedicated application on the medical image processing device 300. On the other hand, in the second implementation configuration, the judgment process P1 and log analysis P2 are implemented in the analysis application, meaning that the judgment process P1 and log analysis P2 are developed by the same vendor. The vendor can understand what kind of log information is output and for what reason, and can utilize this knowledge in log analysis P2, so it is expected that log analysis P2 will be executed with high accuracy.

[0070] Next, we will explain the details of the implementation method for the processing.

[0071] The storage device 32 of the medical image processing device 300 stores multiple executable analysis applications. Each of the multiple analysis applications includes an analysis process specific to that application and a determination process to determine whether the medical image meets the conditions for analysis. In addition, some applications also implement pre-processing to convert the image into a medical image suitable for the analysis process, and post-processing to convert the image into a medical image suitable for understanding the analysis results of the analysis process. In this embodiment, the processing is replaced by pre-processing and / or post-processing.

[0072] Figure 11 schematically shows the changes in judgment processing P1, log analysis P2, processing processing P3, and analysis processing P4 across multiple analysis applications. Figure 11 assumes a second implementation form in which log analysis P2 is implemented in a dedicated application. Figure 11 illustrates three analysis applications 1, 2, and 3 as analysis applications that can be performed by the processing circuit 31. Analysis application 1 includes judgment processing P1-1 and extraction processing P4-1. Extraction processing P4-1 is an example of analysis processing P4, and is a process for extracting the heart, liver, hippocampus, vertebrae, and other specific anatomical parts. Analysis application 2 includes judgment processing P1-2, MPR processing P3-2, and judgment processing P4-2. MPR processing P3-2 is an example of processing processing P3, and is a pre-processing step that converts from sagittal section (SG) to axial section (AX). Judgment processing P4-2 is an example of analysis processing P4, and is a process for determining the presence or absence of specific diseases such as heart disease. Analysis application 3 includes judgment processing P1-3, detection processing P4-3, and reslicing processing P3-3. Detection processing P4-2 is an example of analysis processing P4, and is a process for detecting specific lesions such as tumors and stenosis. Reslicing processing P3-3 is an example of processing P3, and is a post-processing step that converts the slice thickness of medical images from 5 mm to 1 mm.

[0073] As shown in Figure 11, if the extraction process P4-1 is an analysis process to be performed, the processing circuit 31 reads the analysis application 1 and executes the determination process P1-1 of the analysis application 1 to determine whether the first medical image satisfies the conditions for analysis. If it is determined that the conditions for analysis are not met (NG), the processing circuit 31 stops the analysis application 1, reads a dedicated application, and executes log analysis P2. As a result, it is found that the conditions for analysis are image direction = AX image and slice thickness = 1 mm, but the input image conditions of the first medical image are image direction = SG image and slice thickness = 5 mm, and therefore the conditions for analysis are not met.

[0074] Therefore, the processing circuit 31 uses a model (hereinafter referred to as the application determination model) that associates the available analysis applications with the image processing content provided by those analysis applications to determine a specific analysis application that includes the processing to be applied to the first medical image in order to satisfy the analysis conditions. The processing circuit 31 reads the specific analysis application and applies the processing to the first medical image. The application determination model may be a list-type LUT that takes the processing content of the processing as a key and outputs an identifier of the analysis application that includes that processing, or it may be a trained neural network that takes the processing content of the processing as input and outputs an identifier of the analysis application that includes that processing. The method for creating the application determination model will be described later.

[0075] In this embodiment, it is assumed that analysis application 2, which includes MPR processing P3-2, and analysis application 3, which includes reslicing processing P3-3, are determined as specific applications. In this case, the processing circuit 31 reads analysis application 2 and, without executing the judgment processing P1-2, performs MPR processing P3-2 on the first medical image to convert it from a sagittal section to an axial section. Subsequently, the processing circuit 31 stops analysis application 2 without executing the judgment processing P4-2, reads analysis application 3, and, without executing the judgment processing P1-3 and detection processing P4-3, performs reslicing processing P3-3 on the first medical image of the axial section to convert the slice thickness from 5 mm to 1 mm. This obtains a second medical image that satisfies the conditions for analysis. Then, the processing circuit 31 restarts analysis application 1 and performs extraction processing P4-4 on the second medical image to output the analysis results.

[0076] As described above, by substituting the pre-processing and / or post-processing functions provided by the analysis application as processing steps, it becomes possible to reduce the effort required to add processing steps to the analysis application to be used.

[0077] Next, the process of creating and operating the application decision model will be described. The processing circuit 31 creates an application decision model based on the difference between the input image to the analysis application and the output image from the analysis application.

[0078] Figure 12 schematically shows the creation and operation processes of the application determination model. In this embodiment, a list-format LUT is assumed as the application determination model, and is referred to as "list" in Figure 12. Multiple analysis applications, each equipped with pre-processing and / or post-processing, are provided. The processing circuit 31, in accordance with the execution instructions of the analysis application, inputs an input image to the analysis application, performs a series of processes such as judgment processing, pre-processing, analysis processing, and post-processing provided by the analysis application, and outputs an output image. Based on the difference between the input image and the output image, the processing circuit 31 estimates the processing content of the pre-processing and / or post-processing provided by the analysis application. For example, in the case of analysis application 2 in Figure 11, the image type of the input image is a sagittal section, while the output image is an axial section, so the processing content of the pre-processing is estimated to be MPR processing, which converts from a sagittal section to an axial section.

[0079] The processing circuit 31 associates a string representing the estimated processing content with the identifier (app ID) of the analysis application and registers it in the unregistered list. The string is registered as input, and the app ID is registered as output. The string is used as a string representing the processing content of the machining process. Each time a new analysis application is executed, the processing circuit 31 performs the above process to register the string representing the processing content of the machining process and the app ID. By performing the same process for multiple analysis applications, a registered list is created. The registered list is stored in the storage device 32.

[0080] As shown in Figure 12, during operation, the processing circuit 31 performs log analysis on the first medical image as in the above embodiment to determine the processing content. Next, the processing circuit 31 searches the registered list using the string representing the determined processing content as a key and extracts the application ID associated with that key. Next, the processing circuit 31 reads out the analysis application corresponding to the extracted application ID. The read-out analysis application has pre-processing and / or post-processing implemented to execute the processing content determined using the registered list. The processing circuit 31 performs the pre-processing and / or post-processing on the first medical image to generate a second medical image that satisfies the conditions for analysis. After that, the processing circuit 31 performs analysis processing on the second medical image.

[0081] As described above, it is possible to automatically create an application decision model based on the input and output images of the analysis application.

[0082] The method for creating the application decision model is not limited to the method described above. For example, natural language processing may be applied to the instruction manual or user guide of the analysis application to estimate the processing content of the pre-processing and / or post-processing of the analysis application. Alternatively, the user may input the processing content of the pre-processing and / or post-processing of the analysis application via the input interface 34, etc.

[0083] According to the above embodiment, the medical image processing device 300 has a processing circuit 31. The processing circuit 31 acquires a first medical image. The processing circuit 31 determines whether the first medical image satisfies the analyzable conditions regarding the image characteristics required for the planned analysis process, and if it is determined that the first medical image does not satisfy the analyzable conditions, it outputs log information representing the content of the determination. Based on the log information, the processing circuit 31 determines the processing to be applied to the first medical image in order to satisfy the analyzable conditions. The processing circuit 31 applies the processing to the first medical image to generate a second medical image. The processing circuit 31 performs analysis on the second medical image.

[0084] With the above configuration, even if the first medical image does not meet the analyzable conditions, it becomes possible to automatically generate a second medical image that meets the analyzable conditions from the first medical image. This eliminates the need for the user to re-capture medical images or estimate the processing details. Furthermore, since analysis results can be obtained for the second medical image based on the first medical image, it is possible to satisfy the user's need to obtain at least some analysis results.

[0085] According to at least one embodiment described above, analysis results can be easily obtained even when medical images do not meet the conditions for analysis.

[0086] In the above description, the term "processor" refers to circuits such as CPUs, GPUs, or Application Specific Integrated Circuits (ASICs), programmable logic devices (e.g., Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and Field Programmable Gate Arrays (FPGAs)). A processor functions by reading and executing a program stored in a memory circuit. Alternatively, instead of storing the program in a memory circuit, the program may be directly incorporated into the processor's circuitry. In this case, the processor functions by reading and executing the program incorporated into the circuitry. On the other hand, if the processor is an ASIC, for example, the program is not stored in a memory circuit; instead, the function is directly incorporated into the processor's circuitry as a logic circuit. In this embodiment, each processor is not limited to being configured as a single circuit; multiple independent circuits may be combined to form a single processor and realize its functions. Furthermore, multiple components shown in Figures 1 and 2 may be integrated into a single processor to realize its functions.

[0087] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0088] 31 Processing Circuit 32 Storage device 33 Display equipment 34 Input Interfaces 35 Communication Interfaces 100 Medical Image Processing Systems 200 Medical Imaging Devices 300 Medical Image Processing Equipment 311 Acquisition function 312 Judgment Function 313 Decision Function 314 Machining functions 315 Analysis Function 316 Communication control function 317 Display control function 400 Medical Image Display Devices

Claims

1. The first medical image acquisition unit, A determination unit determines whether the first medical image satisfies the analyzable conditions related to the image characteristics required for the planned analysis process, and if it is determined that the first medical image does not satisfy the analyzable conditions, it outputs log information representing the content of the determination. A determination unit that determines the processing to be applied to the first medical image in order to satisfy the analyzable conditions based on the log information, A processing unit that applies the processing to the first medical image to generate a second medical image, An analysis unit that performs the analysis processing on the second medical image, A medical image processing device equipped with the following features.

2. The medical image processing apparatus according to claim 1, further comprising a display control unit that displays the analysis results of the analysis process, the first medical image, and the second medical image on a display device provided by the apparatus or on a medical image display device connected via a network.

3. The medical image processing apparatus according to claim 2, wherein the display control unit displays, in addition to the analysis results of the analysis process and the second medical image, a string of characters and / or a schematic diagram representing the processing content of the processing process.

4. The medical image processing apparatus according to claim 2, wherein the display control unit displays a label on the second medical image indicating that the second medical image was generated by the processing.

5. The medical image processing apparatus according to claim 2, wherein the display control unit visually emphasizes the difference between the first medical image and the second medical image, which are the first medical image and / or the second medical image.

6. It further includes a communication control unit, The acquisition unit acquires the first medical image from a medical imaging device or PACS server. If the communication control unit determines that the first medical image does not meet the analyzable conditions and the processing is determined to be performed, it transmits a command to the medical imaging device or the PACS server to confirm whether or not to perform the processing on the first medical image. The medical image processing apparatus according to claim 1.

7. The medical image processing apparatus according to claim 6, wherein the determination unit performs the processing on the first medical image when permission to perform the processing on the first medical image is transmitted from the medical imaging device or the PACS server.

8. The medical image processing apparatus according to claim 1, wherein the determination unit analyzes the log information to identify the analyzable conditions and input image conditions relating to the image characteristics of the first medical image, and determines the processing based on the identified analyzable conditions and input image conditions.

9. The medical image processing apparatus according to claim 8, wherein the determination unit determines the processing by applying a rule-based and / or artificial intelligence to the identified analyzable conditions and the input image conditions.

10. The medical image processing apparatus according to claim 1, wherein the determination unit determines whether the first medical image satisfies the analyzable conditions based on the DICOM tag information attached to the first medical image.

11. The medical image processing apparatus according to claim 1, wherein the determination unit determines whether the first medical image satisfies the analyzable conditions based on the processing results of the preprocessing applied to the first medical image.

12. The aforementioned determination unit, Using a model that associates feasible applications with the image processing content of those applications, a specific application equipped with the aforementioned processing is determined. The specific application is read and the processing is applied to the first medical image. The medical image processing apparatus according to claim 1.

13. The medical image processing apparatus according to claim 12, wherein the determination unit creates the model based on the difference between an input image to the application and an output image from the application.

14. The determination unit reads a dedicated application for determining the machining process based on the log information and determines the machining process. The aforementioned dedicated application is a separate application from the application that provides the analysis processing. The medical image processing apparatus according to claim 1.

15. Computers The acquisition process for obtaining the first medical image, A determination step which determines whether the first medical image satisfies the analyzable conditions for the image characteristics required for the planned analysis process, and if it is determined that the first medical image does not satisfy the analyzable conditions, outputs log information representing the content of the determination, A decision step of determining the processing to be applied to the first medical image in order to satisfy the analyzable conditions based on the log information, A processing step of applying the processing to the first medical image to generate a second medical image, An analysis step of applying the analysis process to the second medical image, A medical image processing method comprising the following:

16. On the computer, The first medical image acquisition function, A determination function that determines whether the first medical image satisfies the analyzable conditions for the image characteristics required for the planned analysis process, and outputs log information representing the determination if it is determined that the first medical image does not satisfy the analyzable conditions, A decision function that determines the processing to be applied to the first medical image in order to satisfy the conditions for analysis based on the log information, A processing function that generates a second medical image by applying the processing to the first medical image, An analysis function that performs the analysis processing on the second medical image, A medical image processing program that makes this possible.

Images