Image processing device, image processing method, and program
The image processing device automates the selection of images for difference image generation, addressing the inefficiencies in manual image selection by using priority settings and image pair generation algorithms to streamline the process for lesion visualization.
Patent Information
- Application Number
- JP2023081469
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-17
- Publication Date
- 2024-11-28
AI Technical Summary
The existing image processing systems require users to manually select and combine images from multiple medical examinations to generate difference images, which is time-consuming and cumbersome, especially when selecting between temporal and contrast-enhanced images for lesion visualization.
An image processing device that automatically selects images for generating difference images based on examination date, time, and contrast state, using priority settings and image pair generation algorithms to streamline the process.
Efficiently selects images for difference image generation, reducing user effort and time required to visualize lesions by automating the selection process based on examination purpose and image characteristics.
Smart Images

Figure 2024165345000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an image processing device, an image processing method, and a program. [Background technology]
[0002] In the medical field, diagnosis is performed using images acquired by various medical imaging devices (modalities) such as computed tomography (CT) devices. For efficient diagnosis, technology is required that can visualize changes in lesions over time and the contrast effect of contrast agents to assist doctors in their diagnosis.
[0003] A temporal difference image calculation technique is known as a technique for visualizing changes over time. In this technique, two images (e.g., a current image and a past image) of the same patient taken at different times are aligned, and a difference image that visualizes the difference between the images is calculated. By displaying the temporal difference image calculated in this way, it is possible to support comparison between images. Note that the examination that is the basis for calculating the difference between images is called the target examination. The aforementioned current image corresponds to an image taken in the target examination, and the past image corresponds to an image of the same patient as the subject of the target examination, but taken in an examination earlier than the target examination.
[0004] On the other hand, a contrast time phase difference image calculation technique (Non-Patent Document 1) is known as a technique for visualizing the contrast effect of a contrast agent. This technique makes it easier to see tumors and infiltrations by visualizing the difference between a contrast image and a non-contrast image. This utilizes the characteristic that cancer cells divide more actively than other cells and require more oxygen and nutrients. Due to this characteristic, tumors and infiltrations have more blood flow and take up more contrast agent, so the contrast effect is stronger than other parts. The contrast time phase difference image calculation technique includes a method of subtracting a non-contrast image from a contrast image, and a method of subtracting contrast images of different time phases such as an early phase and a late phase. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] Atsushi Urikura, "Utilizing the characteristics of contrast CT images for accurate image diagnosis," Advaned Imaging Seminar 2019 (May 2019 issue) Internet<URL:https: / / www.innervision.co.jp / sp / ad / suite / canonmedical / sup201905 / session2-1_ct3> Summary of the Invention [Problem to be solved by the invention]
[0006] In order to visualize the occurrence of a lesion such as a tumor or infiltration using the above-mentioned time-lapse difference image calculation technique, both a current image and a past image are required. On the other hand, in order to visualize a lesion using the contrast interphase difference image calculation technique, both a contrast image and a non-contrast image are required. Conventionally, it was necessary for a user to check the presence or absence of a past image or a contrast image, and to appropriately select an image to be used for generating a difference image from among a plurality of medical images that matches the purpose of the examination. In addition, at that time, the user had to select each time whether to perform the time-lapse difference image calculation process or the contrast interphase difference image calculation process, taking into consideration the images, etc. For this reason, the selection of an image to be used for generating a difference image was a cumbersome task for the user.
[0007] The present disclosure has been made in consideration of the above circumstances, and has as one of its objects to provide an image processing device that can efficiently select images to be used for generating a difference image. [Means for solving the problem]
[0008] In order to achieve the above object, an image processing device according to one aspect of the present disclosure includes: A first image acquisition means for acquiring first image data which is image data obtained by photographing a subject through a first examination; The method further includes a comparison image acquisition means for acquiring comparison image data for comparison with the first image data, the comparison image data being acquired according to a priority for a combination including the first image data and the candidate comparison image data, the priority being calculated based on the examination date and time of the first examination and the contrast state of the first image data. Effect of the Invention
[0009] According to one aspect of the present disclosure, it becomes possible to efficiently select images to be used in generating a difference image. [Brief description of the drawings]
[0010] [Figure 1] 1 is a diagram illustrating a schematic diagram of an overall configuration of an image processing apparatus according to a first embodiment. [Diagram 2] 2 is a flowchart showing all steps executed in the image processing device shown in FIG. 1. [Diagram 3] 11 is a flowchart showing each step in an image pair generation process. [Figure 4(a)] FIG. 13 is a diagram for explaining a method for determining an image pair, and is a schematic diagram showing a case where a contrast image and a non-contrast image are mainly recorded at different dates and times. [Figure 4(b)] FIG. 13 is a diagram for explaining a method for determining an image pair, and is a diagram showing the relationship between an image pair and a priority level. [Figure 4(c)] FIG. 13 is a diagram for explaining a method for determining an image pair, and is a diagram showing the relationship between an image pair and a priority level. [Diagram 5] 13 is a flowchart showing steps in an image pair generation process executed in image processing according to a second modified example of the first embodiment. [Figure 6] 10 is a flowchart showing steps in a difference image generation process executed in an image processing device according to a second embodiment. [Figure 7] 10 is a flowchart showing each step in a difference image generation process. [Figure 8]FIG. 13 is a diagram illustrating a schematic overall configuration of an image processing apparatus according to a third embodiment. [Figure 9] 9 is a flowchart showing all steps executed in the image processing device shown in FIG. 8. [Figure 10] 13 is a flowchart showing each step in a process of generating a difference image with an evaluation value. [Figure 11] FIG. 13 is a diagram illustrating a schematic overall configuration of an image processing apparatus according to a fourth embodiment. [Figure 12] 12 is a flowchart showing all steps executed in the image processing device shown in FIG. [Figure 13] 13 is a flowchart showing each step in a process for displaying a confirmation dialog. [Figure 14] 11 is a flowchart showing steps in a callback process when displaying a list of images. [Figure 15] 4 is a flowchart showing each step in an image display process. [Figure 16] FIG. 13 illustrates an example of a confirmation dialog. [Figure 17] FIG. 13 is a diagram showing an example of an image list. [Figure 18] FIG. 13 is a diagram showing an example of an image detail screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, the present disclosure will be described in detail based on preferred embodiments thereof with reference to the attached drawings. Note that the configurations shown in the following embodiments are merely examples, and the present disclosure is not limited to the configurations shown in the drawings. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the present disclosure, and multiple features may be combined arbitrarily. In addition, in the drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.
[0012] <First embodiment> The image processing device according to the present embodiment acquires a difference image for visualizing a lesion of a target patient by comparing multiple images of the target patient. The image processing device selectively acquires an image to be used for generating a difference image suitable for visualizing a lesion from among a time-lapse difference image or a contrast phase difference image of the target patient. In the present disclosure, the time-lapse difference image is a difference image between a current image and a past image. In addition, the contrast phase difference image is a difference image between a contrast phase image and a non-contrast phase image, or a difference image between contrast images of different phases such as an early phase and a late phase. The present disclosure relates to a method for selecting an image to be used for generating a difference image, but the generation of a difference image is also included in one aspect of the present disclosure.
[0013] Fig. 1 is a diagram showing an example of the configuration of an image processing device 10 according to the first embodiment. As shown in Fig. 1, the image processing device 10 has a processing circuitry 11, a communication interface 12, a storage circuitry 13, a display 14, an input interface 15, and a connection unit 16. In addition, the image processing device 10 is communicably connected to a medical image diagnostic device (modality), a medical image storage device, each department system, and the like via a network (not shown).
[0014] The medical image diagnostic equipment includes an X-ray CT (Computed Tomography) equipment, a Magnetic Resonance Imaging (MRI) equipment, an X-ray diagnostic equipment, etc. The medical image storage equipment is realized by a PACS (Picture Archiving and Communication System) etc. The medical images taken by the medical image diagnostic equipment are stored in a DICOM (Digital Information and Communications Interface) format. The data is stored in a format that complies with the "Standard for Medical Imaging and Communications in Medicine." Each department system includes, for example, a Hospital Information System (HIS) and a Radiology Information System (RIS). In addition, various systems such as a diagnostic report system and a Laboratory Information System (LIS) are also included in each department system.
[0015] The processing circuit 11 according to this embodiment includes a control function 11a, an image acquisition function 11b, a comparison image acquisition function 11c, a difference image acquisition function 11d, and a priority setting function 11e. These functions are executed in response to an input operation received from a user via an input interface 15 to control the image processing device 10. Here, the image acquisition function 11b is an example of an image acquisition means. The comparison image acquisition function 11c is an example of a comparison image acquisition means. The difference image acquisition function 11d is an example of a difference image acquisition means. The priority setting function 11e is an example of a priority setting means.
[0016] The control function 11a executes control to generate various GUIs (Graphical User Interfaces) and various display information in response to operations via the input interface 15 and display them on the display 14. The control function 11a also controls transmission and reception of information with devices and systems on a network (not shown) via the communication interface 12. Specifically, the control function 11a controls acquisition of three-dimensional medical images (volume data) from a medical image diagnostic device, a medical image storage device, and the like connected to the network. The control function 11a also controls acquisition of information related to a subject from an external system connected to the network. The control function 11a also controls output of processing results to devices and systems on the network.
[0017] The image acquiring function 11b acquires image data of a subject captured by a medical image diagnostic device or the like from a medical image storage device or the like. At this time, the image data is acquired based on reference address information of the image (such as a uniform resource locator (URL), various unique identifiers (UIDs), and path character strings). The processing by the image acquiring function 11b will be described in detail later.
[0018] The comparison image acquisition function 11c selects an image (comparison image) to be compared with the image of the target examination from an image list of the target patient, which will be described later. Note that the processing by the comparison image acquisition function 11c will be described in detail later.
[0019] The difference image acquisition function 11d acquires a difference image between the image of the target examination and the comparison image. Note that the process performed by the difference image acquisition function 11d will be described in detail later.
[0020] The priority setting function 11e sets the priority of a pair of images, that is, a combination of an image of a target examination and an image to be acquired as a comparison image, with reference to the examination date and time and the purpose of the examination. Note that the processing by the priority setting function 11e will be described in detail later. Note that the examination date and time includes information on only the date of the examination (examination date) without time information.
[0021] The above-mentioned processing circuitry 11 is realized by, for example, a processor. In this case, each of the above-mentioned processing functions is stored in the storage circuitry 13 in the form of a program executable by a computer. Then, the processing circuitry 11 realizes a function corresponding to each program by reading and executing each program stored in the storage circuitry 13. In other words, the processing circuitry 11 has each of the processing functions shown in FIG. 1 in a state where each program is read.
[0022] The processing circuit 11 may be configured by combining a plurality of independent processors, and each processor may execute a program to realize each processing function. Each processing function of the processing circuit 11 may be appropriately distributed or integrated and realized in a single or a plurality of processing circuits. Each processing function of the processing circuit 11 may be realized by a mixture of hardware such as a circuit and software. Here, an example in which a program corresponding to each processing function is stored in a single storage circuit 13 has been described, but the embodiment is not limited to this. For example, a configuration in which a program corresponding to each processing function is distributed and stored in a plurality of storage circuits, and the processing circuit 11 reads out and executes each program from each storage circuit.
[0023] The communication interface 12 controls the transmission and communication of various data transmitted and received between the image processing device 10 and other devices or systems connected via a network. Specifically, the communication interface 12 is connected to the processing circuit 11, and outputs data received from other devices or systems to the processing circuit 11, or transmits data output from the processing circuit 11 to other devices or systems. For example, the communication interface 12 may be a network card, a network adapter, or a NIC (Network Interface This is realized by using the Proxy Server Controller, etc.
[0024] The memory circuitry 13 stores various data and various programs. Specifically, the memory circuitry 13 is connected to the processing circuitry 11 and stores data input from the processing circuitry 11, or reads out stored data and outputs it to the processing circuitry 11. For example, the memory circuitry 13 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like.
[0025] The display 14 displays various information and data. Specifically, the display 14 is connected to the processing circuit 11, and displays various information and data output from the processing circuit 11. For example, the display 14 is realized by a liquid crystal display, a CRT (Cathode Ray Tube) display, an organic EL display, a plasma display, a touch panel, or the like.
[0026] The input interface 15 accepts input operations of various instructions and various information from a user. Specifically, the input interface 15 is connected to the processing circuit 11, converts the input operation received from the user into an electrical signal, and outputs the electrical signal to the processing circuit 11. For example, the input interface 15 is realized by a trackball, a switch button, a mouse, a keyboard, a touch pad that performs an input operation by touching the operation surface, and a touch screen in which a display screen and a touch pad are integrated. The input interface 15 is also realized by a non-contact input interface using an optical sensor, a voice input interface, and the like. Note that in this specification, the input interface 15 is not limited to only those that have physical operation parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to a control circuit is also included as an example of the input interface 15.
[0027] The connection unit 16 is a bus or the like that connects the processing circuit 11, the communication interface 12, the memory circuit 13, the display 14, and the input interface 15.
[0028] 2 is a flowchart showing all steps of processing executed by the image processing device 10 according to the first embodiment. The image processing device 10 receives as input a target patient ID for identifying an image processing target, a target examination ID, and an examination purpose. Based on these inputs, the image processing device 10 obtains first image data selected from the image data of the target examination and comparison image data which is image data for comparison with the first image data, and outputs difference image data for visualizing a lesion.
[0029] Here, each image data is three-dimensional volume data. In this embodiment, the target examination is an X-ray CT examination, and the first image data and the comparison image data are both X-ray CT images. However, the implementation of the present disclosure is not limited to this, and the present disclosure is also applicable to image data of other modalities such as MRI (Magnetic Resonance Imaging) images and ultrasound images. In addition, the first image data and the comparison image data in this embodiment may be images taken in different examinations, such as a current image and a past image of the target patient, or may be other combinations. For example, they may be two images taken in the same examination under different contrast conditions (for example, a contrast image and a non-contrast image). Furthermore, for example, the non-contrast image may be a virtual plain CT image generated using Dual-Energy CT or the like.
[0030] Next, the process of generating a difference image executed in this embodiment will be described with reference to the flowchart of FIG. 2. For example, as shown in FIG. 2, in this embodiment, the target patient ID of a certain subject, the target examination ID, and the examination purpose determined by an instruction operation via the input interface 15 are input, and image processing is started (step S200). In this embodiment, the user selects from predetermined options such as "visualization of intraspinal canal invasion" and "visualization of liver tumor" as the "examination purpose" via the input interface 15, and information indicating which of the options is the purpose of this examination is obtained. Note that the examination purposes listed here are examples for facilitating the following description, and the options also include examination purposes using known difference images, such as "visualization of bone metastasis," "visualization of lung metastasis," and "visualization of brain tumor."
[0031] The control function 11a searches for images (images of the target examination) captured in the examination (target examination) indicated by the target examination ID and images of past examinations related to the patient (target patient) indicated by the target patient ID. Specifically, using a C-FIND message of the DICOM protocol or a SELECT statement of SQL, the control function 11a searches for image data that is linked to the target patient ID and has an examination date that is the same as or earlier than the examination date of the target examination ID, and obtains an image list of the target patient (step S201).
[0032] Next, the priority setting function 11e sets various parameters for setting the priority based on the examination purpose acquired in step S200 (step S202). Specific examples include the "examination date and time difference boundary value" when determining whether the examination dates and times of the images are sufficiently apart, "date and time weighting large" and "date and time weighting small" which are weighting values to be applied to the date and time difference of the examination dates and times when calculating the priority, and "contrast weighting" which is a weighting value to be applied to the difference in the contrast state (contrast difference). In step S202, the priority setting function 11e sets these parameters according to the examination purpose.
[0033] Here, the "large date and time weight" is a weight for increasing the priority as the date and time difference increases when generation of a temporal subtraction image is prioritized, and is therefore set to a relatively large value of around 100. The "small date and time weight" is a weight for decreasing the priority as the date and time difference increases when generation of a contrast interphase subtraction image is prioritized, and is therefore set to a relatively small value of around -0.1.
[0034] For example, when the purpose of the examination is "visualization of intraspinal infiltration," it is desirable to prioritize the generation of temporal subtraction images, since the presence or absence of intraspinal infiltration is clearly indicated as a change over time. For this reason, the "examination date and time difference boundary value" is set to a predetermined value of about one week, and the weight value of "contrast weight" is set to be smaller than the weight value of "large date and time weight." This allows the priority of image pairs with a large date and time difference between the examination dates and times and small differences in the contrast state to be increased.
[0035] Also, for example, when the purpose of the examination is "visualization of liver tumors," it is desirable to prioritize the generation of contrast-enhanced interphase difference images, since the presence or absence of a liver tumor is clearly depicted as a difference in contrast effect. For this reason, the "examination date and time difference boundary value" is set to a predetermined value of about 2 days, and the weight value of "contrast weight" is set to be larger than the weight value of "large date and time weight." This makes it possible to increase the priority of image pairs that were taken on the same day and have differences in contrast state.
[0036] Moreover, regardless of the above-mentioned purpose of the examination, the weight value for "large date and time weight" is set to a value that is sufficiently larger than the weight value for "small date and time weight." Also, it is desirable to set a small weight value for the "date and time difference boundary value of examination date and time" when the progression speed of tumor or infiltration is predicted to be fast, for example, when the patient is young.
[0037] In this embodiment, the user sets the parameters (weighting values) that determine the priority according to the purpose of the examination, but the method of setting the parameters is not limited to the user. For example, weighting values (weighting values) that are predetermined according to the purpose of the examination, the date and time difference, the contrast difference, etc. may be stored in advance as a table in the memory circuitry 13, and these may be read out appropriately according to the image pairs used in the priority calculation described later. The contrast difference described here is a numerical value of the difference in the time phase of the contrast. In addition, a case in which the priority setting function 11e automatically sets the "date and time difference boundary value of the examination date and time" according to the age of the patient can also be one embodiment of the present invention. When the patient is young, it is generally considered that the growth of a tumor, etc. is fast. Therefore, it is desirable to set the "date and time difference boundary value of the examination date and time" shorter as the patient becomes younger.
[0038] Next, the comparison image acquisition function 11c acquires a list of image pairs, which are combinations including a first image and a comparison image, according to a priority based on at least one of the examination date and time and the contrast state (step S203). That is, the comparison image acquisition function 11c selects a first image from the images of the target examination in the image list of the target patient, and selects a comparison image from the images other than the first image in the image list of the target patient.
[0039] The sub-process of generating an image pair executed in step S203 will be described below with reference to the flowchart from step S300 to step S312 in Fig. 3. In step S301, specifically, all images having the same examination ID as the target examination ID in the image list obtained in step S201 (i.e., images of the target examination) are set as candidates for the first image. Then, the comparison image acquisition function 11c executes a loop process from steps S302 to S310 using information of the candidate first image (location of image, examination date and time, contrast state) as information of the target image.
[0040] In this embodiment, the comparison image acquisition function 11c selects the first image from all images of the target examination in the image list of the target patient. However, this embodiment is not limited to this, and for example, the user may select one or more images suitable for the desired difference image from the images of the target examination displayed in advance on the display 14. Alternatively, selection conditions that match the purpose, such as the imaging site and reconstruction function, may be determined in advance, and the comparison image acquisition function 11c may select one or more images from the images of the target examination according to the selection conditions.
[0041] In the actual loop process, first, a list excluding the first image from the image list is obtained as a comparison image list (step S302). Next, all images in the comparison image list are treated as candidates for comparison images, and steps S304 to S310 are looped for information of the candidate comparison images (e.g., image location, examination date and time, contrast enhancement state) (step S303). First, a date and time difference is calculated by subtracting the examination date and time of the comparison image from the examination date and time of the first image (step S304). Here, the date and time difference may be the difference between examination dates or examination times.
[0042] Next, the contrast difference is calculated by subtracting [the comparison image is a contrast image] from [the first image is a contrast image] (step S305). Here, [statement] is Iverson's notation, and is 1 when the statement is true and 0 when it is false. The method of calculating the contrast difference may be any other method that quantifies the difference in time phase, such as the difference in the elapsed time from the administration of the contrast agent. In step S306, if the date and time difference is greater than the "date and time difference boundary value of the examination date and time", it is determined that priority is given to calculating a temporal difference image as the difference image, and the process proceeds to step S307. Otherwise, it is determined that priority is given to calculating a contrast phase difference image as the difference image, and the process proceeds to step S308.
[0043] In step S307, a process of calculating the priority of the image pairs is executed so that an image pair having no contrast difference and a large date and time difference is given priority. Specifically, the following calculation is performed. Priority ← Date and time difference × Date and time weighting + (1-|Contrast Difference|) x (1+[The first image is a contrast image])- 0.1×|contrast difference|×[first image is contrast image] According to this, the first term can increase the priority of an image pair with a large date and time difference. Also, when the date and time difference is the same, the second term can increase the priority of an image pair with no contrast difference, and among them, the priority of a pair of contrast images can be increased. When the date and time difference is the same but there is a contrast difference between the image pairs, the third term can decrease the priority of the pair in which the first image is a contrast image.
[0044] This has the effect of lowering the priority of an image pair with low specificity when detecting a lesion from differences between images. Note that the above priority calculation formula is one example of a priority calculation formula that prioritizes temporal subtraction image calculation, and other calculation formulas may be used as long as it is possible to increase the priority of an image pair with high specificity and decrease the priority of an image pair with low specificity.
[0045] In step S308, a process of calculating the priority of the image pair is executed so that an image pair having a contrast difference and a small date and time difference is given priority. Specifically, the following calculation is performed. Priority ← Date and time difference × (Date and time weight small) + Contrast difference × Contrast weight
[0046] According to this, the priority of an image pair having a contrast difference can be made higher than the priority of an image pair having no contrast difference. Furthermore, when the contrast difference is the same, the priority of an image pair having a date and time difference can be made lower because the "small date and time weight" is a negative value. Furthermore, the priority of an image pair having a small date and time difference and no contrast difference, or an image pair in which a contrast image is subtracted from a non-contrast image, is set to 0 or lower. Note that the above priority calculation formula is one example of a priority calculation formula that prioritizes the calculation of a contrast inter-phase difference image, and other calculation formulas may be used as long as it is possible to increase the priority of an image pair having a contrast difference and to decrease the priority of an image pair having no contrast difference.
[0047] Next, in step S309, a determination is made based on the priority calculated in step S307 or step S308. Specifically, if the calculated priority is equal to or less than 0, it is determined that the image combination is not suitable for calculating a difference image, and the process returns to step S303 to proceed to calculation of the priority for the next image pair. If the calculated priority is greater than 0, the process proceeds to step S310.
[0048] In step S310, the information obtained by the above process, which is a combination of the location of the first image, the location of the comparison image, and the priority, is added to the list of image pairs (step S310). After adding to the list, the flow returns to step S303, and the priority is calculated for each image pair based on the comparison image list obtained in step S302, and the image pairs are added to the list as appropriate.
[0049] When the calculation of priorities for all image pairs based on the comparison image list obtained in step S302 is completed, the flow returns to step S301, and the processing of steps S302 to S310 is performed for the new first image. After the processing of calculating the priorities of image pairs for all images having the same examination ID as the target examination ID is completed and the double loop is exited, the list of image pairs is sorted in descending order using the priorities as a key (step S311).
[0050] These processes are realized, for example, by the processing circuitry 11 calling up and executing programs corresponding to the control function 11a, the priority setting function 11e, and the comparison image acquisition function 11c from the storage circuitry 13. By the processes from step S300 to step S312 described above, a list of image pairs arranged in descending order of priority is acquired.
[0051] 3 executed in step S203 described above will be described in detail with reference to specific examples shown in Fig. 4(a) to 4(c). The following describes a case where contrast and non-contrast images of a subject are acquired as shown in Fig. 4(a) and the purpose of the examination is "visualization of intraspinal canal invasion."
[0052] In Fig. 4(a), a subject with the same examination ID is examined at three points in time (three dates and times): T1 (past), T2 (slightly past), and T3 (present), and contrast and non-contrast images are taken in each examination. In addition, changes such as infiltration and tumor development occur in the subject between T1 and T2, and these changes are reflected as differences between the examination images of T1 and T2. In addition, an examination is also performed at T3, which is shortly after T2, and the date and time difference is within the range of the "date and time difference boundary value of examination date and time".
[0053] First, assume that contrast image C3 and non-contrast image N3 are acquired at examination date and time T3 (=13th) to be subtracted. Also assume that contrast image C2 and non-contrast image N2 are acquired at time T2 (=12th), one day before T3. Assume that the examination dates and times of images C2, C3, N2, and N3 are within a predetermined range of "examination date and time difference boundary value" from the examination date and time T3 of the examination subject. Also assume that contrast image C1 and non-contrast image N1 are acquired at examination date and time T1 (=1st) as images taken before the occurrence of infiltration or tumor.
[0054] As mentioned above, since the presence or absence of spinal canal invasion is clearly evident as a change over time, it is desirable to prioritize the generation of temporal subtraction images. In this example, since the purpose of the examination is "visualization of spinal canal invasion," the contrast weight is set small at 1.0. As a result, the image pair list for the target examination at examination date and time T3 is a list that prioritizes image pairs that result in temporal subtraction, as shown in the table in Figure 4(b), for example, and among these, the image pair (C3-C1) that takes the difference between contrast images is given the highest priority.
[0055] In this example, the priority of contrast phase difference (C3-N3, C3-N2) is set low. However, if the purpose of the examination is to "visualize liver tumors," the presence or absence of a liver tumor is clearly depicted as a difference in the contrast effect. Therefore, in this case, it is desirable to set the contrast weight to a large value of 10000.0. The results of setting parameters in this way are shown in the table of FIG. 4(c). With such settings, it is also possible to set the priority of contrast phase difference to a high value, as shown in FIG. 4(c).
[0056] In the description of this embodiment, an example was given in which the priority was determined according to the presence or absence of a contrast agent and the time phase difference, but the present invention may also determine the priority according to the type of contrast agent. For example, in the case of MRI image diagnosis of liver cancer, the priority of an image pair including an examination image using Primovist contrast agent may be increased. Specifically, the contrast weight used in calculating the priority in steps S307 and S308 of FIG. 3 may be multiplied by 1.1 when Primovist contrast agent is used to increase the priority.
[0057] A list of image pairs is obtained by executing the above-mentioned sub-steps in step S203. As shown in Fig. 2, the control function 11a repeatedly executes the following steps S205 to S209 as a loop process for all pair information (location of the first image, location of the comparison image) in the obtained list of image pairs, starting with the pair information with the highest priority (step S204). These processes are realized, for example, by the processing circuitry 11 calling up and executing a program corresponding to the control function 11a and the image acquisition function 11b from the storage circuitry 13.
[0058] In the loop process, the image acquiring function 11b acquires image data from the location of the first image (step S205). Specifically, the image acquiring function 11b acquires image data using a C-MOVE / C-STORE message of the DICOM protocol according to the location of the image. Alternatively, the image data can be acquired using HTTP (HyperText Transfer Protocol), SMB (Server Message Block), file input / output, or the like. If the acquisition of the first image data fails due to a communication error, file I / O error, or the like, the process returns to step S204, and the loop process continues with the next pair in the list of image pairs as the processing target (step S206).
[0059] Next, the image acquisition function 11b similarly acquires image data from the location of the comparison image (step S207). If the acquisition of the comparison image data fails due to a communication error, file I / O error, or the like, the process returns to step S204, and the loop continues with the next pair in the image pair list as the processing target (step S208). If the acquisition is successful, the process proceeds to step S209.
[0060] Next, the difference image acquisition function 11d acquires difference image data from the first image data and the comparison image data selected by the process of step S204 (step S209). Specifically, the difference image acquisition function 11d realizes acquisition of the difference image by calling a difference image calculation process. Here, the difference image calculation process may be executed inside the image processing device 10, or may be executed by another device in the cloud or the like. The difference image data is data obtained by subtracting the voxel values of the corresponding coordinates of the comparison image data associated by alignment from the voxel values of each coordinate of the first image data.
[0061] Also, it is not necessary to acquire a difference image in step S209. A configuration may be used in which the first image and the comparison image are displayed on the display unit in a form that allows comparison (side by side, overlaid, or switchable). Also, it is possible to simply store or output information on the image pair (so that it can be used by other image display devices or difference image generating devices) without displaying or acquiring a difference image.
[0062] These processes are realized, for example, by the processing circuitry 11 calling up and executing a program corresponding to the image acquiring function 11b or the difference image acquiring function 11d from the storage circuitry 13. When difference image data has been obtained or displayed for all pairs in the image pair list, the flow proceeds to step S210, and the above processes are completed.
[0063] In the above description, an example has been described in which both contrast and non-contrast images are captured at multiple time points, but the present invention is not limited to this example, and the image pair generation process may be performed on a more limited number of images.
[0064] In the above example, the previous image (comparison image) must be an image taken before the onset of infiltration or tumor, and the current image (first image) must be an image taken after the onset of infiltration or tumor. However, in cases such as immediately after hospitalization or transfer to another hospital, previous images may not exist. In addition, in the contrast phase subtraction imaging method, it is necessary to administer an appropriate contrast agent to the patient and obtain multiple images with different contrast conditions after examining whether or not the contrast agent has an effect. However, depending on the patient's condition or constitution (allergies, etc.), imaging using a contrast agent may not be possible, and contrast images may not exist. The following describes in detail how to handle such cases.
[0065] As a specific example of such a case, a case where only three images, contrast images C3 and C1, and non-contrast image N3, exist in FIG. 4(a) will be described. If the first image data is image C3, the priority of the image pair of image C3 and other images is (C3-C1):1202.0, (C3-N3):1.0, respectively, when the examination purpose is "visualization of intraspinal canal invasion". Therefore, the image processing device 10 attempts to acquire image C1 of a past examination as a comparison image, and if it cannot be acquired due to a communication error, file I / O error, etc., acquires image C3 and image N3 with a different contrast state as a comparison image. Furthermore, even if image C1 of a past examination does not exist in FIG. 4(a) and only two images, contrast image C3 and non-contrast image N3, exist, image C3 and image N3 with a different contrast state are acquired as a comparison image.
[0066] Moreover, when the examination purpose is "visualization of liver tumor," the priorities are (C3-C1): 1202.0, (C3-N3): 10000.0, respectively. Therefore, the image processing device 10 attempts to acquire image N3 of an examination with a different contrast state as a comparison image, and if the image cannot be acquired due to a communication error, a file I / O error, or the like, acquires image C1 of a previous examination of image C3 as a comparison image. Furthermore, even if image N3 of an examination with a different contrast state does not exist in FIG. 4(a) and only two images, contrast image C3 and previous image C1, exist, the previous image C1 of contrast image C3 is acquired as a comparison image.
[0067] <Modification 1 of the First Embodiment> In the first embodiment described above, the priority is calculated according to the examination purpose, and an image pair with a high priority is selected, but this modified example is not limited to this. For example, the examination purpose may be fixed to "visualization of intraspinal canal invasion" and an image of an examination with a different contrast state may be acquired if there is no image of a previous examination. Also, the examination purpose may be fixed to "visualization of liver tumor" and an image of a previous examination with a different contrast state may be acquired if there is no image of a previous examination.
[0068] Furthermore, processing according to the purpose of the examination is not necessarily required. For example, regardless of the purpose of the examination, an examination image of a past examination may be preferentially acquired as a comparison image, and if there is no past examination image, an examination image with a different contrast state in the target examination may be acquired as a comparison image. Alternatively, regardless of the purpose of the examination, an examination image with a different contrast state in the target examination may be preferentially acquired as a comparison image, and if there is no examination image with a different contrast state, an examination image of a past examination may be acquired as a comparison image.
[0069] As described above, by using the image processing device according to the present embodiment, it is possible to appropriately switch between the time-lapse subtraction image generation process and the contrast phase subtraction image generation process. As a result, even when there is no past image to compare with the image of the target examination, such as immediately after hospitalization or transfer to another hospital, or when there is no contrast image due to poor physical condition or allergies, it is possible to easily generate a subtraction image for visualizing a lesion. In addition, by assigning a priority based on the examination date and time and the contrast state according to the purpose of the examination, it is possible to select a subtraction method more suitable for visualizing a lesion. As a result, it is possible to select an appropriate image pair for visualizing a lesion without causing the user troublesome work, and to generate a subtraction image using the selected image pair.
[0070] <Modification 2 of the First Embodiment> In the first embodiment, the date and time difference and the contrast phase are used as parameters as selection conditions when generating image pairs. For example, when the examination purpose is "visualization of intraspinal canal invasion" or "visualization of liver tumor", it is useful to use these as parameters. However, depending on the examination purpose, for example, the priority may be set in consideration of the degree of agreement of the imaging site.
[0071] In the following, the sub-process of image pair generation executed in step S203 in this second modification will be described with reference to the flowchart from steps S500 to S509 in Fig. 5. In step S501, specifically, all images having the same examination ID as the target examination ID in the image list obtained in step S201 (i.e., images of the target examination) are set as first images. Then, using information of the first images (image location, photographed part), loop processing from steps S502 to S507 is executed by the comparison image acquisition function 11c.
[0072] First, a list excluding the first image from the image list is obtained as a comparison image list (step S502). Next, all images in the comparison image list are set as candidates for comparison images, and the information of the comparison image (e.g., the location of the image, the photographed part) is looped through steps S504 to S507 (step S503). First, the photographed part included in the first image is compared with the photographed part included in the comparison image to confirm the degree of match between the two photographed parts. If the photographed parts are the same, the degree of match is maximum, and if the photographed parts are completely different, the degree of match is 0. If the photographed parts are different in size or partially overlap, the degree of match is determined according to the range of the overlapping parts. If the degree of match is equal to or less than a first predetermined value, it is determined that it is not suitable to form an image pair, and if it is equal to or more than the first predetermined value, it is determined that a difference image can be calculated as an image pair (step S505).
[0073] If the difference image can be calculated, the degree of match is set as the priority in step S506. Information combining the location of the first image, the location of the comparison image, and the priority obtained by the above process is added to a list of image pairs (step S507). After adding to the list, the flow returns to step S503, and a priority is calculated for each image pair based on the comparison image list obtained in step S502, and image pairs are added to the list as appropriate.
[0074] When the calculation of the priorities of all image pairs based on the comparison image list obtained in step S502 is completed, the flow returns to step S501, and the processing of steps S502 to S507 is performed for the new first image. After the processing of obtaining the priorities of image pairs for all images having the same examination ID as the target examination ID is completed and the double loop is exited, the list of image pairs is sorted in descending order using the priority as a key (step S508). By the processing of steps S500 to S509 described above, a list of image pairs arranged in descending order of priority is obtained.
[0075] As described above, the image processing device according to the first embodiment includes an image acquiring function 11b as a first image acquiring means and a comparison image acquiring function 11c as a second image acquiring means. The image acquiring function 11b acquires first image data, which is image data obtained by capturing an image of a subject by a first examination. The comparison image acquiring function 11c acquires comparison image data for comparison with the first image data. The comparison image acquiring function 11c then calculates a priority for a combination (image pair in the embodiment) including the first image data and a candidate for comparison image data, based on the examination date and time of the first examination and the contrast state of the first image data. The comparison image acquiring function 11c further acquires comparison image data used to generate a difference image according to the calculated priority. The image processing device according to the present disclosure can also execute the following image processing method. That is, the image acquiring function 11b executes a first image acquiring step of acquiring the first image data, and the comparison image acquiring function 11c executes a comparison image acquiring step of acquiring the comparison image data.
[0076] As described in the first modification, there may be cases where no past image or contrast image exists when the comparison image data is determined. In such a case, an image pair for calculating the priority based on the examination date and time of the first examination and the contrast state of the first image data cannot be formed. In such a case, a combination of images for which the priority can be calculated can be formed based on either the examination date and time of the first examination or the contrast state of the first image data. In such a case, the comparison image acquisition function 11c skips, for example, either step S304 or S305 in FIG. 3, and determines the comparison image data based on a pair for which the priority can be calculated. More specifically, the comparison image acquisition function 11c attempts to acquire, as the comparison image data, second image data, which is image data obtained by photographing the subject by a second examination that is an examination prior to the first examination. If the image data cannot be acquired, the comparison image data is acquired by third image data, which is image data having a different contrast state from the first image data. Alternatively, the comparison image acquisition function 11c may also attempt to acquire, as the comparison image data, second image data, which is image data obtained by photographing the subject by a second examination that is performed in a contrast state different from that of the first examination. In this embodiment, if the second image data cannot be acquired, the comparison image acquisition function 11c acquires, as comparison image data, third image data, which is image data of the subject photographed in a third examination, which is an examination that occurs before the first examination.
[0077] The first image data and the comparison image data constituting the image pair are acquired by the image processing device described above. However, as described above, the difference image acquisition function 11d, which is the difference image acquisition means, can be configured separately from the configuration having the image acquisition function and the comparison image acquisition function. The difference image acquisition function acquires the difference image data by calculating the difference between the first image data constituting the image combination and the comparison image data.
[0078] The image processing device according to the present disclosure further includes a priority setting function 11e as a priority setting means for setting a priority. The priority setting function 11e compares the date and time difference between the first date and time when the first examination was performed and the second date and time when the comparison image data was obtained with a predetermined value. In the above-mentioned embodiment, the predetermined value is set to, for example, one week or two days. Then, the priority of a combination including the comparison image data and the first image data with a date and time difference larger than the predetermined value is set higher than the priority of a combination including the comparison image data and the first image data with a date and time difference smaller than the predetermined value.
[0079] The priority setting function 11e further changes the setting of the priority between a combination including comparison image data in which the date and time difference between the first date and the second date is greater than a predetermined value and a combination including comparison image data in which the date and time difference is equal to or less than the predetermined value. Specifically, when the date and time difference is greater than a predetermined value, the priority of an image pair including comparison image data in which the difference in the contrast state from the first image data is small is set higher than the priority of an image pair including comparison image data in which the difference in the contrast state is large. Also, when the date and time difference is equal to or less than a predetermined value, the priority of an image pair including comparison image data in which the difference in the contrast state from the first image data is large is set higher than the priority of an image pair including comparison image data in which the difference in the contrast state is small. Then, this priority setting means can set these priorities based on the purpose of the examination. Also, here, when selecting the comparison image data, image pairs are generated and priorities are calculated at that time, but a method of calculating priorities for all image pairs in advance and selecting image pairs according to the priorities may be used.
[0080] In the present disclosure, the image acquisition function 11b acquires candidates for first image data, which are image data obtained by capturing an image of a subject by a first examination, and the comparison image acquisition function 11c acquires candidates for comparison image data for comparison with the first image data. In an embodiment, the priority setting function 11e, as a priority acquisition means, can also acquire priorities for combinations including each of the first image data candidates and each of the comparison image data candidates. When priorities are acquired for such combinations, the image acquisition function 11b can acquire first image data to be used for generating, for example, a difference image from among the candidates for the first image data, according to the acquired priority. Also, the comparison image acquisition function 11c can acquire comparison image data to be used for generating, for example, a difference image from among the candidates for the comparison image data, according to the acquired priority.
[0081] As described above, the present disclosure can determine the priority of generating an image pair based on the degree of coincidence of the imaging site and the size of the overlapping area. According to this modification, it is possible to select an appropriate image pair for visualizing a lesion without causing the user trouble based on the degree of coincidence, and generate a difference image using this. In addition, the priority may be determined using the degree of coincidence and the date and time difference used in this modification. Alternatively, in the first embodiment, when an appropriate image pair cannot be obtained due to a lack of images, for example, the parameter for determining the priority may be changed to the degree of coincidence described in this modification to search for a comparison image. In addition, when the image data includes other information, the degree of coincidence of, for example, incidental information other than the image pair, or the target for referring to the above-mentioned parameters may be these incidental information. Therefore, as the priority when obtaining an image pair consisting of the first image data and the second image data, the priority determined for the combination including the first image data and the second image data may be used.
[0082] <Second embodiment> In the first embodiment described above, the process of step S209 acquires a difference image between the first image data and the comparison image data acquired in the previous step. The process of acquiring this difference image may switch the calculation method based on the relationship between the first image data and the comparison image data, and this case may also be one embodiment of the present invention. An example of this case will be described below as a second embodiment of the present disclosure.
[0083] The overall configuration of the image processing device of this embodiment is similar to that of the first embodiment described with reference to Fig. 1, and therefore detailed description thereof will be omitted here. Also, the flowchart of the process executed by the image processing device of this embodiment is similar to that of the first embodiment described with reference to Fig. 2. However, the process of acquiring differential image data executed as step S209 differs from that of the first embodiment, and therefore the process of acquiring differential image data will be described in detail below.
[0084] Fig. 6 is a flowchart for explaining in detail the process performed in step S209 in this embodiment. As shown in Fig. 6, the image processing device according to this embodiment switches the calculation method of the difference image data depending on whether the first image data and the comparison image data have different shooting dates and times (step S601). Specifically, in step S601, if the first image data and the comparison image data have the same shooting dates and times, the process proceeds to step S602, and if not, the process proceeds to step S603.
[0085] When the imaging dates and times are different, the posture and body shape of the patient often differ between the first image and the comparative image, and it is desirable to perform an appropriate comparison calculation for these images. For this reason, the differential image data is calculated using the first differential image calculation method (step S602).
[0086] A flowchart of the specific process performed in step S602 is shown in Fig. 7(a). In the process shown in Fig. 7(a), the first image data and the comparison image data are aligned using a method that is robust against changes in the posture and body shape of the subject between the images. In the example shown in Fig. 7(a), in step S701, advanced alignment such as the LDDMM (Large Deformation Diffeomorphic Metric Mapping) method is performed. Then, difference image data is calculated using advanced difference processing such as the adaptive voxel matching method that can calculate the difference robustly against the residual error of the deformation alignment (step S702).
[0087] On the other hand, when the imaging dates and times are the same, the difference in the patient's posture and body shape between the first image and the comparative image is small, and the difference is mainly in the contrast state. Therefore, the difference image is calculated using a second difference image calculation method that is simpler than the first difference image calculation method (step S603). Specifically, as shown in FIG. 7(b), a simple registration is performed to correct only the translation and rotation between the two images (step S711). After the registration is completed, the difference image data is calculated using a simple difference process such as a simple subtraction of pixel values at corresponding coordinates of both images (step S712).
[0088] In this case, after the image pair is generated, the method of calculating the difference image is changed considering that the posture and body shape of the patient differ depending on whether the image capturing date and time are different. However, the difference image calculation method may be switched considering, for example, the difference in the contrast state between the first image data and the comparison image data regardless of the image capturing date and time. Specifically, for example, when calculating the difference image between images with different contrast states, it is desirable to exclude organs and blood vessels whose density values change significantly depending on the presence or absence of contrast, and use a difference image calculation method using a registration process or a difference process for other parts.
[0089] As described above, in this embodiment, the difference image acquisition function 11d can execute at least one of the first difference image calculation method (step S602) and the second difference image calculation method (step S603), which are calculation methods different from each other. The difference image acquisition function 11d can select one of these calculation methods based on the comparison image data, and calculate the difference image data using the selected difference image calculation method. More specifically, the first difference image calculation method and the second difference image calculation method each execute a registration process with different characteristics. Then, the difference image data is calculated based on the result of registration between the first image data and the comparison image data using the registration process.
[0090] According to the method described above, by switching the difference image calculation method based on the comparison image data, it is possible to perform an appropriate comparison operation between the first image and the comparison image. In addition, in the case of forming an image pair by referring to the priority based on the degree of match described in the second modification of the first embodiment, this embodiment may be applied so that the alignment method is changed depending on whether the degree of match is low or high.
[0091] <Third embodiment> Another embodiment of the present disclosure includes calculating a difference image from the first image data and a plurality of comparison image data, calculating an evaluation value for each calculation process, and calculating the priority of the image pair based on the evaluation value. In this case, the difference image with the best evaluation value is the difference image that is finally presented to the user. According to this embodiment, a difference image that more appropriately visualizes the lesion can be obtained based on the result of the calculation process of the actual difference image between the first image data and the plurality of comparison image data.
[0092] In this case, the plurality of comparison image data may be all image data other than the first image data obtained by capturing the same subject as the first image data, or may be a part of the image data. In order to select the part of the image data, the method of calculating the priority described in the first embodiment may be used, and a process of calculating a difference image may be executed for image data having a priority equal to or higher than a predetermined value.
[0093] FIG. 8 shows the overall configuration of an image processing device 50 according to this embodiment. The processing circuit 51 of this embodiment is different from the processing circuit 11 in the first and second embodiments in the processing content by the difference image acquisition function 11d. Also, an evaluation function 11f is added. Specifically, the difference image acquisition function 11d according to this embodiment acquires a difference image with an evaluation value, which will be described later, to acquire difference image data and an evaluation value used for calculating the difference image data. Then, the evaluation function 11f selects the difference image data with the highest evaluation value. The following mainly describes these.
[0094] Fig. 9 is a flowchart showing all steps of the image processing device 50 according to this embodiment. Here, in step S901 shown in a dotted line frame in Fig. 9, the processes of steps S201 to S203 in Fig. 2 are executed, and in step S903, the processes of steps S205 to S208 in Fig. 2 are executed. In addition, the process of step S904 shown in Fig. 9 is realized, for example, by the processing circuitry 51 calling a program corresponding to the difference image acquisition function 11d from the storage circuitry 13 and executing it. In addition, the processes of steps S902 and steps S905 to S906 shown in Fig. 9 are realized, for example, by the processing circuitry 11 calling a program corresponding to the evaluation function 11f from the storage circuitry 13 and executing it.
[0095] In step S902, the image processing device 50 repeatedly executes the following steps S903 to S905 as a loop process for the information of the top N image pairs in the image pair list (location of the first image, location of the comparison image). That is, the loop process is performed in order from the information of the image pairs with the highest priority. Here, if N is made large, the calculation process for the difference images will actually be performed for a large number of image pairs, which will take a lot of calculation time, so it is desirable to set N to a value that is about half to a quarter of the length of the list.
[0096] In step S904, the difference image acquisition function 11d acquires difference image data between the first image data and the comparison image data when acquisition of the comparison image data is successful, similar to step S208 in the first embodiment. However, the image processing device 50 of this embodiment differs from the first embodiment in that it acquires difference image data with an evaluation value (step S904).
[0097] Specifically, in step S904, the difference image acquisition function 11d acquires a difference image by calling a calculation process of a difference image with an evaluation value, as shown in Fig. 10(a) (step S1001). Fig. 10 is a flowchart showing steps S1000 to S1001 showing details of the sub-processing performed in step S904. Here, the calculation process of a difference image with an evaluation value may be executed not only inside the image processing device 50, but also in another device in the cloud or the like.
[0098] The evaluation value when calculating the difference image is, for example, a representative value calculated from the following three evaluation values: The evaluation value calculation process executed in steps S1011 to S1013 will be described below with reference to the flowchart in FIG.
[0099] 10(b) is an evaluation value calculated in step S1011, and is an evaluation value related to the alignment process between the first image data and the comparison image data. For example, it is a good evaluation value when there is no local distortion in the spatial correspondence between the images resulting from the alignment process, or when there is high convergence in the transition of the similarity in the iterative process that maximizes the similarity between the images during alignment.
[0100] In general, when images of the same patient are properly aligned, there is little local distortion in the correspondence between the images. Therefore, by evaluating the local distortion, it is possible to evaluate whether the alignment was properly performed. In addition, in general, when there are image features that serve as clues for alignment between images, the image similarity in the repeated processing shows high convergence, and highly reliable alignment is performed. On the other hand, when there are few image features, the convergence of the image similarity is low, and the reliability of the alignment is low. Therefore, it is possible to evaluate the reliability of the alignment by evaluating the convergence.
[0101] The evaluation value 2 in Fig. 10(b) is an evaluation value calculated in step S1012, and is an evaluation value related to the subtraction process. For example, the less noise caused by the partial volume effect or the like in the calculated subtraction image, the better the evaluation value. The amount of noise in the subtraction image can be calculated using known methods such as machine learning and spatial frequency analysis.
[0102] The evaluation value 3 in FIG. 10(b) is an evaluation value calculated in step S1013, and is an evaluation value that indicates how well the difference image data visualizes a lesion. Specifically, for a portion of the calculated difference image with a large difference value, it is determined whether the image characteristics of the first image or the comparison image corresponding to that portion are lesion-like. If that portion is determined to be lesion-like, the evaluation value is good. Note that the determination of whether that portion of the first image or the comparison image is lesion-like can be performed using a known method such as machine learning or spatial autocorrelation.
[0103] The evaluation value at the time of calculating the final difference image can be calculated by, for example, calculating the average value as an integrated value of the above three evaluation values (step S1014). In addition to the average value, the maximum value or minimum value may be used, or the evaluation values may be multiplied together. The above process allows for the calculation of an evaluation value that evaluates whether or not a difference image has been generated in which the images are appropriately aligned, there is little unnecessary noise that hinders the visualization of the lesion, and the lesion portion, which is the purpose of visualization, is easily visible. The difference data with an evaluation value is calculated by the processes from steps S1010 to S1015 described above.
[0104] The differential image data and evaluation value calculated in this way are added to a differential image data list and an evaluation value list, respectively (step S905). After the above calculation process is completed for all image pairs, the evaluation function 11f calculates the index in the evaluation value list that maximizes the evaluation value, and obtains the data in the differential image data list that corresponds to that index as the best differential image data that maximizes the evaluation value (step S906).
[0105] The image processing device according to this embodiment includes an evaluation function 11f (evaluation means) in addition to the above-mentioned image acquisition function 11b (image acquisition means), comparison image acquisition function 11c (comparison image acquisition means), and difference image acquisition function 11d (difference image acquisition means). The image acquisition function 11b acquires first image data, which is image data obtained by photographing a subject by a first examination. The comparison image acquisition function 11c acquires image data obtained by photographing a subject under contrast conditions different from those of the first image data in the first examination as first comparison image data, and acquires image data obtained by photographing a subject by a second examination performed on a different examination date from the first examination as second comparison image data. The difference image acquisition function 11d acquires first difference image data representing the difference between the first image data and the first comparison image data, and second difference image data representing the difference between the first image data and the second comparison image data. The evaluation function 11f evaluates each of the first and second differential image data, and the differential image acquisition function 11d selects the differential image data with the better evaluation result from the first and second differential image data.
[0106] As described above, by using the image processing device according to the present embodiment, it is possible to selectively generate a difference image suitable for visualizing a lesion from among a plurality of difference images, thereby making it possible to generate a difference image for visualizing a lesion without requiring the user to go through complicated procedures.
[0107] <Fourth embodiment> Another embodiment according to the present disclosure also includes a mode in which, when calculating a difference image, a user is asked to confirm the execution using the input interface 15, and the calculated difference image is displayed on the display 14. For example, when the image processing device 10 is implemented as an application executed on a PC, a workstation, or the like, this is useful for improving the usability of the application.
[0108] Fig. 11 shows the overall configuration of an image processing device 60 according to this embodiment. In this embodiment, a display control function 11g is added compared to the first, second, and third embodiments. Specifically, the display control function 11g according to this embodiment performs user confirmation when calculating a difference image and displays the difference image on the display 14. The following mainly describes these functions.
[0109] Fig. 12 is a flowchart showing all steps of the image processing device 60 according to this embodiment. Here, in step S1601 shown in a dotted line frame in Fig. 11, the processes of steps S201 to S203 in the first embodiment are executed, and in step S1603, the processes of steps S205 to S208 are executed. In addition, in step S1605, the process of step S209 in the first embodiment is executed. In addition, the processes of steps S1604, S1606, and S1607 shown in Fig. 11 are realized by, for example, the processing circuitry 61 calling up a program corresponding to the display control function 11g from the storage circuitry 13 and executing it.
[0110] In step S1604, the display control function 11g judges whether to acquire difference image data when the comparison image data acquisition is successful in the process of step S1603, similarly to step S208 in the first embodiment. However, it differs from the first embodiment in that the process is switched based on the user's confirmation (step S1604).
[0111] The process performed in step S1604 will be specifically described with reference to the flowchart of the sub-process shown in Fig. 13. In step S1604, the display control function 11g calls a process for determining whether or not to acquire difference image data, as shown in Fig. 13 (step S1700). In this process, first, for example, as shown in Fig. 16, a confirmation dialog 20 displaying information on the first image and information on the comparison image is displayed on the display 14 (step S1701). Examples of information to be displayed include the examination date and time, series number, etc. In addition to this, cross-sectional images and projection images of the first image and the comparison image may be displayed.
[0112] Next, it is determined whether the user pressed the "Yes" button 21 (to obtain) or the "No" button 22 via the input interface 15 (step S1702). The determination result is stored in the Boolean variable ok (steps S1703, S1704), and after the confirmation dialog 20 is closed (step S1705), the determination result is returned (S1706).
[0113] If the determination result is No (do not acquire), the next candidate in the list of image pairs is taken out and repeated processing is performed (step S1602). Here, instead of performing repeated processing, the processing may be terminated. If the determination result is Yes (acquire), the differential image data is acquired as in the first embodiment (step S1605). If the acquisition of the differential image data has failed, the processing is terminated. At that time, the display control function 11g may display some kind of error dialog on the display 14 (step S1606).
[0114] If the differential image data acquisition is successful, the display control function 11g displays the first image, the comparison image, and the difference image, for example, on an image list screen (step S1607). The process performed in step S1607 will be specifically described with reference to the sub-processing flowchart shown in Fig. 14. Specifically, as shown in Fig. 14, a callback for displaying the given image data on an image detail screen together with the first image data, the comparison image data, and the difference image data is specified, and a process for displaying them in the image list is called (step S1800).
[0115] In this process, an image list screen 30 such as that shown in FIG. 16 is displayed on the display 14 (step S1801). The image list screen 30 may display thumbnails in a list, instead of in a matrix. Next, a thumbnail 31 of the first image is displayed at a predetermined position determined by the examination date and time, modality, etc. of the first image (step S1802). Next, a thumbnail 32 of a comparison image is also displayed so as to be contrasted with the thumbnail of the first image (step S1803). At this time, the display is made so that the difference between the examination date and time and the difference between the contrast conditions can be understood. For example, text information on the difference between the examination dates and times (number of days) and marks indicating whether the examination dates and times are within or outside the range of the date and time difference boundary value can be displayed adjacent to or superimposed on the thumbnail. Alternatively, text information or marks indicating the presence or absence of a difference in the contrast condition and its size can be displayed adjacent to or superimposed on the thumbnail.
[0116] Furthermore, thumbnails 33 (in the case of time-lapse subtraction) and 34 (in the case of contrast phase subtraction) of the subtraction images are also displayed (step S1804). In this case, if the purpose of the examination is to visualize intraspinal infiltration, for example, the contrast phase subtraction images with low priority are marked with a star icon 35 in FIG. 17 so as to be distinguished from the time-lapse subtraction images, thereby alerting the user (step S1804). Conversely, a time-lapse subtraction image with high priority may be marked so as to be distinguished from other time-lapse subtraction images and contrast phase subtraction images with low priority. In addition, a callback is set for each thumbnail, which is called when the thumbnail is selected by the input interface 15 (step S1805). When this callback is called, related image data is passed together to the argument of the callback (step S1806).
[0117] Fig. 15 shows a flowchart from step S1900 to step S1905 regarding the callback process for displaying each image on the image detail screen. In the callback process, first, the display control function 11g displays an image detail screen (any of 40a, 40b, 40c, 40d, or 40e) on the display 14 as shown in Fig. 18 (step S1901). The image detail screen may not only display images in two columns, but also in a single vertical or horizontal row.
[0118] Next, the first image (current image) is displayed on the image details screen (step S1902). Next, a comparison image (past image) is displayed so that it can be compared with the first image (step S1903). At this time, for example, labels indicating the current image and past image and labels indicating contrast and non-contrast are displayed at the top of each image, so that the difference in examination date and time and the difference in contrast state of the images can be seen. Furthermore, a difference image is also displayed on the image details screen (step S1904). At this time, for example, the image is displayed so that it can be seen whether it is a time-dependent difference image or a contrast-enhanced interphase difference image, as in the label display of the image details screen 42a to image details screen 42e in FIG. 18.
[0119] Furthermore, for example, if the purpose of the examination is to visualize intraspinal infiltration, contrast phase difference images with a lower priority may be marked, for example, with a star icon 41 in Fig. 18, so as to be distinguished from time-lapse difference images. Conversely, a time-lapse difference image with the highest priority may be marked so as to be distinguished from other time-lapse difference images and contrast phase difference images.
[0120] 12, it has been explained that thumbnails are first displayed on the image list screen, and when a thumbnail is selected via the input interface 15, an image detail screen is displayed. However, the aspects of this embodiment are not limited to the example given here, and the image detail screen may be displayed directly at the time of step S1607. The processes from steps S1600 to S1608 described above make it possible to appropriately display the desired difference image.
[0121] As described above, the image processing device according to this embodiment further includes a display control function 11g (display control means) that performs display processing of the image pair on the display unit. This display control function 11g can be configured to display a confirmation screen (FIG. 15) for asking whether or not to acquire differential image data on the display unit. The display control function 11g can also be configured to display the first image data and the comparison image data in a contrastive manner (FIG. 16). The display control function 11g can also be configured to display the first image data, the comparison image data, and the differential image data in a contrastive manner (FIG. 17). And the display control function 11g can also be configured to display information corresponding to the priority (star icon 35 in FIG. 17).
[0122] As described above, by using the image processing device according to this embodiment, it is possible to present to the user whether each of the various subtraction images is suitable for the purpose of the examination, or the degree of suitability for the purpose of the examination. This improves the usability of the application, and makes it possible to generate subtraction images for visualizing lesions without requiring the user to go through complicated procedures.
[0123] The above disclosure includes the following configurations, methods, and programs. (Configuration 1) A first image acquisition means for acquiring first image data which is image data obtained by photographing a subject through a first examination; an image processing device comprising: a comparison image acquisition means for acquiring comparison image data for comparison with the first image data, the comparison image data being acquired according to a priority for a combination including the first image data and a candidate for the comparison image data, the priority being calculated based on the examination date and time of the first examination and the contrast state of the first image data. (Configuration 2) 20. The image processing device according to any one of configurations 1, 16, 17, and 19, further comprising a difference image acquisition means for acquiring difference image data between the acquired first image data and the comparison image data. (Configuration 3) 3. The image processing device according to configuration 2, wherein the differential image acquisition means acquires the differential image data by calculating a difference between the acquired first image data and the comparison image data. (Configuration 4) The image processing device according to configuration 3, wherein the difference image acquisition means selects at least one of a first difference image calculation method and a second difference image calculation method, which are mutually different calculation methods, based on the comparison image data, and calculates the difference image data using the selected difference image calculation method. (Configuration 5) The image processing device according to configuration 4, wherein the first difference image calculation method and the second difference image calculation method each include an alignment process having different characteristics, and the difference image data is calculated based on a result of alignment between the first image data and the comparison image data using the alignment process. (Configuration 6) 20. The image processing device according to any one of configurations 1 to 5, 16, 17, and 19, further comprising a priority setting means for setting the priority. (Configuration 7) The image processing device of configuration 6, wherein the priority setting means sets a priority of a combination including the first image data and comparison image data in which the date and time difference between a first date and time when the first inspection was performed and a second date and time when the comparison image data was obtained is greater than a predetermined value, higher than a priority of a combination including the first image data and comparison image data in which the date and time difference is smaller than the predetermined value. (Configuration 8) The image processing device according to configuration 6, wherein the priority setting means sets a priority of a combination including comparison image data in which the difference in contrast state from the first image data is small, higher than a priority of a combination including comparison image data in which the difference in contrast state from the first image data is large, when the combination includes comparison image data in which the date and time difference between a first date and time when the first examination was performed and a second date and time when the comparison image data was obtained is larger than a predetermined value. (Configuration 9) The image processing device according to configuration 6, wherein the priority setting means sets a priority of a combination including comparison image data in which the difference in contrast state from the first image data is large higher than a priority of a combination including comparison image data in which the difference in contrast state from the first image data is small when the combination includes comparison image data in which the date and time difference between a first date and time when the first examination was performed and a second date and time when the comparison image data was obtained is less than a predetermined value. (Configuration 10) 7. The image processing apparatus according to configuration 6, wherein the priority setting means sets priorities based on an inspection purpose. (Configuration 11) 20. The image processing device according to claim 1, further comprising a display control means for performing a display process of an image pair including the first image data and the selected comparison image data on a display unit. (Configuration 12) A display control unit is further provided for performing a display process of an image pair including the first image data and the selected comparison image data on a display unit, 19. The image processing device according to any one of configurations 2 to 5 and 18, wherein the display control means causes the display unit to display a confirmation screen for asking whether or not to obtain the differential image data. (Configuration 13) 12. The image processing device according to configuration 11, wherein the display control means displays the first image data and the comparison image data so as to be contrasted with each other. (Configuration 14) A display control unit is further provided for performing a display process of an image pair including the first image data and the selected comparison image data on a display unit, 19. The image processing device according to any one of configurations 2 to 5 and 18, wherein the display control means displays the first image data, the comparison image data, and the difference image data in a comparative manner. (Configuration 15) 12. The image processing device according to claim 11, wherein the display control means displays information corresponding to the priority. (Configuration 16) A first image acquisition means for acquiring first image data which is image data obtained by photographing a subject through a first examination; a comparison image acquisition means for acquiring comparison image data for comparison with the first image data, The comparative image acquisition means attempts to acquire second image data, which is image data obtained by photographing the subject in a second examination that is an examination prior to the first examination, as comparative image data; When the second image data cannot be acquired, the image processing device acquires third image data, which is image data in a contrast state different from that of the first image data, as comparison image data. (Configuration 17) A first image acquisition means for acquiring first image data which is image data obtained by photographing a subject through a first examination; a comparison image acquisition means for acquiring comparison image data for comparison with the first image data, The comparative image acquisition means attempts to acquire second image data, which is image data in a contrast state different from that of the first image data, as comparative image data; an image processing device that, when the second image data cannot be acquired, acquires third image data, which is image data obtained by photographing the subject through a third examination that is an examination prior to the first examination, as comparison image data. (Configuration 18) A first image acquisition means for acquiring first image data which is image data obtained by photographing a subject through a first examination; a comparison image acquisition means for acquiring image data obtained by imaging the subject under contrast conditions different from those of the first image data in the first examination as first comparison image data, and acquiring image data obtained by imaging the subject in a second examination performed on a different examination date from that of the first examination as second comparison image data; a differential image acquisition means for acquiring first differential image data representing a difference between the first image data and the first comparison image data, and second differential image data representing a difference between the first image data and the second comparison image data; an evaluation means for evaluating each of the first differential image data and the second differential image data, an image processing device that selects, from the first differential image data and the second differential image data, differential image data that has a better evaluation result; (Configuration 19) a priority acquisition means for acquiring priorities for combinations including candidates of first image data, which are image data obtained by capturing an image of a subject in a first examination, and candidates of comparison image data for comparing with the first image data; a first image acquisition means for acquiring the first image data from the candidates of the first image data according to the priority; a comparison image acquisition means for acquiring the comparison image data from the comparison image data candidates according to the priority; An image processing device comprising: (Method 1) Acquiring first image data which is image data obtained by photographing a subject through a first examination; and acquiring comparison image data for comparison with the first image data, the comparison image data being acquired according to a priority for a combination including the first image data and the candidate comparison image data, the priority being calculated based on the examination date and time of the first examination and the contrast state of the first image data. (Method 2) Acquiring first image data which is image data obtained by photographing a subject through a first examination; obtaining comparison image data for comparison with the first image data; In the acquiring of the comparison image data, an attempt is made to acquire second image data, which is image data obtained by capturing an image of the subject by a second examination that is an examination prior to the first examination, as the comparison image data; When the second image data cannot be acquired, third image data, which is image data in a contrast state different from that of the first image data, is acquired as comparison image data. (Method 3) Acquiring first image data which is image data obtained by photographing a subject through a first examination; obtaining comparison image data for comparison with the first image data; The acquisition of the comparison image data includes attempting to acquire second image data, which is image data in a contrast state different from that of the first image data, as the comparison image data; An image processing method, in which, when the second image data cannot be acquired, third image data, which is image data of the subject photographed by a third examination that is an examination prior to the first examination, is acquired as comparison image data. (Method 4) Acquiring first image data which is image data obtained by photographing a subject through a first examination; acquiring image data obtained by imaging the subject under contrast conditions different from those of the first image data in the first examination as first comparison image data, and acquiring image data obtained by imaging the subject in a second examination performed on an examination date different from that of the first examination as second comparison image data; obtaining first differential image data representing a difference between the first image data and the first comparison image data, and second differential image data representing a difference between the first image data and the second comparison image data; evaluating each of the first difference image data and the second difference image data; the image processing method further comprising selecting, from the first differential image data and the second differential image data, differential image data which provides a better evaluation result. (Method 5) acquiring priorities for combinations including candidates of first image data, which are image data obtained by photographing a subject in a first examination, and candidates of comparison image data for comparing with the first image data; acquiring the first image data from the candidates of the first image data according to the priority; acquiring the comparison image data from the candidates of the comparison image data according to the priority; An image processing method comprising: (program) A program for causing a computer to execute the image processing method according to any one of Methods 1 to 5.
[0124] <Other embodiments> Although the embodiment has been described above in detail, the present invention can be embodied as, for example, a system, an apparatus, a method, a program, or a recording medium (storage medium), etc. Specifically, the present invention may be applied to a system composed of multiple devices (for example, a host computer, an interface device, an imaging device, a Web application, etc.), or may be applied to an apparatus composed of a single device.
[0125] Needless to say, the object of the present disclosure can be achieved by the following: That is, a recording medium (or storage medium) on which is recorded a program code (computer program) of software that realizes the functions of the above-mentioned embodiments is supplied to a system or device. Needless to say, such a storage medium is a computer-readable storage medium. Then, a computer (or a CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-mentioned embodiments, and the recording medium on which the program code is recorded constitutes the present disclosure. [Explanation of symbols]
[0126] 10, 50, 60: Image processing device 11, 51, 61: Processing circuit 11a: Control functions 11b: Image acquisition function 11c: Comparison image acquisition function 11d: Differential image acquisition function 11e: Priority setting function 12: Communication interface 13:Memory circuit 14: Display 15: Input interface
Claims
1. a first image acquisition means for acquiring first image data which is image data obtained by photographing a subject through a first examination; an image processing device comprising: a comparison image acquisition means for acquiring comparison image data for comparison with the first image data, the comparison image data being acquired according to a priority for a combination including the first image data and a candidate for the comparison image data, the priority being calculated based on the examination date and time of the first examination and the contrast state of the first image data.
2. The image processing apparatus according to claim 1 , further comprising a differential image acquisition unit that acquires differential image data between the acquired first image data and the comparison image data.
3. The image processing apparatus according to claim 2 , wherein the differential image acquisition means acquires the differential image data by calculating a difference between the acquired first image data and the comparison image data.
4. 4. The image processing device according to claim 3, wherein the difference image acquisition means selects at least one of a first difference image calculation method and a second difference image calculation method, which are mutually different calculation methods, based on the comparison image data, and calculates the difference image data using the selected difference image calculation method.
5. 5. The image processing device according to claim 4, wherein the first difference image calculation method and the second difference image calculation method each include a registration process having different characteristics, and the difference image data is calculated based on a result of registration between the first image data and the comparison image data using the registration process.
6. The image processing apparatus according to claim 1 , further comprising a priority setting unit for setting the priority.
7. 7. The image processing device according to claim 6, wherein the priority setting means sets a priority of a combination including comparison image data and the first image data in which the date and time difference between a first date and time when the first inspection was performed and a second date and time when the comparison image data was obtained is greater than a predetermined value, higher than a priority of a combination including comparison image data and the first image data in which the date and time difference is smaller than the predetermined value.
8. 7. The image processing device according to claim 6, wherein the priority setting means sets a priority of a combination including comparison image data in which a date and time difference between a first date and time when the first examination was performed and a second date and time when the comparison image data was obtained is greater than a predetermined value, to be higher than a priority of a combination including comparison image data in which a difference in contrast state from the first image data is small.
9. 7. The image processing device according to claim 6, wherein the priority setting means sets a priority of a combination including comparison image data in which a difference in contrast state from the first image data is large higher than a priority of a combination including comparison image data in which a difference in contrast state from the first image data is small when the combination includes comparison image data in which the date and time difference between a first date and time when the first examination was performed and a second date and time when the comparison image data was obtained is less than a predetermined value.
10. The image processing apparatus according to claim 6 , wherein the priority setting means sets the priority based on an inspection purpose.
11. 11. The image processing apparatus according to claim 1, further comprising a display control unit that performs a display process for an image pair including the first image data and the selected comparison image data on a display unit.
12. A display control unit is further provided for performing a display process of an image pair including the first image data and the selected comparison image data on a display unit, The image processing apparatus according to claim 2 , wherein the display control means causes the display unit to display a confirmation screen for asking whether or not to obtain the difference image data.
13. The image processing apparatus according to claim 11 , wherein the display control means displays the first image data and the comparison image data so as to be contrasted with each other.
14. A display control unit is further provided for performing a display process of an image pair including the first image data and the selected comparison image data on a display unit, The image processing apparatus according to claim 2 , wherein the display control means displays the first image data, the comparison image data, and the difference image data in a comparative manner.
15. The image processing device according to claim 11 , wherein the display control means displays information corresponding to the priority.
16. a first image acquisition means for acquiring first image data which is image data obtained by photographing a subject through a first examination; a comparison image acquisition means for acquiring comparison image data for comparison with the first image data, The comparative image acquisition means attempts to acquire second image data, which is image data obtained by photographing the subject in a second examination that is an examination prior to the first examination, as comparative image data; When the second image data cannot be acquired, the image processing apparatus acquires third image data, which is image data in a contrast state different from that of the first image data, as comparison image data.
17. a first image acquisition means for acquiring first image data which is image data obtained by photographing a subject through a first examination; a comparison image acquisition means for acquiring comparison image data for comparison with the first image data, The comparative image acquisition means attempts to acquire second image data, which is image data in a contrast state different from that of the first image data, as comparative image data; an image processing device that, when the second image data cannot be acquired, acquires third image data, which is image data obtained by photographing the subject through a third examination that is an examination prior to the first examination, as comparison image data.
18. a first image acquisition means for acquiring first image data which is image data obtained by photographing a subject through a first examination; a comparison image acquisition means for acquiring image data obtained by imaging the subject under contrast conditions different from those of the first image data in the first examination as first comparison image data, and acquiring image data obtained by imaging the subject in a second examination performed on a different examination date from that of the first examination as second comparison image data; a differential image acquisition means for acquiring first differential image data representing a difference between the first image data and the first comparison image data, and second differential image data representing a difference between the first image data and the second comparison image data; an evaluation means for evaluating each of the first differential image data and the second differential image data, an image processing device that selects, from the first differential image data and the second differential image data, differential image data that has a better evaluation result;
19. a priority acquisition means for acquiring priorities for combinations including candidates of first image data, which are image data obtained by photographing a subject in a first examination, and candidates of comparison image data for comparing with the first image data; a first image acquisition means for acquiring the first image data from candidates of the first image data according to the priority; a comparison image acquisition means for acquiring the comparison image data from the comparison image data candidates according to the priority; An image processing device comprising:
20. acquiring first image data obtained by imaging a subject through a first examination; and acquiring comparison image data for comparison with the first image data, the comparison image data being acquired according to a priority for a combination including the first image data and the candidate comparison image data, the priority being calculated based on the examination date and time of the first examination and the contrast state of the first image data.
21. acquiring first image data obtained by imaging a subject through a first examination; obtaining comparison image data for comparison with the first image data; The acquisition of the comparison image data includes attempting to acquire second image data, which is image data of the subject photographed by a second examination that is an examination prior to the first examination, as the comparison image data; When the second image data cannot be acquired, third image data, which is image data in a contrast state different from the first image data, is acquired as comparison image data.
22. acquiring first image data obtained by imaging a subject through a first examination; obtaining comparison image data for comparison with the first image data; The acquisition of the comparison image data includes attempting to acquire second image data, which is image data in a contrast state different from that of the first image data, as the comparison image data; An image processing method, in which, when the second image data cannot be acquired, third image data, which is image data of the subject photographed by a third examination that is an examination prior to the first examination, is acquired as comparison image data.
23. acquiring first image data obtained by imaging a subject through a first examination; acquiring image data obtained by imaging the subject under contrast conditions different from those of the first image data in the first examination as first comparison image data, and acquiring image data obtained by imaging the subject in a second examination performed on an examination date different from that of the first examination as second comparison image data; acquiring first differential image data representing a difference between the first image data and the first comparison image data, and second differential image data representing a difference between the first image data and the second comparison image data; evaluating each of the first difference image data and the second difference image data; the image processing method further comprising: selecting, from the first differential image data and the second differential image data, differential image data that provides a better evaluation result.
24. acquiring priorities for combinations including candidates of first image data, which are image data obtained by photographing a subject in a first examination, and candidates of comparison image data for comparing with the first image data; acquiring the first image data from the candidates of the first image data according to the priority; acquiring the comparison image data from the candidates of the comparison image data according to the priority; An image processing method comprising:
25. A program for causing a computer to execute the image processing method according to any one of claims 20 to 24.
Citation Information
Cited By
Solid-state imaging device
CN121728846A
