Medical image processing device, medical image processing method, and program

The medical image processing apparatus enhances alignment of images by using feature points and user corrections to address inefficiencies in aligning images with varying tumor shapes and subject conditions, ensuring accurate correspondence and efficient image interpretation.

JP2024162451AActive Publication Date: 2024-11-21CANON KK
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2023077961
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-21
Estimated Expiration
2043-05-10

AI Technical Summary

Technical Problem

Efficient alignment of medical images taken at different times is complicated and inefficient due to variations in tumor shape and subject condition, making manual matching difficult and prone to errors.

Method used

A medical image processing apparatus with features including an acquisition unit, extraction unit, association unit, reception unit, calculation unit, and correction unit, which aligns medical images using feature points, accepts user corrections, and adjusts positional differences to improve alignment accuracy.

Benefits of technology

Facilitates efficient alignment of medical images by allowing users to correct alignment errors, presenting correction candidates, and ensuring accurate correspondence between images taken at different times, thereby supporting efficient image interpretation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162451000001_ABST
    Figure 2024162451000001_ABST
Patent Text Reader

Abstract

To efficiently align medical images captured at different times.SOLUTION: A medical image processing device includes an acquisition unit, an extraction unit, a first association unit, a reception unit, a calculation unit, and a correction unit. The acquisition unit acquires a first medical image, and a second medical image that is captured at a time different from that of the first medical image and includes an area captured in the first medical image. The extraction unit extracts a first feature point from the first medical image and extracts a second feature point from the second medical image. The first association unit associates the first feature point and the second feature point with each other that exist in corresponding parts of the first medical image and the second medical image on the basis of an alignment result between the first medical image and the second medical image. The reception unit receives a correction input that associates the first feature point with another second feature point among the pairs of the first feature point and the second feature point that are associated by the first association unit. The calculation unit calculates an alignment difference on the second medical image between the second feature points before and after the correction related to the correction input. The correction unit corrects the alignment result between the first medical image and the second medical image on the basis of the alignment difference.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to a medical image processing apparatus, a medical image processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, medical professionals such as image diagnosticians and radiologists perform image interpretation using a plurality of medical images captured by a medical image diagnostic apparatus.

[0003] Furthermore, when a user who is a medical professional interprets a tumor that requires follow-up, the user may interpret the tumor by associating the position of the tumor in a medical image that captures the current condition of the subject with the position of the corresponding tumor in a medical image that captures the past condition of the subject. For example, if a tumor that existed in a past medical image has shrunk or enlarged in the current medical image, the corresponding tumors are associated with each other so that the transition of the tumor condition can be confirmed in chronological order.

[0004] However, it is cumbersome and inefficient for a user to perform the matching work manually. In addition, although it is possible to use a known registration technique, for example, when a specific tumor shape is used for registration, the registration may not be performed normally depending on the condition of the subject, such as when the shape of the tumor is significantly different between medical images to be registered. For this reason, a technique that can perform registration efficiently is desired. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2009-160045 A Summary of the Invention [Problem to be solved by the invention]

[0006] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to efficiently align medical images taken at different times. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0007] The medical image processing apparatus according to the present embodiment includes an acquisition unit, an extraction unit, a first matching unit, a reception unit, a calculation unit, and a correction unit. The acquisition unit acquires a first medical image and a second medical image that is captured at a time different from the first medical image and includes a region captured in the first medical image. The extraction unit extracts a first feature point from the first medical image and extracts a second feature point from the second medical image. The first matching unit matches the first feature point and the second feature point that exist in the corresponding parts of both images based on the alignment result between the first medical image and the second medical image. The reception unit receives a correction input that matches the first feature point with another second feature point among the pairs of the first feature point and the second feature point that are matched by the first matching unit. The calculation unit calculates the position difference on the second medical image between the second feature points before and after the correction related to the correction input. The correction unit corrects the result of alignment between the first medical image and the second medical image based on the position difference. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a medical image processing system including a medical image processing apparatus according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a medical image processing apparatus according to the first embodiment. [Diagram 3] FIG. 3 is a diagram for explaining an example of a display process of a disease site association result according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a reception process for correcting an association result of a disease site according to the embodiment. [Diagram 5]FIG. 5 is a diagram illustrating an example of a presentation process of correction candidates for a disease site association result according to the embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of processing executed by the medical image processing apparatus according to the embodiment. [Figure 7] FIG. 7 is a diagram for explaining an example of a display process of a disease site association result according to a modified example. [Figure 8] FIG. 8 is a diagram for explaining an example of a receiving process for correcting an association result of a disease site according to a modified example. [Figure 9] FIG. 9 is a diagram for explaining an example of a presentation process of correction candidates for a disease site association result according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of a medical image processing apparatus, a medical image processing method, and a program will be described in detail with reference to the drawings. Note that the following description will be given by taking a medical image processing system including the medical image processing apparatus as an example.

[0010] 1 is a diagram showing an example of the configuration of a medical image processing system 1 including a medical image processing device 100 according to an embodiment. The medical image processing system 1 shown in FIG. 1 includes the medical image processing device 100, a medical image diagnostic device 2, and an image storage device 3.

[0011] The medical image processing device 100 is connected to a medical image diagnostic device 2 and an image storage device 3 via a network 4, such as an in-hospital LAN (Local Area Network) installed in a hospital. Here, each device is capable of communicating with each other directly or indirectly. For example, when a PACS (Picture Archiving and Communication System) is introduced in the medical image processing system 1, each device transmits and receives medical images and the like to each other in accordance with the DICOM (Digital Imaging and Communications in Medicine) standard.

[0012] The medical image diagnostic apparatus 2 generates and collects medical images. The medical image diagnostic apparatus 2 is, for example, an X-ray CT (Computed Tomography) apparatus or an X-ray diagnostic apparatus.

[0013] The medical image diagnostic apparatus 2 may be, for example, a magnetic resonance imaging (MRI) apparatus, an ultrasonic diagnostic apparatus, a single photon emission computed tomography (SPECT) apparatus, a positron emission computed tomography (PET) apparatus, etc. The medical image diagnostic apparatus 2 may also be a SPECT-CT apparatus in which a SPECT apparatus and an X-ray CT apparatus are integrated, a PET-CT apparatus in which a PET apparatus and an X-ray CT apparatus are integrated, or a group of these apparatuses.

[0014] The medical image diagnostic apparatus 2 is capable of generating two-dimensional medical images, three-dimensional medical images (volume data), two-dimensional medical images along a time series, three-dimensional medical images along a time series, and the like.

[0015] Here, the medical image diagnostic device 2 collects medical images by photographing a subject. For example, an X-ray CT device, which is an example of the medical image diagnostic device 2, rotates an X-ray tube and an X-ray detector around the subject, detects X-rays that have passed through the subject, and collects projection data.

[0016] Based on the collected projection data, X-ray CT devices generate various medical images, such as two-dimensional CT images, three-dimensional CT images (volume data), two-dimensional CT images in time series, or three-dimensional CT images in time series.

[0017] Furthermore, the X-ray CT apparatus may generate a plurality of two-dimensional CT images along a predetermined direction based on the collected projection data. For example, the X-ray CT apparatus may generate a plurality of two-dimensional CT images of axial slices along the body axis direction.

[0018] The medical image diagnostic apparatus 2 transmits the generated medical images to the image storage apparatus 3. When transmitting the medical images to the image storage apparatus 3, the medical image diagnostic apparatus 2 transmits, as additional information, for example, a patient ID for identifying the patient, an examination ID for identifying the examination, an apparatus ID for identifying the medical image diagnostic apparatus 2, a series ID for identifying one imaging session by the medical image diagnostic apparatus 2, and the like.

[0019] The image storage device 3 is a database that stores medical images. Specifically, the image storage device 3 includes a memory circuit, and stores the medical images transmitted from the medical image diagnostic device 2 in the memory circuit, thereby storing the medical images.

[0020] The memory circuit of the image storage device 3 is, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The medical images stored in the image storage device 3 are stored in association with a patient ID, an examination ID, an apparatus ID, a series ID, etc. Therefore, the medical image processing device 100 can acquire the required medical images from the image storage device 3 by performing a search using the patient ID, the examination ID, the apparatus ID, the series ID, etc.

[0021] The medical image processing device 100 is an image processing device that performs image processing on medical images. The medical image processing device 100 is, for example, a workstation, an image server or viewer of a PACS (Picture Archiving and Communication System), or various devices of an electronic medical record system.

[0022] The medical image processing device 100 performs various processes on medical images acquired from the medical image diagnostic device 2 or the image storage device 3. In this embodiment, the medical image processing device 100 is a device used by medical professionals such as image diagnosticians and radiologists to interpret medical images.

[0023] 2 is a block diagram showing an example of a configuration of a medical image processing apparatus 100 according to an embodiment. As shown in FIG. 2, the medical image processing apparatus 100 includes an input interface 110, a display 120, a communication interface 130, a storage circuitry 140, and a processing circuitry 150.

[0024] The input interface 110 has a pointing device such as a mouse, a keyboard, etc., and receives various operation inputs for the medical image processing device 100 from a user (doctor), and transfers the instructions and setting information received from the user to the processing circuitry 150.

[0025] The display 120 is a monitor viewed by a user. Under the control of the processing circuit 150, the display 120 displays an image to the user and displays a GUI (Graphical User Interface) for receiving various instructions and various settings from the user via the input interface 110. The display 120 is an example of a display unit. The communication interface 130 is a NIC (Network Interface Card) or the like, and communicates with other devices.

[0026] The memory circuitry 140 is, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The memory circuitry 140 stores medical images acquired from the medical image diagnostic device 2 or the image storage device 3.

[0027] The processing circuitry 150 controls the components of the medical image processing device 100. For example, as shown in FIG. 2, the processing circuitry 150 executes a display control function 151, an acquisition function 152, a positioning function 153, an extraction function 154, a correspondence function 155, a reception function 156, a calculation function 157, a correction function 158, and a presentation function 159.

[0028] Here, for example, each processing function executed by the components of the processing circuit 150, such as the display control function 151, the acquisition function 152, the alignment function 153, the extraction function 154, the matching function 155, the reception function 156, the calculation function 157, the correction function 158, and the presentation function 159, is recorded in the memory circuit 140 in the form of a program executable by a computer.

[0029] The processing circuitry 150 is a processor that realizes a function corresponding to each program by reading and executing each program from the storage circuitry 140. In other words, the processing circuitry 150 in a state in which each program has been read has each function shown in the processing circuitry 150 in FIG.

[0030] Here, display control function 151 is an example of a display control unit. Acquisition function 152 is an example of an acquisition unit. Extraction function 154 is an example of an extraction unit. Correlation function 155 is an example of a first correlation unit and a second correlation unit. Reception function 156 is an example of a reception unit. Calculation function 157 is an example of a calculation unit. Correction function 158 is an example of a correction unit. Presentation function 159 is an example of a presentation unit.

[0031] The term "processor" used in the above description refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), or an application specific integrated circuit (ASIC).

[0032] Furthermore, the term "processor" refers to a circuit such as a programmable logic device. Examples of programmable logic devices include a simple programmable logic device (SPLD) and a complex programmable logic device (CPLD).

[0033] Also, an example of a programmable logic device is a field programmable gate array (FPGA). When the processor is, for example, a CPU, the processor realizes a function by reading and executing a program stored in the memory circuit 140. On the other hand, when the processor is, for example, an ASIC, instead of storing a program in the memory circuit 140, the program is directly built into the circuit of the processor.

[0034] In addition, each processor in this embodiment is not limited to being configured as a single circuit, but may be configured as a single processor by combining multiple independent circuits to realize its functions. Furthermore, multiple components in FIG. 2 may be integrated into a single processor to realize its functions.

[0035] The overall configuration of the medical image processing system 1 including the medical image processing device 100 according to this embodiment has been described above. With such a configuration, the medical image processing device 100 supports efficient alignment of medical images. Specifically, the processing circuit 150 of the medical image processing device 100 includes a display control function 151, an acquisition function 152, an alignment function 153, an extraction function 154, a matching function 155, a reception function 156, a calculation function 157, a correction function 158, and a presentation function 159 as functional units, and performs the following processes.

[0036] The display control function 151 controls the display of various information on a display device. For example, the display control function 151 displays medical images, GUIs, and the like on the display 120. The various information that the display control function 151 displays on the display 120 will be described later.

[0037] The acquisition function 152 acquires medical images. For example, the acquisition function 152 acquires a medical image (hereinafter also referred to as a current medical image) capturing a current state of a target subject to be diagnosed from the medical image diagnostic device 2 or the medical image storage device 3 using a patient ID of the target subject to be diagnosed. The current medical image in this case is an example of a first medical image. The current medical image may be an image capturing the most recently captured state of the target subject. The current medical image includes an area representing the state of the target part to be diagnosed of the target subject.

[0038] Similarly, the acquisition function 152 acquires a medical image of the target subject captured earlier than the current medical image (hereinafter, also referred to as a past medical image) to be associated with the current medical image. The past medical image in this case is an example of a second medical image. The past medical image includes an area representing the target part of the target subject.

[0039] The registration function 153 aligns a medical image with another medical image captured at a different time from the medical image in question. For example, the registration function 153 aligns a current medical image with a past medical image.

[0040] As an example, the registration function 153 extracts one or more anatomical landmarks from the current medical image and the past medical image, and aligns the anatomical landmarks of the current medical image and the past medical image by superimposing the anatomical landmarks of the current medical image and the past medical image. If the target site is the liver, the registration function 153 performs the above registration using anatomical feature points of the liver based on morphological characteristics as anatomical landmarks.

[0041] The registration function 153 may align the current medical image and the past medical image using a segmentation result of a known segmentation technique.

[0042] The extraction function 154 extracts a disease site from a medical image. For example, the extraction function 154 extracts a disease site from a current medical image and a past medical image. In this case, the disease site on the current medical image is an example of a first feature point, and the disease site on the past medical image is an example of a second feature point. The disease site is, for example, a tumor developed in the target subject. A known technique can be appropriately used to extract the disease site. The extraction function 154 uses the same method to extract the disease site from the current medical image and the disease site from the past medical image.

[0043] As an example, the extraction function 154 extracts a disease site from an input medical image using a trained model that is functionalized to infer information representing a disease site on the medical image (information representing the position of the disease site and the shape of the disease site in the medical image). The trained model in this case is a trained model that has been trained using a medical image as input-side teacher data and information representing a disease site as output-side teacher data using known machine learning, deep learning, or the like.

[0044] For past medical images, the extraction results of disease sites may be stored in a storage device or the like of the medical image storage device 3, and the acquisition function 152 may acquire the extraction results of disease sites of the medical images together with the medical images.

[0045] The matching function 155 matches a disease site on a medical image with a disease site on another medical image different from the medical image. For example, the matching function 155 uses a block matching method to match the position of the disease site on the current medical image extracted by the extraction function 154 with the position of the disease site on the past medical image.

[0046] In this case, for example, the matching function 155 detects, for each area representing a diseased site on the current medical image, a small area that has a degree of correspondence higher than a certain level between a specific pixel of the area and a small area including the pixels surrounding the pixel, from among the areas representing the diseased site on the past medical image.

[0047] Here, methods for calculating the degree of match include SAD (Sum of Absolute Difference) and SSD (Sum of Squared Difference). In SAD, the association function 155 calculates the absolute value of the difference for each pixel between a small area on the current medical image and a small area on the past medical image, and the sum of the absolute values ​​is used as an evaluation value of the degree of match. The smaller the evaluation value, the higher the degree of match.

[0048] In addition, in the SAD, the matching function 155 squares the difference between each pixel of a small area on the current medical image and a small area on the past medical image, and the sum of the squares is used as an evaluation value of the degree of match. The smaller the evaluation value, the higher the degree of match.

[0049] When the position of a detected small area on a past medical image is within a specified range of a position on the past medical image corresponding to the position of a small area on the current medical image, the matching function 155 matches the disease area relating to the small area on the current medical image with the disease area relating to the small area on the past medical image.

[0050] The above is just one example of a method for associating disease sites, and disease sites may be associated using a method other than the block matching method.

[0051] The result of the association is displayed on the display 120 or the like under the control of the display control function 151. Here, Fig. 3 is a diagram for explaining an example of a display process of the disease site association result. Fig. 3 shows an example of a first screen 120A that displays the disease site association result.

[0052] In the example of Fig. 3, the first screen 120A is composed of a display area with two vertical columns. In the upper part of Fig. 3, diseased parts P1 to P3 on the past medical image are displayed. In addition, in the lower part, diseased parts C1 to C3 on the current medical image are displayed. Here, if the target part is the liver, diseased parts P1 to P3 and diseased parts C1 to C3 represent tumors and the like occurring in the liver of the target subject.

[0053] As an example, the diseased sites P1-P3 are displayed in a horizontal order according to the positions of the diseased sites P1-P3 on the previous medical image in the body axis direction of the subject. The same applies to the diseased sites C1-C3. Note that the diseased sites P1-P3 and C1-C3 may be displayed in a horizontal order according to the positions of the diseased sites in a direction perpendicular to the body axis direction.

[0054] 3 is an example, and the number of disease sites on the current medical image and disease sites on the past medical images is not limited to 3. In addition, the number of disease sites on the current medical image and the number of disease sites on the past medical images may be different.

[0055] In the example of FIG. 3, the matching result by the matching function 155 is displayed in the first display frame S (S1 and S2). In this case, the first display frame S is an example of matching information. The first display frame S1 indicates that the disease site P1 and the disease site C1 are matched. The first display frame S2 indicates that the disease site P3 and the disease site C2 are matched. In the example of FIG. 3, the first screen 120A indicates that no matching has been performed for the disease site P2 and the disease site C3.

[0056] By displaying the correspondence between disease areas on past medical images and disease areas on current medical images in this manner, the user can easily understand which disease areas on the current medical image correspond to which disease areas on the past medical images.

[0057] In Fig. 3, the correspondence between the disease site on the current medical image and the disease site on the past medical image is shown in the first display frame S, but the method of expressing the correspondence is not limited to this. For example, the correspondence may be expressed by connecting the disease site on the current medical image with the disease site on the past medical image corresponding to the disease site with a straight line or the like. Also, for example, the correspondence may be expressed by simply displaying the corresponding disease sites side by side, vertically or horizontally, without displaying the first display frame S.

[0058] Returning to Fig. 2, the explanation will be continued. The reception function 156 receives a correction input for the association of one of pairs of a disease site on a medical image and a disease site on another medical image different from the medical image. For example, the reception function 156 receives a correction input from the user via the input interface 110 for associating one disease site on the current medical image with another disease site on the past medical image among pairs of disease sites on the past medical image and disease sites on the current medical image associated by the association function 155.

[0059] Here, Fig. 4 is a diagram for explaining an example of a process for accepting correction input for disease site association. Fig. 4 shows an example of a case where correction input is accepted from the user on the first screen 120A shown in Fig. 3.

[0060] For example, the user performs a correction input for the association by dragging any one of the disease sites C1 to C3 on the current medical image with a mouse, to move it below any one of the disease sites P1 to P3 on the past medical image to which it is to be associated. In the example of Fig. 4, the reception function 156 receives a correction input for associating the disease site C2 with the disease site P3 by moving the disease site C2 below the disease site P2.

[0061] The white frame WF is a frame representing the diseased part on the current medical image that is the target of correction input. In the example of Fig. 4, the white frame WF represents that the diseased part C2 is the target of correction input.

[0062] Also, for example, when a disease site on the current medical image to be the target of correction input is selected, a white frame WF is displayed around the selected disease site. At this time, if a first display frame S is displayed around the disease site, the first display frame S is erased. In the example of Fig. 4, the first display frame S2, which was displayed in Fig. 3 and in which the disease site P3 and the disease site C2 are associated, is erased, and instead a white frame WF is displayed around the disease site C2.

[0063] Returning to Fig. 2, the description will be continued. The calculation function 157 calculates the position difference between before and after the correction of the disease site related to the correction input. For example, the calculation function 157 calculates the position difference on the past medical image between the disease sites before and after the correction related to the correction input.

[0064] As an example, the calculation function 157 calculates, as a vector, the position difference between the center of gravity of the disease area on the past medical image that was associated with the disease area on the current medical image related to the correction input before correction, and the center of gravity of the disease area on the past medical image that is associated with the disease area on the current medical image after correction.

[0065] In the examples of Figures 3 and 4, the calculation function 157 calculates as a vector the position difference between the center of gravity of the disease area P3 that was associated with the disease area C2 before the correction input and the center of gravity of the disease area P2 that is associated with the disease area C2 after the correction input.

[0066] The above is just one example of a method for calculating the position difference of the diseased site before and after correction, and the method for calculating the position difference is not limited to the above.

[0067] The correction function 158 corrects the result of the alignment between the medical images. For example, the correction function 158 corrects the result of the alignment between the current medical image and the past medical image performed by the alignment function 153, based on the position difference calculated by the calculation function 157.

[0068] As an example, the correction function 158 corrects the alignment result by the alignment function 153 by moving the past medical image in the direction represented by the vector calculated by the calculation function 157 by the distance represented by the vector while keeping the position of the current medical image fixed.

[0069] The presentation function 159 presents correction candidates for the disease site, of which the target of association is to be changed, among one or more disease sites, based on the result of the corrected alignment. For example, the association function 155 first re-executes the association process between the disease site on the current medical image and the disease site on the past medical image according to the result of the alignment corrected by the correction function 158. In this case, the association function 155 may perform the association process only for disease sites other than the pair of disease sites related to the correction input.

[0070] The presentation function 159 compares the association result before and after the correction of the alignment result, and detects a pair of disease sites on the current medical image and disease sites on the past medical image for which the association result changes. The presentation function 159 presents the pair as a correction candidate for the disease site association result. The correction candidates presented by the presentation function 159 are displayed on the display 120, for example, under the control of the display control function 151.

[0071] Here, Fig. 5 is a diagram for explaining an example of a process of presenting correction candidates of the disease site association result. Fig. 5 shows an example of a case where correction candidates are presented according to a correction input received from a user on the first screen 120A shown in Fig. 4.

[0072] 5, the first display frame S1 indicates that the diseased part P1 and the diseased part C1 are associated with each other. The first display frame S2 indicates that the diseased part P2 and the diseased part C2 are associated with each other. The first display frame S3 indicates that the diseased part P3 and the diseased part C3 are associated with each other.

[0073] In addition, in the example of Figure 5, the second display frame CF (CF1 and CF2) shows the pair of disease sites on the current medical image and disease sites on the past medical image whose matching results have been changed due to correction of the alignment results.

[0074] The second display frame CF1 shows that the disease site P2 and the disease site C2, which were not associated before the correction, are associated with each other as a result of the correction of the alignment result. Similarly, the second display frame CF2 shows that the disease site P3 and the disease site C3, which were not associated before the correction, are associated with each other.

[0075] This allows the user to easily grasp the pair of disease sites on the previous medical image and disease sites on the current medical image, the association of which will be changed by correcting the registration result.

[0076] In the example of FIG. 5, the pair of disease site P1 and disease site C1, whose combination does not change before and after correction, is also displayed on the first screen 120A, but the display form is not limited to this.

[0077] For example, when presenting candidates for correction of association, combinations that do not change may not be displayed on the first screen 120A. Also, for example, when disease sites are displayed horizontally from the left in a predetermined order as in Fig. 5, only the pairs located to the right of the disease site pair related to the correction input may be displayed. In this case, the association function 155 may not need to execute association processing again for disease sites located to the left of the disease site pair related to the correction input.

[0078] Note that the reception function 156 may further receive a correction input for the association from the user on the first screen 120A of Fig. 5. In this case, the correction function 158 may further correct the alignment result, or may correct only the association result. This allows the user to easily correct the association even if there is still an error in the association even after the alignment result has been corrected.

[0079] When the association result is confirmed by the reception function 156 receiving a confirmation input of the association result from the user on the first screen 120A, the processing circuitry 150 may transmit the association result to the medical image storage device 3 or the like via the communication interface 130. In this case, the medical image storage device 3 stores the association result together with the medical image.

[0080] Next, a description will be given of processing executed by the medical image processing apparatus 100 according to this embodiment. Fig. 6 is a flowchart showing an example of processing executed by the medical image processing apparatus 100 according to this embodiment.

[0081] First, the acquisition function 152 acquires a current medical image (step S101). For example, the acquisition function 152 acquires a medical image (current medical image) that captures the current state of the target subject from the medical image diagnostic device 2 or the medical image storage device 3 using the patient ID of the target subject or the like.

[0082] Next, the acquisition function 152 acquires a past medical image (step S102). For example, the acquisition function 152 similarly acquires a medical image (past medical image) of the target patient captured earlier than the current medical image acquired in step S101.

[0083] Next, the registration function 153 performs registration between the current medical image and the past medical image (step S103). For example, the registration function 153 performs registration between the current medical image and the past medical image by using anatomical landmarks or the like.

[0084] Next, the extraction function 154 extracts disease sites from the current medical image and the past medical image (step S104). For example, the extraction function 154 uses a trained model using known machine learning, deep learning, or the like to extract disease sites from the current medical image and the past medical image.

[0085] Next, the association function 155 associates the disease site on the current medical image with the disease site on the past medical image (step S105). For example, the association function 155 uses a method such as a block matching method to associate the disease site on the current medical image with the disease site on the past medical image.

[0086] Next, the reception function 156 receives a correction input for the combination of disease sites (step S106). For example, the reception function 156 receives a correction input for one combination of disease sites from the user via a screen displaying a set of disease sites.

[0087] Next, the calculation function 157 calculates the position difference on the past medical image between the disease site before and after the correction related to the correction input (step S107). For example, the calculation function 157 calculates, as a vector, the position difference between the center of gravity of the disease site on the past medical image that was associated with the disease site on the current medical image related to the correction input before the correction, and the center of gravity of the disease site on the past medical image that is associated with the disease site on the current medical image after the correction.

[0088] Next, the correction function 158 corrects the alignment result (step S108). For example, the correction function 158 corrects the alignment result of step S103 by moving the previous medical image in the direction represented by the vector calculated in step S107 by the distance represented by the vector while fixing the position of the current medical image.

[0089] Next, the presentation function 159 presents correction candidates for the combination of disease sites (step S109), and ends this process. For example, the association function 155 performs association processing again based on the alignment result corrected in step S108. The presentation function 159 presents, as a correction candidate, those in which the combination of disease sites changes before and after correction as a result of the reassociation.

[0090] As described above, the medical image processing apparatus 100 according to this embodiment associates a disease site on a current medical image with a disease site on a past medical image according to the result of the registration between the current medical image and the past medical image. Also, the medical image processing apparatus 100 according to this embodiment accepts a correction input from a user for associating a disease site on a current medical image with another disease site on a past medical image, calculates a position difference between the disease sites related to the correction input on the past medical image according to the content of the correction input, corrects the registration result based on the position difference, and presents correction candidates for a set of disease sites whose combination is changed by the correction of the registration result.

[0091] This automatically associates the disease site on the current medical image with the disease site on the previous medical image, so the user does not need to interpret the image while considering the correspondence between the disease site on the current medical image and the disease site on the previous medical image. In other words, the medical image processing device 100 according to this embodiment can support the user in efficiently interpreting medical images.

[0092] However, since the alignment between the current medical image and the past medical image is not perfect, it is possible that an error occurs in the alignment result. In this case, an error in the alignment result may cause an error in the result of matching the disease site on the current medical image with the disease site on the past medical image. Even in such a case, the medical image processing device 100 according to this embodiment can accept a correction input from the user for one combination of disease sites and correct the alignment result according to the content of the correction input. In other words, the medical image processing device 100 according to this embodiment can efficiently perform alignment between medical images taken at different times.

[0093] Furthermore, the medical image processing apparatus 100 according to the present embodiment can present the user with correction candidates for the correlation result by executing the correlation process again based on the corrected registration result. Therefore, even if there is an error in the correlation between multiple disease sites, the user can grasp multiple correction candidates by simply inputting a correction for one combination of disease sites.

[0094] The above-described embodiment can be modified as appropriate by changing a part of the configuration or function of each device of the medical image processing system 1. Therefore, the following describes a modified example of the above-described embodiment as another embodiment. The following mainly describes the differences from the above-described embodiment, and omits detailed description of the commonalities with the contents already described. The modified examples described below may be implemented individually or in appropriate combination.

[0095] (Modification) In the above embodiment, the disease site on the current medical image and the disease site on the past medical image are displayed side by side as images showing only the disease sites. However, the disease site on the current medical image and the disease site on the past medical image may be displayed together with an image showing the whole image of the target site.

[0096] Here, Fig. 7 is a diagram for explaining an example of a display process of the disease site association result according to the modified example. Fig. 7 shows an example of a second screen 120B that displays the disease site association result together with the whole image of the target site.

[0097] In the example of Fig. 7, the second screen 120B is configured with two horizontal columns of display areas. In the left column of Fig. 7, a subject image PT1 including diseased areas PA1-PA3 on a past medical image is displayed. In addition, in the right column, a subject image PT2 including diseased areas CA1-CA3 on a current medical image is displayed. Here, if the target area is the liver, the subject images PT1 and PT2 are medical images that allow an overall image of the liver to be grasped.

[0098] 7 is an example, and the number of disease sites on the current medical image and the disease sites on the past medical images is not limited to 3. In addition, the number of disease sites on the current medical image and the number of disease sites on the past medical images may be different.

[0099] 7, the result of the association by the association function 155 is represented by a first line SL (SL1 and SL2). The first line SL1 represents that the diseased area PA1 and the diseased area CA1 are associated with each other. The second line SL2 represents that the diseased area PA2 and the diseased area CA3 are associated with each other.

[0100] 8 is a diagram for explaining an example of a process for accepting a correction input for the disease site association result according to the modified example. FIG. 8 shows an example of a case where a correction input is accepted from a user on the second screen 120B shown in FIG.

[0101] For example, the user performs a correction input for the association by dragging any one of the disease sites CA1 to CA3 on the current medical image with a mouse or the like to a predetermined range of any one of the disease sites PA1 to PA3 on the past medical image to which the disease site is to be associated. In the example of Fig. 8, the reception function 156 receives a correction input for associating the disease site PA2 with the disease site C2 by moving the disease site CA2, which is not associated, to a predetermined range of the disease site PA2.

[0102] The white frame WF is a frame representing the diseased area on the current medical image that is the target of correction input, similar to the example of Fig. 4. In the example of Fig. 8, the white frame WF represents that the diseased area CA2 is the target of correction input.

[0103] The second line CS is a line representing the combination of the disease site on the current medical image and the disease site on the past medical image that is the subject of the correction input. In the example of Fig. 8, the second line CS represents that the pair of the disease site PA2 and the disease site CA2 is the subject of the correction input.

[0104] Also, for example, when a disease site on the current medical image to be modified is selected, a white frame WF is displayed around the selected disease site. If a first straight line SL is displayed around the disease site, the first straight line SL is erased.

[0105] In the example of Fig. 8, the first straight line SL2, which was displayed in Fig. 7 and indicates that the diseased part P3 and the diseased part C2 are associated with each other, is erased, and instead, a white frame WF is displayed around the diseased part CA2. Similarly, in the example of Fig. 8, a second straight line CS is displayed around the pair of the diseased part PA2 and the diseased part CA2 in place of the first straight line SL2.

[0106] 9 is a diagram for explaining an example of a process for presenting correction candidates for the disease site association result according to the modified example. Fig. 9 shows an example of a case where correction candidates for the association result are presented according to a correction input received from a user on the second screen 120B shown in Fig. 8.

[0107] In the second screen 120B of Fig. 9, the first straight line SL1 indicates that the diseased area PA1 corresponds to the diseased area CA1. The first straight line SL2 indicates that the diseased area PA2 corresponds to the diseased area CA2. The first straight line SL3 indicates that the diseased area PA3 corresponds to the diseased area CA3.

[0108] In addition, in the example of Figure 9, the second display frame CF (CF3 and CF4) represents the pair of disease sites on the current medical image and disease sites on the past medical image, for which the matching result has changed due to correction of the alignment result.

[0109] The second display frame CF3 shows that the diseased area PA2 and the diseased area CA2, which were not associated before the correction, are associated with each other as a result of the correction of the alignment result. Similarly, the second display frame CF4 shows that the diseased area PA3 and the diseased area CA3, which were not associated before the correction, are associated with each other.

[0110] According to this modification, an image showing the whole image of the target part and the correspondence between the disease part on the current medical image and the disease part on the past medical image are displayed on one screen, which makes it easier for the user to understand the condition of the disease part of the target subject.

[0111] In addition, each component of each device illustrated in the present embodiment is a functional concept, and does not necessarily have to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to that illustrated, and all or a part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0112] The method described in this embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. This program can also be recorded on a non-transitory computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, or a DVD, and executed by being read from the recording medium by a computer.

[0113] According to at least one of the embodiments described above, it is possible to efficiently align medical images captured at different times.

[0114] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0115] 100 Medical image processing device 120 Display 151 Display control function 152 Acquisition Function 153 Alignment function 154 Extraction Function 155 Matching Function 156 Reception Function 157 Calculation Function 158 Correction Function 159 Presentation function

Claims

1. an acquisition unit that acquires a first medical image and a second medical image that is captured at a time different from that of the first medical image and includes a region captured in the first medical image; an extracting unit that extracts a first feature point from the first medical image and extracts a second feature point from the second medical image; a first matching unit that matches first feature points and second feature points that exist in corresponding parts of the first medical image and the second medical image based on a result of the registration between the first medical image and the second medical image; a receiving unit that receives a correction input for associating the first feature point with another second feature point among the pairs of the first feature point and the second feature point associated by the first association unit; a calculation unit that calculates a position difference on the second medical image between second feature points before and after the correction related to the correction input; a correction unit that corrects a result of alignment between the first medical image and the second medical image based on the position difference; A medical image processing device comprising:

2. a second matching unit that matches the first feature points with the second feature points based on the corrected alignment result; a presentation unit that presents, as a correction candidate for the correspondence, a pair of the pairs associated by the second association unit, the pair having a different combination of the first feature point and the second feature point from the pair associated by the first association unit. The medical image processing device according to claim 1 .

3. a display control unit that causes a display unit to display, on the display unit, association information that represents the extracted first feature points and the extracted second feature points, and the sets associated by the first association unit or the second association unit; when the correction candidate is presented, the display control unit causes the display unit to display the association information corresponding to the presented correction candidate and the association information not corresponding to the correction candidate in a distinguishable manner. The medical image processing device according to claim 2 .

4. the display control unit causes the display unit to display the first feature points and the second feature points corresponding to the first feature points side by side, either vertically or horizontally, thereby displaying the association information. The medical image processing device according to claim 3 .

5. the display control unit causes the display unit to display the first feature points, the second feature points, and the association information in an order based on the positions of the first feature points; the second association unit associates the first feature points related to the correction input with the second feature points only for the first feature points related to the order that is lower than the first feature points related to the correction input; The medical image processing device according to claim 4 .

6. the display control unit causes the display unit to display the association information by surrounding the first feature point and the second feature point corresponding to the first feature point with a frame. The medical image processing device according to claim 4 .

7. the display control unit causes the display unit to display the correspondence by connecting the first feature point and the second feature point corresponding to the first feature point with a line. The medical image processing device according to claim 4 .

8. the display control unit, when the correction candidate is presented, causes only the association information corresponding to the presented correction candidate to be displayed on the display unit; The medical image processing apparatus according to claim 3 .

9. the display control unit causes the display unit to display the extracted first feature point and the extracted second feature point, and the association information, for each target site of the subject to be diagnosed. The medical image processing apparatus according to claim 3 .

10. the display control unit displays the first medical image and the second medical image including the target site side by side, and connects the first feature point displayed on the first medical image with the second feature point displayed on the second medical image corresponding to the first feature point with a line, thereby displaying the association information on the display unit. The medical image processing device according to claim 9 .

11. A medical image processing method using a medical image processing device, comprising: An acquisition step of acquiring a first medical image and a second medical image captured at a time different from that of the first medical image and including a region captured in the first medical image; an extraction step of extracting a first feature point from the first medical image and extracting a second feature point from the second medical image; a first matching step of matching first feature points and second feature points present in corresponding parts of both images based on a result of registration between the first medical image and the second medical image; a receiving step of receiving a correction input for associating the first feature point with another second feature point among the pairs of the first feature point and the second feature point associated in the first association step; a calculation step of moving the other second feature point related to the correction input to a position on the second medical image corresponding to the first feature point related to the correction input, and calculating a position difference between a position of the other second feature point before the movement and a position of the other second feature point after the movement; a correction step of correcting the alignment result based on the position difference; a second matching step of matching the first feature points with the second feature points based on the corrected alignment result; a presenting step of presenting, as a correction candidate for the correspondence, a pair of the pairs associated in the second correspondence step, the pair having a different combination of the first feature point and the second feature point from the pair associated in the first correspondence step; A medical image processing method comprising:

12. On the computer, An acquisition step of acquiring a first medical image and a second medical image captured at a time different from that of the first medical image and including a region captured in the first medical image; an extraction step of extracting a first feature point from the first medical image and extracting a second feature point from the second medical image; a first matching step of matching first feature points and second feature points present in corresponding parts of both images based on a result of registration between the first medical image and the second medical image; a receiving step of receiving a correction input for associating the first feature point with another second feature point among the pairs of the first feature point and the second feature point associated in the first association step; a calculation step of moving the other second feature point related to the correction input to a position on the second medical image corresponding to the first feature point related to the correction input, and calculating a position difference between a position of the other second feature point before the movement and a position of the other second feature point after the movement; a correction step of correcting the alignment result based on the position difference; a second matching step of matching the first feature points with the second feature points based on the corrected alignment result; a presenting step of presenting, as a correction candidate for the correspondence, a pair of the pairs associated in the second correspondence step, the pair having a different combination of the first feature point and the second feature point from the pair associated in the first correspondence step; A program that executes the following.

Citation Information

Patent Citations

  • Diagnostic imaging support apparatus

    JP2005124895A

  • Medical image display device and image display method

    JP2009160045A

  • Image diagnosis support device and image diagnosis support method

    JP2013123528A

  • Image processing system, and control method and program for image processing system

    JP2016106781A

  • Image processing device, image processing method, and program

    JP2016147026A