Image processing device, image processing method and program

The image processing device automates the determination and display of image processing conditions to enhance the efficiency of medical image observation by reducing the need for manual adjustments, focusing on image characteristics.

JP2024164447APending Publication Date: 2024-11-27CANON KK +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023079923
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-15
Publication Date
2024-11-27

AI Technical Summary

Technical Problem

Existing image processing methods require significant user effort for adjusting image conditions to observe image characteristics, as they may not be suitable for observing the properties of medical images, necessitating manual adjustments by doctors.

Method used

An image processing device that automatically determines and displays image processing conditions suitable for observing selected image findings, including a finding acquisition unit, selection unit, determining unit, and image acquisition unit, to reduce the need for manual adjustments.

Benefits of technology

This approach reduces the effort required for image adjustment by automatically determining and displaying images that meet optimal processing conditions, enhancing the efficiency of medical image observation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024164447000001_ABST
    Figure 2024164447000001_ABST
Patent Text Reader

Abstract

To reduce the time and effort required by a user for image adjustment when observing an image remark.SOLUTION: An image processing device includes: remark acquisition means which acquires a plurality of image remarks as a first image remark group for a medical image of a diagnostic target; remark selection means which selects at least one image remark from the plurality of image remarks; determination means which determines an image processing condition suitable for observation of the image remark for the selected image remark; and image acquisition means which acquires a display image that satisfies the determined image processing condition.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The disclosure of this specification relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] In the medical field, "image diagnosis" is performed based on medical images obtained by imaging devices such as X-ray CT (Computer Tomography) and MRI (Magnetic Resonance Imaging). The task of observing medical images and deriving a diagnosis is called "image interpretation." In image diagnosis, an image interpretation doctor, who is a doctor specialized in image diagnosis, interprets images, for example, at the request of the attending physician. The image interpretation doctor makes a comprehensive judgment based on the image findings that show the characteristics of the image and various measured values ​​that can be part of the basis for the diagnosis, and identifies the lesion depicted in the medical image and the symptoms of the patient, who is the subject. The image interpretation doctor then writes an interpretation report on how the diagnosis was reached using the image findings and measured values, and responds to the attending physician who made the request.

[0003] Nowadays, various medical information is used for diagnosis, and there are high expectations for a system that uses a computer to analyze medical information, including medical images, and uses the obtained results to assist users, such as doctors, in making diagnoses. Patent Document 1 discloses a method for a system that infers image findings and diagnosis names from pulmonary nodule images, in which position information of an area showing the relevant image findings in a pulmonary nodule image is presented to make it easier for a user to grasp the position of the image findings that are the basis for the inferred diagnosis name. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-214323 A Summary of the Invention [Problem to be solved by the invention]

[0005] In order for a user such as a doctor to understand an image finding, it is necessary to not only know the position but also to observe the characteristics of the image. However, in the method disclosed in Patent Document 1, the display conditions of the image may not be suitable for observing the characteristics of the image. In such a case, the user must observe the image finding while adjusting the image.

[0006] The present disclosure has been made in consideration of such circumstances, and has as one object to reduce the effort required by a user to adjust an image when observing image findings. [Means for solving the problem]

[0007] In order to solve the above problem, an image processing device according to one aspect of the present disclosure includes: a finding acquisition means for acquiring a plurality of image findings as a first image finding group from a medical image of a diagnosis target; A finding selection means for selecting at least one image finding from the plurality of image findings; A determination means for determining image processing conditions suitable for observing the selected image findings; an image acquisition means for acquiring an image for display that satisfies the determined image processing conditions; Equipped with. Effect of the Invention

[0008] According to one aspect of the present disclosure, it is possible to reduce the effort required by a user to adjust an image when observing image findings. [Brief description of the drawings]

[0009] [Figure 1] 2 is an example of a hardware configuration of an image processing device according to a first embodiment of the present disclosure. [Diagram 2] 2 is an example of a functional configuration of an image processing device according to the first embodiment. [Diagram 3] 5 is a flowchart showing an example of processing by the image processing device according to the first embodiment. [Figure 4]4 is an example of input information generated by the image processing device according to the first embodiment. [Diagram 5] 4 is an example of a correspondence table between findings and priorities referred to in the image processing apparatus according to the first embodiment. [Figure 6(a)] 4 is an example of a correspondence table between findings and display conditions referred to in the image processing device according to the first embodiment. [Figure 6(b)] 13 is an example of a correspondence table between findings and display conditions referred to in an image processing device according to a modified example of the first embodiment. [Figure 7] 4 is an example of a display method by the image processing device according to the first embodiment. [Figure 8] 4 is an example of a display method by the image processing device according to the first embodiment. [Figure 9] 13 is an example of a functional configuration of an image processing device according to a second embodiment. [Figure 10] 10 is a flowchart showing an example of processing by an image processing device according to a second embodiment. [Figure 11] 13 is a diagram illustrating an example of a display method by an image processing device according to a third embodiment. [Figure 12] 13 is an example of a functional configuration of an image processing device according to a third embodiment. [Figure 13] 13 is a diagram illustrating an example of a method for saving a report according to the third embodiment. [Figure 14] 13 is a flowchart showing an example of processing by an image processing device according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of an image processing device, an image processing method, and a program according to the present disclosure will be described with reference to the drawings. The following embodiments do not limit the present disclosure according to the claims. Note that the configurations shown in the following embodiments are merely examples, and the present disclosure is not limited to the illustrated configurations. In addition, although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the present disclosure, and multiple features may be combined in any manner. In addition, in the drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] [First embodiment] An image processing device according to the first embodiment will be described below. In the first embodiment, an image optimal for observing image findings is automatically determined and displayed during the process of an image interpretation doctor performing image interpretation. The image processing device according to the first embodiment will be described below in the order of an example of a hardware configuration, an example of a functional configuration, and an example of a processing flow.

[0012] (Hardware configuration) 1 is a diagram showing an example of the hardware configuration of an image processing device 100 according to this embodiment. The image processing device according to this embodiment includes, for example, a CPU (Central Processing Unit) 11, a main memory 12, a magnetic disk 13, a display memory 14, a monitor 15, a mouse 16, and a keyboard 17. These components are connected to each other via a common bus 18 so as to be able to communicate with each other.

[0013] The CPU 11 mainly controls the operation of each component. The main memory 12 stores, for example, a control program executed by the CPU 11, and provides a working area when the CPU 11 executes the program. The magnetic disk 13 stores, for example, an operating system (OS), device drivers for peripheral devices, and programs for implementing various application software including programs for performing processes described below. The CPU 11 executes the programs stored in the main memory 12 or the magnetic disk 13 to implement the functions (software) of the image processing device in this embodiment.

[0014] The display memory 14 temporarily stores display data to be displayed on, for example, the monitor 15. The monitor 15 is, for example, a CRT (Cathode Ray Tube) monitor or a liquid crystal monitor, and displays images, text, and the like based on the data from the display memory 14. The mouse 16 and keyboard 17 are used by the user to input pointing and characters, etc., respectively.

[0015] Note that the configuration of the image processing device 100 is not limited to the above. For example, the image processing device 100 may have multiple processors. It may also have a GPU (Graphic Processing Unit) or an FPGA (Field-Programmable Gate Array) in which some of the processing is programmed.

[0016] (Functional configuration) 2 is a diagram showing an example of the functional configuration of an image processing device that performs diagnostic support in this embodiment. The image processing device 100 is communicably connected to a case information terminal 200. The image processing device 100 has a finding acquisition unit 104, a finding selection unit 110, a decision unit 112, an image acquisition unit 114, and a display control unit 116. The functional components of the image processing device 100 are connected via an internal bus or the like.

[0017] The case information terminal 200 acquires information on a case to be diagnosed from a server (not shown). The case information is medical information such as medical images and clinical information recorded in an electronic medical record. The case information terminal 200 may be connected to an external storage device (not shown) such as a hard disk drive (HDD), a solid state drive (SSD), a CD drive, or a DVD drive, and may acquire medical images from the external storage device.

[0018] Furthermore, the case information terminal 200 provides a GUI (Graphical User Interface) that allows the user to select one of the acquired medical images via the display control unit 116. The selected medical image is displayed large on the monitor 15. The case information terminal 200 transmits the medical image selected by the user via the GUI to the image processing device 100 via a network or the like.

[0019] The finding acquisition unit 104 acquires image findings related to a target image based on a medical image to be interpreted (hereinafter, a target image) transmitted from the case information terminal 200 to the image processing device 100. First, the finding acquisition unit 104 performs image processing on the target image to acquire image features of the target image. Next, the finding acquisition unit 104 infers image findings related to the target image based on the acquired image features, and outputs the inferred image findings. The inference of the image findings is performed by using an inference device that infers image findings based on image features, which is constructed in advance by deep learning or the like. Note that the configuration of the finding acquisition unit 104 is not limited to this, and may be configured to acquire image findings related to the target image from an external server (not shown) that provides a similar function. Note that the finding acquisition unit 104 may acquire image findings input / selected by a user, or may acquire image findings transmitted from another image processing device, and is not limited to acquisition by inference by the image processing device 100.

[0020] The finding selection unit 110 selects an image finding to be displayed (hereinafter, selected image finding) from among the image findings output by the finding acquisition unit 104, and outputs information on the selected image finding. The determination unit 112 determines display conditions for the target image based on the selected image findings output by the finding selection unit 110, and outputs information on the display conditions. The image acquisition unit 114 acquires or generates an image for display (hereinafter, display image) related to the target image based on the display conditions output by the determination unit 112, and displays it on the monitor 15 via the display control unit 116. The display control unit 116 displays the display image generated by the image acquisition unit 114 and a GUI for user operation on the monitor 15.

[0021] (Processing flow) Next, a processing flow will be described with reference to a flowchart of Fig. 3 which shows an example of processing performed by the above-mentioned image processing device 100. In the first embodiment, the CPU 11 executes a program which realizes the functions of each unit stored in the main memory 12, thereby realizing the processing (image processing method) shown in Fig. 3.

[0022] In the following description, the image feature quantity for the target image is I m (m=1~M), and the items of image findings (finding items) are F n (n=1 to N), where M is the number of image feature items and N is the number of image findings. m and F n Each item in has a value. m Each item in takes a continuous value, and F n F can take continuous or discrete (categorical) values ​​depending on the item. n If takes discrete values, each discrete value is expressed as f nk In other words, the information that identifies which of the k elements (categories) the finding belongs to is the value of the finding item. Note that k is the number of elements in each F n The image feature I m , Image findings F n If takes continuous values, set the value to i m ,f n It shall be expressed as:

[0023] In the first embodiment, as an example, a case will be described in which an image of a nodule-like shadow in a chest CT image is used as a target image. In this case, items (finding items) and values ​​(finding values) as shown in FIG. 4 are handled as image findings. For example, a finding item F 1 If the "shape" of is f, the element is 11 "spherical",f 12 "lobulated",f 13 "Polygon",f 14 "irregular shape" is one of the four values. 2 If the "intranodule calcification ratio" is f 21 "67~100%", f 22 "34~66%", f 23 It is one of three values ​​of "0 to 33%." In the first embodiment, it is assumed that all the elements of the image findings are discrete values.

[0024] When the actual processing flow starts, in step S301, the findings acquisition unit 104 first performs image processing based on a target image and acquires image features of the target image. The image features acquired here may be general image features such as the average value or variance of density (brightness) in a processing target region of the image, or may be image features based on a filter output.

[0025] Next, the finding acquisition unit 104 converts the image feature amount into a group of image findings with likelihood. For example, 1 ,i 2 ,…,i M}, the image findings "shape" is inferred. More specifically, the findings item F 1 The value of "shape" is f 11 , f 12 , f 13 , f 14 For each element f 1k We infer the likelihood of f 1k The likelihood of L(f 1k ), then L(f 11 )+L(f 12 )+L(f 13 )+L(f 14)=1.0. Various methods capable of outputting values ​​with likelihood can be used to infer image findings. In the first embodiment, a multi-valued neural network is used. Note that the method of inferring image findings described here is just one example, and any known method may be used to infer image findings from images.

[0026] In step S302, the finding selection unit 110 selects at least one of the multiple image findings (items and their values) inferred in step S301, and acquires it as a selected image finding. Details of acquiring the selected image finding will be described below.

[0027] When selecting an image finding, the finding selection unit 110 first selects the most likely element (or the identification information representing the element) for each image finding item as the value of the image finding. For example, the element (f 11 , f 12 , f 13 , f 14 ) are the likelihoods of L(f 11 )=0.1,L(f 11 )=0.0,L(f 12 )=0.2,L(f 13 )=0.7,L(f 14 ) = 0.0, the image finding item "shape" F 1 The value of f 13 (Polygon).

[0028] Next, the finding selection unit 110 derives the priority of each image finding (item and its value) inferred in step S301 based on a database (illustrated in FIG. 5) in which priorities according to the image findings (item and its value) are stored. FIG. 5 is an example of a database showing the relationship between the image findings and the associated priorities, which are medical information stored in, for example, the main memory 12. In the first embodiment, the priority is set according to the difficulty of confirming the image findings. For example, the intranodule calcification ratio is set to a high priority because a small ratio is more difficult to confirm than a large ratio. Note that the priority may be set based on other criteria, for example, according to the degree of influence on the final diagnosis. That is, the priority may be set so that the higher the degree of influence on the diagnosis is for each image finding (combination of item and value), the higher the priority is. Also, a configuration may be used in which a priority is set for each image finding item.

[0029] Next, the finding selection unit 110 selects one image finding from the image findings inferred in step S301 according to the priority, and acquires it as a selected image finding. In the first embodiment, the image finding with the highest priority is selected. Note that the method of selecting an image finding is not limited to this, and for example, the image finding with the second highest priority may be selected. Also, for example, when there are multiple image findings with a small difference from the highest priority (for example, the difference in priority is less than 5), a GUI that displays multiple image findings may be prepared to allow the user to select an image finding. Alternatively, the finding selection unit 110 may select a predetermined number of image findings according to the priority.

[0030] After the image findings are selected in step S302, the determination unit 112 determines the display conditions in step S303. Specifically, the determination unit 112 obtains the display conditions corresponding to the selected image findings (items and their values) obtained in step S302 based on a database (illustrated in FIG. 6(a)) in which optimal display conditions are previously stored in association with the image findings (items and their values). The determination unit 112 then determines the obtained display conditions as the image processing conditions for obtaining the display image. FIG. 6(a) is an example of a database that is medical information stored in, for example, a main memory, and indicates the relationship between the image findings and the associated display conditions that are previously determined. In the first embodiment, the reconstruction function, slice thickness, WL (Window Level), and WW (Window Width) used in reconstructing the signal obtained by the X-ray CT apparatus are used as an example of the display conditions.

[0031] After the display conditions are determined, in step S304, the image acquisition unit 114 first acquires an image corresponding to the display conditions acquired in step S303 as an image for display. In the first embodiment, the image acquisition unit 114 applies image processing to the target image based on the display conditions acquired in step S303, and acquires the processed image as an image for display. For example, based on the values ​​of WL and WW, which are display conditions related to the contrast of the image, the target image is adjusted so that WL-WW / 2 is the minimum value of the image value and WL+WW / 2 is the maximum value of the image value, and the adjusted image is acquired as an image for display.

[0032] The image acquiring unit 114 may be configured to acquire an image corresponding to the display conditions acquired in step S303 from a server (not shown) or the like. For example, the image acquiring unit 114 may acquire an image matching the display conditions of the reconstruction function and slice thickness from a server (not shown). At this time, if the corresponding image does not exist in the server, the image acquiring unit 114 may acquire signal data before reconstruction from an imaging device, reconstruct the image, and then acquire the image.

[0033] The image acquisition unit 114 may also acquire an image that matches some of the display conditions acquired in step S303 from a server (not shown) or the like, and acquire the image as a display image after applying additional processing. For example, when the display conditions include a reconstruction function, a slice thickness, and WL, WL, an image that matches only the reconstruction function and the slice thickness may be acquired from a server (not shown), and an image that has been adjusted based on the values ​​of WL and WW may be acquired as a display image. When the display conditions acquired in step S303 and the display conditions of the target image are the same, the image acquisition unit 114 may acquire the target image as a display image.

[0034] In the display image step S305, the display control unit 116 displays the display image acquired in step S304 on the monitor 15 as shown in FIG. 7(b). At this time, as shown in FIG. 7(c), the target image before the application of image processing based on the display conditions and the display image may be displayed side by side for comparison. Note that FIG. 7 shows an example of an image after image processing displayed on the monitor 15 in the first embodiment and a modified example of the first embodiment described later. FIG. 7(a) shows an example of a display of the target image before the above-mentioned image processing is performed, FIG. 7(b) shows an example of a display of an image after image processing in the first embodiment, and FIG. 7(c) shows an example of a display of the image before processing and the image after processing side by side.

[0035] As described above, the image processing device according to the present disclosure includes the finding acquisition unit 104, the finding selection unit 110, the determination unit 112, and the image acquisition unit 114. The finding acquisition unit 104 functions as a finding acquisition unit in the present disclosure, and executes a process of acquiring a plurality of image findings as a first image finding group for a medical image of a diagnosis target, as described in step S301, for example. In this embodiment, the finding acquisition unit 104 acquires image findings by inference for a medical image. The finding selection unit 110 functions as a finding selection unit in the present disclosure, and executes a process of selecting at least one image finding from the plurality of image findings inferred in step S301, as described in step S302, for example. The determination unit 112 functions as a determination unit in the present disclosure, and executes a process of determining an image processing condition suitable for observing an image finding for an image finding selected in step S302, as described in step S303, for example. The image acquisition unit 114 functions as the image acquisition means in this disclosure, and executes the process of acquiring a display image that satisfies the image processing condition determined in step S303, as described in step S304, for example.

[0036] In the above-mentioned image processing device, the determination unit 112 can determine the image processing conditions based on the selected image finding items and image finding values, for example, by using a database exemplified in FIG. 6. The image acquisition unit 114 can acquire, as a display image, a medical image of a diagnosis target to which image processing based on the image processing conditions determined by the determination unit 112 has been applied. As described in step S304, the display image may be acquired from a server that matches the conditions, or may be acquired by the image acquisition unit 114 functioning as an image processing unit and applying the determined image processing conditions to the medical image of the diagnosis target. As the image processing to be applied, at least one of a brightness level change process, a gradation conversion process, a resolution conversion process, and a cutout process can be executed. Furthermore, the image acquisition unit 114 may acquire a display image based on the determined image processing conditions from a plurality of images to which different image processing has been applied.

[0037] The image processing device according to the present disclosure may further include a display control unit 116 that causes a display unit such as the monitor 15 to display a medical image of the diagnostic target. The display control unit 116 functions as a display control unit in the present disclosure. When the image processing conditions applied to the display of the medical image of the diagnostic target are different from the image processing conditions determined by the determination unit 112, the display control unit 116 can cause the monitor 15 to display the display image in a manner that allows comparison with the medical image of the diagnostic target, as illustrated in FIG. 7(c), for example.

[0038] By executing the above-described process, it becomes possible to automatically determine image processing conditions for observing findings and automatically display images that satisfy the image processing conditions, thereby reducing the user's effort in adjusting images.

[0039] The target image exemplified in the first embodiment described above is an example, and the target image to be diagnosed by the present disclosure is not limited to the example shown here. In addition to the chest, other possible diagnosis target areas to which the present disclosure can be applied include, for example, the abdomen, head, or breast. In addition, the lesion is not limited to the nodule-like shadow exemplified above, and all "abnormal shadows" including, for example, ground glass shadows, granular shadows, and infiltrative shadows can be targets. Furthermore, as for the modality for obtaining the image, an embodiment is exemplified here for an X-ray CT device, but an ultrasound diagnostic device, MRI, etc. can also be used, and for example, tumor shadows in breast ultrasound and tumor shadows in abdominal CT images are exemplified as images obtained from other modalities.

[0040] (Modification 1 of the first embodiment) In step S302, the finding selection unit 110 may acquire selected image findings based on the likelihoods of the image findings. First, the finding selection unit 110 selects the highest likelihood of the elements of each image finding as the likelihood of the image finding. For example, the element (f 11 , f 12 , f 13 , f 14 ) are the likelihoods of L(f 11 )=0.1,L(f 12)=0.2,L(f 13 )=0.7,L(f 14 ) = 0.0, the observation item "shape" F 1 The likelihood of is 0.7. Similarly, the likelihood of all image findings is calculated, and the image finding selected based on the likelihood is acquired as the selected image finding. For example, by selecting the image finding with the lowest likelihood, it is possible to acquire the image finding that requires confirmation by a doctor as the selected image finding.

[0041] The method of selecting the image finding is not limited to this, and for example, the image finding with the highest likelihood may be selected. Also, for example, the likelihood and the priority may be used in combination, and when there are multiple image findings with a small difference from the highest priority (for example, the difference in priority is less than 5), the image finding with the lowest likelihood may be selected from among them.

[0042] That is, as described in this modified example, the finding selection unit 110 may select an image finding based on the likelihood of the image finding inferred by the finding acquisition means (finding acquisition unit 104). As described in the description of step S301 in the first embodiment, the likelihood may be obtained by the finding acquisition unit 104 converting the image feature amount of the medical image of the diagnosis target into a group of image findings with likelihoods (first group of image findings).

[0043] In addition, the likelihood of the image finding may be normalized by taking the case where the likelihood of each element is the same as the minimum value. For example, the element (f 11 , f 12 , f 13 , f 14 ) are the likelihoods of L(f 11 )=0.25,L(f 12 )=0.25,L(f 13 )=0.25,L(f 14 ) = 0.25, all elements have the same likelihood, so it is not possible to specify a single value. The likelihood at this time is taken as the minimum value, and the aforementioned observation item "shape" F 1 If we normalize the likelihood of 0.7 so that its minimum value is 0, the observation item "shape" F 1The likelihood of is (0.7-0.25) / (1.0-0.25). Normalizing the likelihood in this way makes it possible to compare the likelihoods of imaging findings with different numbers of elements.

[0044] (Modification 2 of the first embodiment) In step S302, the finding selection unit 110 may acquire two or more selected image findings. In this case, for example, multiple image findings can be selected in descending order of priority according to the image findings and their values. In this case, multiple display images are acquired through the subsequent steps S303 and S304. FIG. 8 shows a display example in which multiple display images are acquired in this way. In the display screen shown in FIG. 8, multiple images are displayed as thumbnails, and an image selected using a cursor indicated by an arrow is enlarged and displayed. That is, when multiple image findings are selected and image processing conditions are determined for each of the multiple image findings, the display control unit 116 can switchably display, on the monitor 15, display images that satisfy each of the multiple image processing conditions together with the medical image of the diagnosis target. By providing a GUI as shown in FIG. 8 and allowing the user to switch the image to be displayed by designation, multiple image findings that require confirmation can be easily switched.

[0045] (Modification 3 of the first embodiment) In step S303, the display conditions acquired by the determination unit 112 may be applied to an abnormal shadow portion in the image, rather than to the entire image. Here, the abnormal shadow refers to a portion in the image that is suspected to be a lesion, such as a nodular shadow in the lung. The area of ​​the abnormal shadow in the target image may be acquired by a user's operation, or may be automatically acquired using a model previously trained by a multi-valued neural network or the like. Here, it is assumed that the case information terminal 200 provides a GUI for surrounding the abnormal shadow in the image with a rectangle, and the abnormal shadow area is acquired by a user's operation.

[0046] At this time, in step S301, the findings acquisition unit 104 infers image findings using only the abnormal shadow portion in the target image. In addition, in step S305, the display control unit 116 may replace only the abnormal shadow region with one after image processing as shown in FIG. 7(d). Also, as shown in FIG. 7(e), only the abnormal shadow region may be displayed in a separate window. In FIG. 7(e), multiple partial images of the abnormal shadow region obtained by performing different image processing on the abnormal shadow region are displayed in a separate window. Also, as shown in FIG. 7(f), an enlarged image including the abnormal shadow region may be displayed. This allows the user to concentrate on observing the abnormal shadow portion.

[0047] The image processing device according to the present disclosure may further include an identifying means (findings acquiring unit 104) for identifying an abnormal shadow. In this modification, the identifying means may receive, for example, an operation by a user to identify an abnormal shadow as an input, or may automatically identify the abnormal shadow using a model previously trained by a multi-valued neural network or the like. When an abnormal shadow can be identified, the determining unit 112 can determine image processing conditions for image findings related to the abnormal shadow. Then, the image acquiring unit 114 can acquire an image that satisfies the determined image processing conditions and includes the abnormal shadow as an image for display.

[0048] Furthermore, when displaying the medical image of the diagnostic target, the display control unit 116 can generate a partial medical image including a region corresponding to the image finding from the display image and display it on the monitor 15. That is, an abnormal shadow is specified as a region corresponding to the image finding, and, for example, as illustrated in FIG. 7(d), a display image in which only the portion specified as the abnormal shadow is obtained as a partial medical image, or a partial medical image obtained from the display image, can be displayed on the monitor 15 together with the medical image of the diagnostic target. More specifically, in the example shown in FIG. 7(d), the image processing conditions applied to the display of the medical image of the diagnostic target are different from the image processing conditions of the partial medical image determined by the determination unit 112. In such a case, the display control unit 116 can display on the monitor 15 a superimposed image in which the partial medical image is superimposed on the medical image of the diagnostic target. Furthermore, as shown in FIG. 7(e), the partial medical image may be displayed on the monitor 15 so as to be contrasted with the medical image of the diagnostic target.

[0049] (Fourth Modification of the First Embodiment) In the above-described embodiment or modification, WW, WL, and the reconstruction function are described as examples of display conditions to be targeted when performing image processing. However, the display conditions to be adjusted when performing image processing are not limited to these, and other conditions may be used.

[0050] For example, as shown in Fig. 6(b), in addition to the above-mentioned display conditions, a condition regarding the image cut-out range may be added. The cut-out range is specified as a ratio assuming that the rectangular area circumscribing the abnormal shadow is 100%. The cut-out range can be changed depending on the image findings. For example, the cut-out range is set wide for image findings that relate to a wide area around the abnormal shadow, such as satellite nodules, and narrow for findings related to the characteristics inside the abnormal shadow, such as calcification or fat.

[0051] Also, for example, a filter process to be applied to the target image may be added as a display condition. The filter process to be applied differs depending on the image finding (item and its value) as with other display conditions. For example, a known filter process to increase sharpness may be applied to an image finding such as a linear pleural indentation that requires confirmation of the detailed shape of an abnormal shadow. Also, for example, a known filter process to binarize an image finding such as a shape or an outer shape of an abnormal shadow such as a pleural indentation may be applied. Note that the binarization process is not applied to an image finding such as an intranodular calcification ratio or intranodular fat ratio that requires confirmation of the internal density of an abnormal shadow. This makes it possible to display an image to which the most suitable image processing has been applied depending on the finding.

[0052] [Second embodiment] In the first embodiment, a case has been described in which an image optimal for observing image findings is automatically determined and displayed during the image interpretation process by a doctor. In contrast, in the image processing device according to the second embodiment, a case will be described in which an image optimal for observing image findings that have a high impact on a diagnosis is automatically displayed during the image interpretation process by a doctor to arrive at a diagnosis. In the following, an example of the functional configuration and an example of the processing flow will be described for the image processing device according to the second embodiment. Note that the hardware configuration is the same as in the first embodiment, and therefore a description thereof will be omitted here.

[0053] (Functional configuration) 9 is a diagram showing an example of the functional configuration of an image processing device according to the second embodiment. In addition to the functional configuration of the image processing device 100 according to the first embodiment, the image processing device 900 according to this embodiment further includes an input information generating unit 102, a diagnostic inference unit 106, and an influence degree acquiring unit 108. Note that functions other than those described below are the same as those described in the description of the first embodiment, and therefore description thereof will be omitted here.

[0054] The input information generation unit 102 generates input information to be input to the diagnostic inference unit 106, based on the target image transmitted from the case information terminal 200 to the image processing device 900. In the second embodiment, the input information generation unit 102 outputs the target image to the findings acquisition unit 104, and as a result, acquires the image findings output by the findings acquisition unit 104. Then, the input information generation unit 102 outputs the acquired image findings to the diagnostic inference unit 106 as input information.

[0055] The finding acquisition unit 104 in the second embodiment infers image findings based on the target image output from the input information generation unit 102, and outputs the inference result. The inference of image findings is performed in the same manner as in the first embodiment. The diagnosis inference unit 106 infers a diagnosis based on the input information (image findings in the second embodiment) output from the input information generation unit 102, and outputs the inference result.

[0056] The influence degree acquisition unit 108 acquires the influence degree of each element included in the input information on the inference result using the input information output from the input information generation unit 102 and the inference result output from the diagnostic inference unit 106, and outputs information on the influence degree. The finding selection unit 110 in the second embodiment acquires selected image findings based on the input information output from the input information generation unit 102, the inference result output from the diagnostic inference unit 106, and the influence degree output from the influence degree acquisition unit 108, and outputs information on the selected image findings.

[0057] (Processing flow) Next, a processing flow will be described with reference to a flowchart of FIG. 10, which shows an example of processing performed by the above-mentioned image processing device 900. Note that the image feature quantity I m , Image findings F n is assumed to be the same as that described in the first embodiment.

[0058] In addition, in the following explanation, m , F n The set whose elements are the values ​​of is denoted as E, and the input information is E fIn the following description, the diagnosis is represented by D. In the second embodiment, the diagnosis inference unit 106 infers the diagnosis of an abnormal shadow in the lung as one of three values: primary lung cancer, metastasis of cancer to the lung, or others. Primary lung cancer, metastasis of cancer to the lung, and others are respectively represented by d. 1 , d 2 , d 3 and the input information E f Given a diagnosis d u The probability of inference for (u=1,2,3) is P(d u |E f ) will be written as

[0059] When the actual processing flow starts, in step S1001, the input information generating unit 102 acquires a target image transmitted from the case information terminal 200 to the image processing device 100.

[0060] In the next step S1002, the same process as that in step S301 in Fig. 3 in the first embodiment is performed. That is, the finding acquisition unit 104 acquires image findings with likelihoods from the target image output in step S1001.

[0061] Next, in step S1003, the input information generating unit 102 generates the set of image findings acquired in step S1002 as input information. At this time, the image findings are treated as input information while maintaining their likelihood.

[0062] In step S1004, the diagnostic inference unit 106 performs inference regarding the abnormal shadow in the lung, which is the diagnostic target, based on the input information generated in step S1003. That is, the diagnostic inference unit 106 performs inference using image finding information acquired based on image feature amounts as input information. In the second embodiment, the diagnostic inference unit 106 performs inference using a Bayesian network, but is not limited thereto, and may perform inference using a support vector machine, a neural network, or the like.

[0063] In the second embodiment, since the values ​​of the image findings are expressed as likelihoods, inference is performed for all combinations of image findings, and the inference results are integrated using the likelihoods. a {f a1 ,f a2}, F b {f b1 ,f b2 First, the diagnostic inference unit 106 generates provisional input information (E z In this example, the diagnostic reasoning unit 106 generates E 1 ={f a1 ,f b1}, E 2 ={f a1 ,f b2}, E 3 ={f a2 ,f b1}, E 4 ={f a2 ,f b2 Then, the diagnostic inference unit 106 generates four pieces of provisional input information, P(d s |E z ) to get the

[0064] Furthermore, the diagnostic inference unit 106 calculates each P(d s |E z ) by the likelihood of the image findings, and the result is added to obtain the final inference result. In the above example, the diagnostic inference unit 106 multiplies L(f a1 )×L(f b1 )×P(d s |E 1 )+…+L(f a2 )×L(f b2 )×P(d s |E 4 ) is the final inference result P(d s |E f In the above example, the inference result can be expressed as Equation 1.

number

[0065] That is, the diagnostic inference unit 106 generates a plurality of provisional input information consisting of at least a portion of the information on findings included in the input information, and infers a diagnosis based on the results inferred based on the plurality of provisional input information and the likelihood, i.e., statistical information. Note that, in order to reduce the amount of data acquired, the diagnostic inference unit 106 may not consider values ​​of image findings having a likelihood equal to or less than a threshold value.

[0066] Next, in step S1005, the influence degree acquisition unit 108 acquires the influence degree of each element of the input information on the inference result by using the input information generated in step S1002 and the inference result executed in step S1004. That is, the influence degree acquisition unit 108 acquires the influence degree, which is the degree of influence that each element of the input information has on the inference of the diagnosis name, for each piece of information used as an input for inference by the diagnostic inference unit 106.

[0067] In the second embodiment, the diagnosis d with the highest inference probability among the diagnoses is f Specifically, we obtain the influence of an element e v (e v ∈E f ) impact of input information E f d when inferred using f From the inferred probability of E f From e v d when only f The influence of the element is I(e v ) and the impact is defined as in Equation 2.

number

[0068] When the influence degree is obtained, in step S1006, the finding selection unit 110 obtains, as a selected image finding, the image finding with the highest influence degree obtained in step S1005 from among the image findings included in the input information obtained in step S1002. Note that, in consideration of the negative influence degree on the inference result, the finding with the highest absolute value of the influence degree may be configured to be obtained as the selected image finding.

[0069] In step S1007, the same process as step S303 in Fig. 3 is performed. That is, the determination unit 112 acquires a display condition corresponding to the finding acquired in step S303.

[0070] In step S1008, the same process as in step S304 in Fig. 3 is performed. That is, the image acquisition unit 114 acquires, as an image for display, an image corresponding to the display condition acquired in step S1007.

[0071] After acquiring the display image, in step S1009, the display control unit 116 causes the monitor 15 to display the display image acquired in step S1008.

[0072] As described above, the image processing device according to the present disclosure may further include an influence degree acquisition unit 108 in addition to the configuration described in the first embodiment. The influence degree acquisition unit 108 functions as an influence degree acquisition means in the present disclosure. In the case of such a configuration, the influence degree acquisition unit 108 acquires the influence degree of the image finding on the diagnosis inferred by the finding acquisition unit 104 or the input information generation unit 102. In addition, at that time, the finding selection unit 110 can select an image finding based on the acquired influence degree.

[0073] The image processing device according to the present disclosure may further include a diagnostic inference unit 106. The diagnostic inference unit 106 functions as a diagnostic inference means in the present disclosure. In such a case, the diagnostic inference unit 106 can infer a diagnosis using image findings as input. The above-mentioned influence degree acquisition unit 108 can acquire the influence degree of the image findings on the diagnosis of the inferred diagnosis as the influence degree on the diagnosis.

[0074] By executing the above-described process, it becomes possible to automatically determine image processing conditions for observing image findings that are the basis for diagnosis, and to automatically display images that satisfy the image processing conditions, thereby reducing the effort required for the user to adjust images.

[0075] (Modification 1 of the second embodiment) In step S1006 in the second embodiment described above, the finding selection unit 110 selects the image finding with the highest influence. However, the subject selected by the finding selection unit 110 is not limited to the image finding with the highest priority. For example, a plurality of image findings may be selected in descending order of influence, and these may be set as the selected image findings.

[0076] In this modified example, a plurality of display images are acquired through the subsequent steps S1007 and S1008. For this purpose, a GUI such as that shown in Fig. 8 may be provided so that the user can switch between images to be displayed. Note that the method of displaying a plurality of images is not limited to this, and for example, the images may be displayed side by side as shown in Fig. 7(c), or only the regions of abnormal shadows may be displayed side by side as shown in Fig. 7(e). This makes it possible to confirm a plurality of image findings that require confirmation.

[0077] (Modification 2 of the second embodiment) In the above-described second embodiment and its modified example 1, in step S1006, the finding selection unit 110 automatically acquires the selected image finding. However, it is also possible to adopt a mode in which the finding selection unit 110 does not select a finding. For example, the image findings included in the input information acquired in step S1003 and the information on the degree of influence of the image findings on the diagnosis acquired in step S1005 may be presented to the user, and the user may manually select an image finding. In that case, an example of a selection screen displayed on the monitor 15 is shown in FIG. 11. As exemplified in FIG. 11, a list of image findings and their values ​​is displayed, and the user can select an image finding through the operation of the case information terminal 200. This makes it possible to perform a display suitable for confirming the image findings that the doctor wants to confirm.

[0078] (Modification 3 of the second embodiment) In the above-described second embodiment, in step S1006, the finding selection unit 110 obtains the selected image findings based on the results and influence of the diagnostic inference. However, the finding selection unit 110 may obtain the selected image findings based only on the likelihood of the image findings, regardless of the results and influence of the diagnostic inference.

[0079] Here, the likelihood of an image finding is the largest of the likelihoods of each element of the image finding. For example, the image finding F shown in FIG. 1 In the case of "shape", the likelihood of each element L(f 11 ),L(f 12 ),L(f 13 ),L(f 14 ) is selected as the largest of the imaging findings, F 1 Here, the likelihood of all findings is calculated, and the image finding with the lowest likelihood is acquired as the selected image finding.

[0080] In addition, the likelihood can be combined with information on the degree of influence on diagnostic inference. For example, the image finding with the lowest likelihood may be acquired as the selected image finding from among a group of image findings whose degree of influence on diagnostic inference is equal to or greater than a certain level. This makes it possible to display image findings with low likelihood of inference and that require confirmation by a doctor in a manner suitable for confirmation.

[0081] [Third embodiment] The image processing device according to the third embodiment automatically displays an optimal image for observing image findings that require confirmation when interpretation results already exist for the target image. An interpretation result already exists when, for example, a previous image and its associated interpretation results exist, or when interpretation results by a different doctor exist. In the following, an example of the functional configuration and an example of the processing flow of the image processing device according to the third embodiment will be described in that order. Note that the hardware configuration is the same as that of the first embodiment, and therefore description thereof will be omitted here.

[0082] (Functional configuration) 12 is a diagram showing an example of the functional configuration of an image processing device in the third embodiment. In addition to the functional configuration of the second embodiment, an image processing device 1200 according to this embodiment includes an interpretation result storage unit 118. Note that functions other than those described below are the same as those described in the first and second embodiments, and therefore description thereof will be omitted here.

[0083] The image interpretation result storage unit 118, in accordance with the user's instruction via the case information terminal 200, associates information for identifying the image being interpreted with the coordinate information and the finding information of the abnormal shadow specified by the user, and stores the information as an image interpretation result. The information for identifying the image being interpreted is, for example, a patient ID, a case ID, an image ID, and a shooting date (shooting time). The coordinate information of the abnormal shadow is, for example, the coordinates (pixel values ​​in the depth, vertical direction, and horizontal direction) of the start point and the end point of a rectangular parallelepiped surrounding the abnormal shadow, and is assumed to be manually input by the user. The finding information may be the finding information generated by the finding acquisition unit 104, or may be manually input by the user. The image interpretation result is stored in a server (not shown). An example of the image interpretation result stored in FIG. 13 is shown.

[0084] (Processing flow) A processing flow performed in the third embodiment will be described with reference to Fig. 14, which is a flowchart showing an example of processing performed by the above-mentioned image processing device 900. Note that since some processing overlaps with the flowchart of Fig. 10 in the second embodiment, the overlapping steps are shown and descriptions thereof will be omitted.

[0085] The processing performed in steps S1401 to S1405 is similar to the processing performed in steps S1001 to S1005 in the second embodiment.

[0086] In step S1406, the finding acquiring unit 104 acquires the interpretation results corresponding to the target image from a server (not shown). In the third embodiment, the interpretation results to be acquired are those with the same patient ID but different imaging dates (imaging times), that is, past interpretation results of the same patient.

[0087] In step S1407, the finding selection unit 110 identifies an abnormal shadow that is the same as the abnormal shadow specified by the user from among all abnormal shadows included in the image interpretation result acquired in step S1406, and acquires image finding information related to the abnormal shadow from the image interpretation result. Note that the abnormal shadow is identified by comparing coordinate information. Before comparing the coordinate information, the image related to the image interpretation result and the image being interpreted may be aligned or transformed to correct the coordinate information.

[0088] Furthermore, the finding selection unit 110 acquires as selected image findings those whose values ​​are different when comparing the elements of each finding in the same abnormal shadow. At this time, the selected image findings may be narrowed down using the information on the degree of influence acquired in step S1405. Also, findings with changes indicating aggravation, such as an increase in the size of the abnormal shadow, may be preferentially acquired.

[0089] The processes performed in the following steps S1408 to S1410 are similar to the processes performed in steps S1007 to S1009 in the second embodiment.

[0090] As described above, in the image processing device according to the present disclosure, as shown in the functional configuration in the third embodiment, the finding acquisition unit 104 (finding acquisition means) may acquire the second image finding from the image interpretation result storage unit 118. In this aspect, the finding acquisition unit 104 functions as the finding acquisition means in the present disclosure. At this time, the second image finding includes, for example, the corresponding image interpretation result acquired in step S1406 described above. The finding selection unit 110 can select at least one image finding based on the first image finding group and the second image finding. Also, as described above, the finding information may be manually input by the user, and in this case, the finding acquisition unit 104 acquires multiple image findings by a method other than inference. In this way, the finding acquisition unit 104 can acquire image findings by various known methods, not limited to the acquisition of image findings by inference described in the first embodiment, for example.

[0091] By carrying out the above-described processing, it becomes possible to automatically determine image processing conditions suitable for confirming image findings that require confirmation by a doctor, such as image findings that change over time, and to automatically display images that satisfy the image processing conditions, thereby reducing the user's effort in adjusting images.

[0092] (Modification 1 of the third embodiment) In the third embodiment, the image interpretation result acquired in step S1406 is the corresponding image interpretation result that can be acquired from a server (not shown). However, the image interpretation result to be acquired is not limited to that from the server, and may be an image interpretation result created by the user himself or another user for the same target image. This makes it possible to display an image finding that is suitable for confirmation, for which the findings acquired by the findings acquisition unit 104 may be incorrect and require confirmation by a doctor.

[0093] [Variations] The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0094] The image processing device in each of the above-mentioned embodiments may be realized as a single device, or multiple devices may be combined so as to be able to communicate with each other and execute the above-mentioned processing, and either form is included in the embodiments of the present invention. The above-mentioned processing may be executed by a common server device or a group of servers. The image processing device and the multiple devices constituting the image processing system only need to be able to communicate at a predetermined communication rate, and do not need to be located in the same facility or the same country.

[0095] Therefore, the program code itself installed in a computer to realize the processing according to the embodiment is also one of the embodiments of the present invention. Also, based on the instructions contained in the program read by the computer, an OS running on the computer performs part or all of the actual processing, and the functions of the above-mentioned embodiments can also be realized by that processing. Furthermore, forms in which the above-mentioned embodiments are appropriately combined are also included in the embodiments of the present disclosure. [Explanation of symbols]

[0096] 100: Image processing device 102: Input information generation unit 104: Findings acquisition department 106: Diagnostic reasoning part 108: Influence degree acquisition part 110: Findings Selection Section 112: Decision section 114: Image acquisition unit 116: Display control section 200: Case information terminal

Claims

1. a finding acquisition means for acquiring a plurality of image findings as a first image finding group from a medical image of a diagnosis target; a finding selection means for selecting at least one image finding from the plurality of image findings; A determination means for determining image processing conditions suitable for observing the selected image findings; an image acquisition means for acquiring an image for display that satisfies the determined image processing conditions; An image processing device comprising:

2. The image processing apparatus according to claim 1 , wherein the finding acquisition means acquires image findings by inference on the medical image.

3. The image processing device according to claim 1 , wherein the finding selection means selects the image findings based on the likelihoods of the image findings inferred by the finding acquisition means.

4. The image processing apparatus according to claim 3 , wherein the finding acquisition means converts image features of the medical image of the diagnostic target into the first group of image findings to which the likelihoods are assigned.

5. The imaging findings inferred by the findings acquisition means may further include an influence degree acquisition means for acquiring an influence degree of the imaging findings on the diagnosis, The image processing apparatus according to claim 1 , wherein the finding selection means selects the image findings based on the acquired degree of influence.

6. The method further includes a diagnosis inference means for inferring a diagnosis using image findings as input, The image processing apparatus according to claim 5 , wherein the influence degree acquisition means acquires an influence degree of the image findings on the inferred diagnosis as the influence degree on the diagnosis.

7. The medical image processing method further includes: acquiring at least one second image finding corresponding to a medical image of the diagnostic subject that is captured at a different time; The image processing apparatus according to claim 1 , wherein the finding selection means selects the at least one image finding based on the first group of image findings and the second group of image findings.

8. The image processing device according to claim 1 , wherein the finding selection means selects the at least one image finding based on the first group of image findings and a second image finding obtained based on a medical image corresponding to the medical image of the diagnostic target.

9. The medical image processing device further includes a means for detecting an abnormal pattern in the medical image of the diagnosis object, The determining means determines the image processing conditions for the image findings related to the abnormal shadow; The image processing apparatus according to claim 1 , wherein the image acquisition means acquires, as an image to be displayed, an image that satisfies the image processing condition and includes the abnormal shadow.

10. The image processing apparatus according to claim 1 , wherein the determining means determines the image processing conditions based on the selected image finding item and image finding value.

11. The image processing apparatus according to claim 1 , wherein the image acquisition means acquires, as the display image, a medical image of the diagnostic object to which image processing based on the image processing conditions determined by the determination means has been applied.

12. The image processing device according to claim 11 , wherein the image processing based on the image processing condition is at least one of a brightness level change process, a tone conversion process, a resolution conversion process, and a cut-out process.

13. The image processing device according to claim 1 , wherein the image acquisition means acquires the display image based on the image processing conditions from a plurality of images to which different image processing has been applied.

14. The image processing apparatus according to claim 1 , further comprising a display control unit for causing a display unit to display the medical image of the diagnostic subject.

15. The image processing device according to claim 14 , wherein the display control means, when displaying the medical image of the diagnostic target, generates a partial medical image including an area corresponding to the image finding from the image for display and displays the partial medical image on the display means.

16. The image processing device described in claim 15, wherein the display control means causes the display means to display a superimposed image in which the partial medical image is superimposed on the medical image of the diagnostic target when image processing conditions applied to the display of the medical image of the diagnostic target differ from the image processing conditions of the partial medical image determined by the determination means.

17. The image processing device described in claim 15, wherein the display control means, when the image processing conditions applied to the display of the medical image of the diagnostic object differ from the image processing conditions of the partial medical image determined by the determination means, causes the display means to display the partial medical image in comparison with the medical image of the diagnostic object.

18. 2. The image processing device according to claim 1, further comprising a display control means for causing a display means to display the display image in comparison with the medical image of the diagnostic object when image processing conditions applied to display of the medical image of the diagnostic object differ from the image processing conditions determined by the determination means.

19. 15. The image processing device according to claim 14, wherein when a plurality of image findings are selected and image processing conditions are determined for each of the plurality of image findings, the display control means causes the display means to switchably display a display image that satisfies each of the plurality of image processing conditions together with the medical image of the diagnostic subject.

20. acquiring a plurality of image findings as a first image finding group for a medical image of a diagnostic target; selecting at least one imaging finding from the plurality of imaging findings; determining image processing conditions suitable for observing the selected image findings; obtaining a display image that satisfies the determined image processing conditions; and An image processing method comprising:

21. A program that, when executed by a computer, executes each step of the image processing method according to claim 20.

Citation Information

Patent Citations

  • Medical diagnosis support device, information processing method, and program

    JP2016214323A