Information processing apparatus, information processing method, and program

Abstract

To enhance efficiency of image interpretation when many lesions are detected.SOLUTION: An information processing apparatus includes: acquisition means for acquiring a plurality of lesion candidates from a medical image; determination means for determining a group of lesion candidates from the plurality of acquired lesion candidates; selection means for selecting a lesion candidate that represents the legion candidates for forming the group; and display means for discriminating between the information on the representative lesion candidate and the information on the lesion candidate other than the representative lesion candidate, forming the same group as the group of the representative legion candidate, and displaying the information together.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to an information processing device, an information processing method, and a program. [Background technology]

[0002] CADe (Computer-Aided Detection) is a method of analyzing medical images using a computer to detect candidates for lesions that are abnormalities associated with diseases. In addition, with the recent development of AI (Artificial Intelligence) technology, the accuracy of detecting target lesions has improved. For example, multiple lesions such as multiple lung metastases (metastatic lung tumors), polycystic kidney disease, and multiple liver tumors can be detected in a single medical image.

[0003] Doctors must individually check the images of the many detected lesions in this way, determine whether they are lesions associated with multiple diseases or lesions associated with different diseases, and make a diagnosis. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2003-033327 A [Patent Document 2] Patent Publication No. 2021-029387 [Patent Document 3] Special Publication No. 2017-519616 [Patent Document 4] JP 2018-097463 A [Patent Document 5] International Publication No. 2019 / 220801 [Patent Document 6] JP 2019-141478 A [Patent Document 7] Special Publication No. 2013-516269 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the efficiency of image interpretation when a large number of lesions are detected. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0006] An information processing device according to an embodiment includes an acquisition means, a determination means, a selection means, and a display means. The acquisition means acquires a plurality of lesion candidates from a medical image. The determination means determines a group of lesion candidates from the acquired plurality of lesion candidates. The selection means selects a representative lesion candidate from among the lesion candidates forming the group. The display means displays information about the representative lesion candidate and information about lesion candidates other than the representative lesion candidate that form the same group as the representative lesion candidate, in a manner that distinguishes them from each other. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a configuration of an information processing system according to the first and second embodiments. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of the information processing device according to the first and second embodiments. [Diagram 3] FIG. 3 is a diagram showing the functional configuration of the information processing device according to the first and second embodiments. [Figure 4] FIG. 4 is a diagram showing an example of a user interface screen of the information processing device according to the first and second embodiments. [Diagram 5] FIG. 5 is a diagram showing an example of a user interface screen when a representative lesion is designated in the information processing device according to the first and second embodiments. [Figure 6] FIG. 6 is a flowchart showing image display processing of the information processing device according to the first and second embodiments. [Figure 7] FIG. 7 is a diagram showing an example of a user interface screen of the information processing device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, the present invention will be described in detail based on the preferred embodiments with reference to the accompanying drawings. Note that unless otherwise specified, the same numbers are used for the items described in other embodiments, and the description thereof will be omitted. In addition, the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[0009] <Embodiment 1> In the first embodiment, an information processing device that displays medical images such as X-ray CT (Computed Tomography) images and MRI (Magnetic Resonance Imaging) images will be described. A doctor performs a diagnosis by displaying the medical images on the device, and creates the results as an image diagnosis report (hereinafter, referred to as an "image interpretation report" or simply as a "report") on a report system (not shown).

[0010] In the information processing device of this embodiment, one or more CADes operate to detect candidates for lesions (hereinafter referred to as "lesion candidates" or simply "lesions") that are clinical abnormalities that appear on medical images, and present the fact that the lesions have been detected to the user. The user selects one or more lesions from the presented lesions, observes the images of the lesions, and makes a diagnosis. In this embodiment, the medical images are X-ray CT images, and the lesions detected by the CADe are described as pulmonary nodules, but the present invention is not limited to this and may be other medical images such as MRI images, ultrasound images, mammography images, and plain X-ray images, or other lesions such as liver tumors, renal tumors, brain tumors, and bone lesions.

[0011] (System Configuration) FIG. 1 is a diagram showing the configuration of an information processing system including an information processing device according to the present embodiment.

[0012] In FIG. 1, the information processing system includes a case database (hereinafter, referred to as a “case DB”) 102, an information processing device 101, and a LAN (Local Area Network) 103.

[0013] The information processing device 101 acquires and displays medical image data (hereinafter, also referred to as "medical images") from the case DB 102. It also detects lesions from the medical images and displays the detection results. It also automatically generates and displays examples of text to be written in an image interpretation report (hereinafter, referred to as "image interpretation report" or "report") for the detected lesions.

[0014] The case DB 102 stores medical images captured by a device for capturing medical images, such as a CT device (not shown). The case DB 102 further provides the medical images to the information processing device 101 via a LAN 103. Specifically, the case DB 102 in this embodiment is a known PACS (Picture Archiving and Communication Systems).

[0015] (Hardware configuration) FIG. 2 is a diagram showing a hardware configuration of the information processing apparatus according to the present embodiment.

[0016] 2, the information processing device 101 includes a storage medium 201, a ROM (Read Only Memory) 202, a CPU (Central Processing Unit) 203, and a RAM (Random Access Memory) 204. It further includes a LAN interface 205, an input interface 208, a display interface 206, and an internal bus 211.

[0017] The storage medium 201 is a storage medium such as an SSD (Solid State Drive) that stores an OS (Operating System), a processing program for performing various processes according to the present embodiment, and various information. The ROM 202 stores a program for initializing hardware and starting the OS, such as a BIOS (Basic Input Output System). The CPU 203 performs arithmetic processing when the BIOS, the OS, and the processing program are executed. The RAM 204 temporarily stores information when the CPU 203 executes the BIOS, the OS, and the program. The LAN interface 205 is an interface that complies with standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.3ab and performs communication via the LAN 103. 207 is a display such as an LCD (Liquid Crystal Display) that displays a user interface screen. The display interface 206 converts screen information to be displayed on the display 207 into a signal for display control and outputs it to the display 207. 209 is a keyboard for key input, and 210 is a mouse for specifying a coordinate position on the display screen and inputting button operations. An input interface 208 receives signals based on key presses, button clicks, coordinate movement, etc. from a keyboard 209 and a mouse 210. An internal bus 211 transmits signals when communication is performed between each block.

[0018] (Functional configuration) FIG. 3 is a diagram showing the functional configuration of the information processing device 101 according to this embodiment.

[0019] 3, the information processing device 101 includes a medical image data acquisition unit 311, a lesion acquisition unit 312, a group determination unit 313, a representative lesion selection unit 314, a display unit 315, a report generation unit 316, and an interpretation status management unit 317. The case DB 102 stores medical image data 321-i (i=1, 2, 3, . . .). The medical image data 321-i (i=1, 2, 3, . . .) are files that comply with DICOM (Digital Imaging and Communications in Medicine), an international standard.

[0020] The medical image data acquisition unit 311 acquires medical image data 321-i (i=1, 2, 3, . . .) to be interpreted from the case DB 102 via the LAN 103. Here, the acquisition of the medical image data 321-i (i=1, 2, 3, . . .) complies with DICOM. It is assumed that the medical image data to be interpreted is designated in advance by the user.

[0021] Lesion acquiring unit 312 acquires a plurality of lesion candidates from a medical image. For example, lesion acquiring unit 312 detects lesions from medical image data 321-i (i=1, 2, 3, . . .) acquired by medical image data acquiring unit 311. Lesion acquiring unit 312 is an example of an acquiring means.

[0022] In this embodiment, pulmonary nodules are detected from X-ray CT images of the chest. Information on the result of lesion detection (hereinafter referred to as "lesion detection result") includes information indicating the area of ​​each detected lesion (hereinafter referred to as "lesion area") and information indicating the type of lesion (hereinafter referred to as "lesion type"). In this embodiment, the lesion area includes coordinate information of the lesion center (hereinafter referred to as "lesion position") and information on the size of the lesion (hereinafter referred to as "lesion size"). For example, when two lesions are detected, the lesion detection result is an array of 3-element tuples of lesion type, lesion position, and lesion size [(LT1, (X1, Y1, Z1), SZ1), (LT2, (X2, Y2, Z2), SZ2)]. Here, LTn (n=1,2) is the lesion type, Xn (n=1,2) is the horizontal coordinate value of the lesion position, Yn (n=1,2) is the vertical coordinate value of the lesion position, Zn (n=1,2) is the depth coordinate value of the lesion position, and SZn (n=1,2) is the lesion size. In addition, an identifier (hereinafter referred to as "lesion ID") that uniquely identifies the lesion is assigned to the detected lesion. For example, a lesion ID of L1 is assigned to a lesion at (LT1, (X1, Y1, Z1), SZ1), and a lesion ID of L2 is assigned to a lesion at (LT2, (X2, Y2, Z2), SZ2). The lesion area may be other information indicating the area, such as coordinate information of a bounding box circumscribing the lesion, coordinate information of each pixel corresponding to the lesion area, or mask image data labeled with the lesion area (hereinafter, "mask image data" is also simply referred to as "mask image").

[0023] A detector that uses machine learning of a convolutional neural network (CNN) is used to detect lesions. In machine learning of CNN, a plurality of medical images are collected in advance, and a mask image in which the lesion area is labeled is created in advance for each collected medical image. Then, machine learning is performed using a pair of a medical image and a mask image as training data. Specifically, a medical image is input to a CNN, and the parameters of the CNN are repeatedly changed so that the difference between the mask image output from the CNN and the mask image of the training data becomes small. Here, the parameters of the CNN are the kernel of the convolution operation, and the weight and bias value of the fully connected layer. Note that the lesion acquisition unit 312 may use a deep neural network (DNN) other than a CNN. Also, a classifier other than deep learning may be used, such as dividing a medical image into regions of a predetermined size, extracting image features such as known radiomics, and determining the presence or absence of a lesion for each region using a support vector machine (SVM). Note that the output of the detector / classifier is appropriately converted into the form of the lesion detection result.

[0024] The lesion acquisition unit 312 may detect multiple types of lesions, such as pulmonary nodules and liver masses. When detecting multiple types of lesions, a detector / classifier may be provided for each lesion type, or multiple types of lesions may be detected / classified at once using one detector / classifier. When multiple types of lesions are detected / classified using one detector / classifier, machine learning is performed using a multi-value mask image having different label values ​​for each lesion type as training data. The lesion acquisition unit 312 may acquire information on the results of detection processing performed in an external server device (not shown) or the like via the LAN 103.

[0025] Furthermore, the lesion acquisition unit 312 may acquire the lesion detection result based on a user operation on a user interface screen (not shown) instead of acquiring the lesion detection result by image analysis such as CNN. Specifically, the user specifies the position and size of the lesion in the displayed medical image. Furthermore, the lesion detection result may be a combination of the lesion detection result by image analysis and the lesion detection result specified by the user.

[0026] Group determination unit 313 determines a group of lesion candidates from among the multiple lesion candidates acquired by lesion acquisition unit 312. Group determination unit 313 is an example of a determination means. For example, group determination unit 313 acquires lesion detection results from lesion acquisition unit 312 and calculates similarities between lesions. Next, group determination unit 313 determines a group of multiple lesions whose mutual similarities are higher than a predetermined value as a group of similar lesions. That is, group determination unit 313 acquires similarities between each of the acquired multiple lesion candidates and other lesion candidates, and determines groups using the similarities.

[0027] The information on the group determination result (hereinafter, referred to as "similar lesion group") is an array of the lesion IDs that make up the group. For example, if the lesion IDs are L1, L2, and L3, the array [L1, L2, L3] becomes the similar lesion group, and an identifier that uniquely identifies the group, for example, LG1, is assigned.

[0028] Furthermore, the group determination unit 313 may output information regarding the similarity calculated when determining the group (hereinafter, referred to as "similarity"). For example, the similarity takes a real value between 0.0 and 1.0. Here, for example, if the similarity between lesions L1 and L2 is 0.79, the information regarding the similarity will be a tuple of three elements (L1, L2, 0.79). This similarity may be output for all combinations of detected lesions, or may be generated only for combinations of lesions that constitute one group.

[0029] In the group determination unit 313, the similarity is calculated, for example, from the distance of the image feature of the image corresponding to the lesion area, which is vectorized. Here, the similarity is normalized so that the distance is 0.0 for images with a long distance and 1.0 for images with a short distance. In addition, for example, a known Radiomics feature can be used as the image feature. The image feature may be a feature based on a histogram such as HOG (Histograms OF Oriented Gradients) or SIFT (Scaled Invariance Feature Transform). The image feature may be a feature such as the major axis (lesion size) of the lesion area, the minor axis, the major axis and minor axis of each of the three cross sections, the volume, the area of ​​each of the three cross sections, and the brightness distribution. In addition, the image feature may be a feature based on an image filter such as an autoencoder using CNN or LoG (Laplacian of Gaussian). In place of the image feature, image findings such as marginal characteristics, overall shape, and internal density, and differential diagnosis names may be estimated by CNN or image analysis, and the estimation results may be used. When estimating image findings using CNN, the image of the lesion area and the correct answer value of the image findings are used as training data, and machine learning is performed to input the image of the lesion area and output the image findings. Similarly, in the case of a differential diagnosis name, machine learning is performed to input the image of the lesion area and output one of the differential diagnosis names, for example, "primary lung cancer", "metastatic lung cancer", and "benign nodule". The correct answer value of the image findings is, for example, "smooth", "partially irregular", "irregular" for the marginal characteristics, and "spherical", "lobulated", "polyhedral", "irregular" for the overall shape. The internal density is determined, for example, by histogram analysis of the lesion area based on the ratio of the density (HU value) ranges corresponding to the solid component and the ground glass component, and classified into "Solid", "Part-Solid", and "Pure-GGN" in descending order of the solid component. Note that the determination of similar lesion groups may be performed using classifiers such as CNN, SVM, and DNN that have been machine-learned as a two-classification problem of "similar" and "not similar". In this case, the similarity is the likelihood of similarity.

[0030] The representative lesion selection unit 314 selects a representative lesion candidate (hereinafter, referred to as a "representative lesion") from among the lesion candidates forming the similar lesion group determined by the group determination unit 313. The representative lesion selection unit 314 is an example of a selection means. In addition, the representative lesion selection unit 314 outputs information related to the representative lesion.

[0031] For example, the representative lesion selection unit 314 selects and outputs the lesion L2 as the lesion representative of the similar lesion group LG1. In this embodiment, the representative lesion selection unit 314 selects a lesion designated by a user operation as the representative lesion of the similar lesion group to which the lesion belongs. That is, the representative lesion selection unit 314 accepts a user operation and selects a representative lesion. The designation by the user operation is performed, for example, via a user interface screen described with reference to Figs. 4 and 5.

[0032] When a representative lesion is designated, the display unit 315 displays information about the designated representative lesion, and also displays information about lesion candidates other than the representative lesion that form the same group as the representative lesion (hereinafter, referred to as "similar lesions"). The information about the lesion includes a partial image corresponding to the lesion area in the medical image, the lesion position, an interpretation report, and the like. Furthermore, the display unit 315 displays the information about the representative lesion and the information about similar lesions in a manner that distinguishes them from each other. That is, the display unit 315 displays the information about the representative lesion and the information about similar lesions together, distinguishing them from each other. The display unit 315 is an example of a display means.

[0033] The information can be distinguished by changing the thickness, line type, line color, etc. of the frame lines displaying the images and information. In the case of text, the information can be distinguished by changing the thickness, line type, line color, character color, background color, etc. In addition, when displaying the identifier of a representative lesion, the display unit 315 may also display the number (quantity) of similar lesions for the representative lesion. The image interpretation report on the lesion is generated by the report generation unit 316 described later.

[0034] When a representative lesion is designated, the report generating unit 316 generates an interpretation report on the designated representative lesion. The report generating unit 316 also generates reports on similar lesions in addition to reports on representative lesions. That is, the report generating unit 316 performs the same type of processing on similar lesions in accordance with the processing on the representative lesion. In other words, the report generating unit 316 performs the same type of processing on the representative lesion and similar lesions. The report generating unit 316 is an example of a processing means.

[0035] A report on a representative lesion is generated by, for example, estimating image findings from an image of the lesion area and applying the estimated image findings to a template. The template is, for example, "{lesion name by density} with {lesion size} mm margin {marginal characteristics} is found in {site}." Here, {} is the part to be replaced with image findings, etc. Specifically, if the site is "left lung lower lobe," the lesion size is "15 (mm)," the marginal characteristics is "smooth," and the density is "Solid" (the lesion name by density is "solid nodule"), the report generated is "15 mm solid nodule with smooth margin is found in the left lung lower lobe." Here, the lesion name by density is the name (text information) of the lesion uniquely associated with the density. In addition, the report on similar lesions is "multiple similar nodules are found in {distribution}" by evaluating the distribution from the lesion location of similar lesions. If the evaluated distribution is "both sides," the result will be "Multiple similar nodules are found on both sides." Other distributions include "same side" and "same lobe." Here, the location is determined using the results of organ segmentation (not shown) for the lung lobe and the location of the lesion. The report may be generated using a deep learning model such as a recurrent neural network (RNN) or a long short-term memory (LSTM). These models are machine-learned using training data in which the sequence of image findings is input and report text is output.

[0036] The image reading status management unit 317 manages the status regarding the image reading of the detected lesion (hereinafter, simply referred to as "image reading status"). The image reading status is information indicating whether the image reading of the lesion is completed or not. Here, the completion of image reading means a state in which the report regarding the lesion is confirmed. The image reading status management unit 317 may change the state regarding the image reading of the similar lesion in accordance with the change in the state regarding the image reading of the representative lesion. Specifically, when the report of the representative lesion and the similar lesion generated by the report generation unit 316 is confirmed by a user operation, the image reading status management unit 317 transfers the report to a report system (not shown) and changes the image reading status of the representative lesion to "completed". In addition, the image reading status management unit 317 also changes the image reading status of the similar lesion to "completed". That is, the image reading status management unit 317 performs the same type of processing on the similar lesion according to the processing on the representative lesion. In other words, the image reading status management unit 317 performs the same type of processing on the representative lesion and the similar lesion. The image interpretation status management unit 317 is an example of a processing means. Information on lesions for which image interpretation has been completed is displayed in a manner that distinguishes it from “incomplete” lesions by, for example, graying out the information or making the report unchangeable.

[0037] (User interface screen) 4 is a diagram showing an example of a user interface screen of the information processing device according to this embodiment. A user interface screen 400 is displayed on the display 207 by the display unit 315, and various operations by the user are input via the keyboard 209 and the mouse 210.

[0038] In FIG. 4, a user interface screen 400 is composed of a medical image display area 401 and a lesion detection result display area 402 .

[0039] The medical image display area 401 displays medical images acquired by the medical image data acquisition unit 311. In addition, for the medical image display area 401, the WL / WW (Window Level / Window Width) of the image, the position of the cross section (hereinafter also referred to as "slice"), the magnification rate, etc. can be changed in response to operations using the keyboard 209 or mouse 210.

[0040] In the lesion detection result display area 402, lesion detection results 421-i (i=1, 2, 3, . . .) related to lesions detected by the lesion acquisition unit 312 are displayed in a list. In the lesion detection result 421-i, text 422 for identifying the detected lesion, a representative image 423 of the detected lesion, and a number of similar lesions 424 for the detected lesion are displayed. Here, the representative image 423 of the lesion is an image obtained by cutting out a predetermined range of the medical image, for example, an axial cross section of a predetermined size from the center of the lesion area, based on the lesion position of the detected lesion. If the lesion size is larger than the display area of ​​the representative image, a predetermined range based on the lesion size is cut out and reduced for display. Note that, if there are multiple types of lesions that can be detected, information related to the type of the detected lesion is also displayed in the lesion detection result 421-i. The representative image may be an image of another cross section, such as an image of a cross section in which the long axis of the lesion is maximum in the axial cross section. If the lesion size is smaller than a predetermined size, the representative image may be enlarged and displayed.

[0041] Fig. 5 is a diagram showing an example of a user interface screen when a representative lesion is designated in the information processing device according to this embodiment. Various displays shown in Fig. 5 are also displayed on the display 207 by the display unit 315. In addition, various operations by the user are input via the keyboard 209 and the mouse 210.

[0042] In FIG. 5, 505 is a mouse pointer, which indicates that the lesion corresponding to the lesion detection result 421-1 is specified by an operation such as a mouse click by the user. In this embodiment, a case will be described in which the lesion specified by the user is selected as a representative lesion of a similar lesion group to which the lesion belongs. When the specification of the representative lesion is detected, the display unit 315 highlights the lesion detection result 421-1 as information on the specified representative lesion. In addition to highlighting the lesion detection result for the representative lesion, the display unit 315 also highlights the lesion detection results for similar lesions of the representative lesion. Specifically, the display unit 315 highlights the lesion detection results 421-2, 421-4, 421-6, and 421-8. Furthermore, the display unit 315 displays the lesion detection results for the representative lesion and the lesion detection results for similar lesions of the representative lesion in a manner that distinguishes them from each other. Specifically, the display unit 315 displays the frame of the lesion detection result 421-1 of the representative lesion with a thick solid line, and the frames of the lesion detection results 421-2, 421-4, 421-6, 421-8 of similar lesions with thin dashed or dotted lines. Furthermore, the display unit 315 changes the display mode of the lesion detection results for similar lesions based on the similarity between the representative lesion and each similar lesion. Specifically, the display unit 315 displays the frame lines of the lesion detection results for lesions whose similarity is higher than a predetermined value with dashed lines, and the frame lines of the lesion detection results for lesions whose similarity is lower than a predetermined value with dotted lines.

[0043] Furthermore, when the designation of the representative lesion is detected, the display unit 315 changes the slice position of the image to be displayed in the medical image display area 401 to the center position of the lesion area of ​​the designated representative lesion. Then, the display unit 315 displays a region of interest 511-1 of the representative lesion (a rectangle surrounding the lesion) as information on the representative lesion on the displayed image. In addition to displaying the region of interest for the representative lesion, the display unit 315 also displays the region of interest for lesions that are included in the image being displayed among similar lesions of the representative lesion. Specifically, the display unit 315 displays the regions of interest 511-2, 511-3, and 511-4 of similar lesions. Furthermore, the display unit 315 displays the region of interest for the representative lesion and the region of interest for the similar lesion of the representative lesion in a manner that distinguishes them from each other. Specifically, the display unit 315 displays the frame of the region of interest 511-1 of the representative lesion with a thick solid line, and the frames of the regions of interest 511-2, 511-3, and 511-4 of similar lesions with a thin dashed or dotted line. Here, similarly to the case of the lesion detection result, the display unit 315 changes the display mode of the region of interest for the similar lesion based on the similarity between the representative lesion and each similar lesion. Specifically, the display unit 315 displays lesions whose similarity is higher than a predetermined value with a dashed line, and lesions whose similarity is lower than the predetermined value with a dotted line.

[0044] Furthermore, when the designation of the representative lesion is detected, the display unit 315 displays a partial image 521-1, which is an image corresponding to the lesion area of ​​the representative lesion in the medical image, as information on the representative lesion. Furthermore, the display unit 315 displays a partial image of a similar lesion of the representative lesion together with the display of the partial image of the representative lesion. Specifically, the display unit 315 displays the partial image 521-i (i=2, . . . , 6). Furthermore, the display unit 315 displays the partial image of the representative lesion and the partial image of the similar lesion of the representative lesion in a manner that distinguishes them from each other. Specifically, the display unit 315 displays the frame of the partial image 521-1 of the representative lesion with a thick solid line, and the frame of the partial image 521-i (i=2, . . . , 6) of the similar lesion with a thin dashed or dotted line. Here, similarly to the case of the lesion detection result described above, the display unit 315 changes the display manner of the partial image of the similar lesion based on the similarity between the representative lesion and each similar lesion. Specifically, the display unit 315 displays lesions with a similarity higher than a predetermined value with a dashed line and lesions with a similarity lower than a predetermined value with a dotted line to further distinguish them from one another. The display unit 315 also arranges partial images of similar lesions on the screen in an order based on the similarity. Specifically, the display unit 315 arranges partial images 521-i (i=2, . . . , 6) of similar lesions on the screen in descending order of similarity. Here, the group determination unit 313 can determine a group based on the similarity corresponding to a position designated by a user operation on the displayed information on the similar lesion. For example, the group determination unit 313 can acquire the similarity of a lesion to be excluded from a similar lesion group based on a designation from the user. Specifically, when the user designates a boundary between partial images, for example, the boundary 522 between partial images 521-6 and 521-7, the group determination unit 313 excludes a lesion with a lower similarity than the similar lesion corresponding to partial image 521-7 from the similar lesion group. The group determination unit 313 can also accept a user's designation of a lesion to be excluded from the similar lesion group. For example, the user can exclude a lesion from the similar lesion group by placing the mouse pointer on a partial image and pressing the Delete key. Furthermore, the display unit 315 can change the position of a slice to be displayed in a partial image based on a user operation on the partial image.That is, the display unit 315 can change the position of the cross section to be displayed in the partial image by accepting an operation on the partial image. For example, when the display unit 315 detects an operation such as the rotation of the mouse wheel or pressing of the up or down key on the partial image 521-i (i=1, . . . , 6), the display unit 315 changes the position of the slice to be displayed in the partial image. Furthermore, the display unit 315 can change the slice position of the medical image to be displayed in the medical image display area to the slice position of the lesion corresponding to the partial image based on the user operation on the partial image. Specifically, when the display unit 315 detects an operation such as a mouse click on the partial image, the display unit 315 changes the slice position of the medical image to be displayed in the medical image display area 401 to the slice position corresponding to the clicked partial image. Furthermore, when displaying the partial images, the display unit 315 can accept a designation of whether to resize (enlarge or reduce) each partial image individually based on the size of each lesion, to set the same magnification for multiple lesions, or to set the same magnification as the medical image to be displayed in the medical image display area. Specifically, "resize" and "actual size" can be specified by using the radio buttons in the partial image display control specification area 502. Here, when the specification of "resize" is detected, the display unit 315 enlarges or reduces the image of the lesion area so that the lesion is approximately the same size in the partial image and displays it. Furthermore, when the display unit 315 detects the specification of "actual size", it displays the image of the lesion area at a predetermined magnification in all of the partial images 521-i (i = 1,...,6). Furthermore, when the display unit 315 detects a check in the "linked with viewer" checkbox in the partial image display control specification area 502, it displays the image of the lesion area at the same magnification as the magnification of the medical image displayed in the medical image display area 401 in all of the partial images 521-i (i = 1,...,6). Note that, when the display unit 315 detects that the image of the lesion area does not fit within the partial image 521-i (i = 1,...,6) by the above operation, it may reduce the size to a size that fits the image of the lesion area. Furthermore, the display unit 315 may impose a restriction so that the magnification does not exceed a predetermined magnification.

[0045] Furthermore, when the display unit 315 detects the designation of the representative lesion, it displays the lesion position 503 based on the lesion position. Here, the lesion position 503 is a display in which a figure (rectangle) indicating the position of the designated representative lesion and a figure (rectangle) indicating the position of a similar lesion are arranged in accordance with the corresponding positions on a two-dimensional schema. Here, the figure such as a rectangle shown in the lesion position 503 is an example of information related to the position. That is, the display unit 315 displays information related to the position of the similar lesion together with information related to the position of the representative lesion. Here, like other displays, the display unit 315 displays the position of the representative lesion and the position of the similar lesion in a manner that distinguishes them. Specifically, the display unit 315 displays the representative lesion with a thick solid line and the similar lesion with a thin dashed or dotted line. Furthermore, the display unit 315 displays the position of the similar lesion in a manner that differs based on the similarity between the representative lesion and the similar lesion. Specifically, the display unit 315 displays lesions with a similarity higher than a predetermined value with a dashed line and lesions with a similarity lower than a predetermined value with a dotted line. Furthermore, when the display unit 315 detects that the user has specified a figure indicating the position of the lesion, it changes the slice position of the medical image displayed in the medical image display area 401 to the slice position corresponding to the position of the specified lesion.

[0046] Furthermore, when the display unit 315 detects the designation of a representative lesion, it displays the image interpretation report 504-1 regarding the designated representative lesion, and also displays the image interpretation report 504-2 of the similar lesion. Furthermore, the display unit 315 displays the image interpretation report of the representative lesion and the image interpretation report of the similar lesion in a manner that distinguishes them from each other. Specifically, the display unit 315 displays the underline of the image interpretation report 504-1 of the representative lesion with a thick solid line, and the underline of the image interpretation report 504-2 of the similar lesion with a thin broken line. Here, the image interpretation reports 504-1 and 504-2 are generated by the report generation unit 316. The image interpretation report 504-1 is an example of a finding for the representative lesion. Furthermore, the image interpretation report 504-2 is an example of a finding for the similar lesion. That is, the report generation unit 316 acquires the finding for the similar lesion together with the acquisition of the finding for the representative lesion, and the display unit 315 displays the finding for the similar lesion together with the finding for the representative lesion. Here, the text of the image interpretation reports 504-1 and 504-2 can be arbitrarily edited by the user by clicking the editing position with the mouse and inputting from the keyboard.

[0047] The display unit 315 also displays a "Findings Confirmation" button 541 along with the image interpretation reports 504-1 and 504-2. When a mouse click of the "Findings Confirmation" button 541 by the user is detected, the image interpretation reports 504-1 and 504-2 are transmitted to a report system (not shown), and the image interpretation status management unit 317 changes the image interpretation status of the corresponding representative lesion to "Completed", and also changes the image interpretation status of the similar lesion to "Completed". The display unit 315 also changes information about lesions whose image interpretation status has become "Completed" to gray text, a frame, or the like, and displays it in a manner that distinguishes it from "Incomplete" lesions.

[0048] (Processing flow) 6 is a flow diagram showing image display processing of the information processing device according to this embodiment. This processing is started based on an instruction from another system or a user after the information processing device 101 is started. When the processing is started, a medical image to be interpreted is specified.

[0049] In step S601, the medical image data acquisition unit 311 acquires medical image data 321-i (i=1, 2, 3, . . . ) designated at the time of startup from the case DB 102 via the LAN 103.

[0050] In step S602, the display unit 315 displays the acquired medical image data 321-i (i=1, 2, 3, . . . ) in the medical image display area 401 of the user interface screen 400.

[0051] In step S603, the lesion acquisition unit 312 detects lesions from the acquired medical image data 321-i (i=1, 2, 3, . . . ).

[0052] In step S604, display unit 315 displays a list of lesion detection results 421-i (i=1, 2, 3, . . . ) based on the lesion detection results obtained from lesion acquisition unit 312.

[0053] In step S605, the group determination unit 313 determines the similar lesion group based on the lesion detection result and the medical image. The display unit 315 also displays the number of similar lesions 424 of the determined similar lesion group in the lesion detection result 421-i (i=1, 2, 3, . . .). That is, the display unit 315 displays the number of similar lesions as information about similar lesions that form the same group as the representative lesion.

[0054] In step S606, the representative lesion selection unit 314 selects the lesion corresponding to the designation as a representative lesion based on the detection of the designation by the user for the lesion detection result (421-1 in the example of FIG. 5). The display unit 315 also highlights the lesion detection result 421-1 for the representative lesion, and also highlights the lesion detection results 421-i (i=2, 4, 6, 8) for similar lesions. Furthermore, the frame of the lesion detection result 421-1 for the representative lesion is displayed with a thick solid line, and the frame of the lesion detection results 421-i (i=2, 4, 6, 8) for similar lesions is displayed with a thin dashed or dotted line, to distinguish between the representative lesion and the similar lesion. In this embodiment, the designated lesion is selected as the representative lesion and is also selected as the representative lesion to be displayed.

[0055] In step S607, the display unit 315 displays information about the representative lesion, and also displays information about similar lesions. Furthermore, the display of the representative lesion and the display of similar lesions are displayed in a manner that distinguishes between them. Here, the information about the representative lesion is, for example, the region of interest 511-1, partial image 521-1, and a rectangle that is a part of the lesion position 503. Moreover, the information about similar lesions is the regions of interest 511-2, 511-3, 511-4, partial images 521-i (i=2, . . . , 7), and a rectangle that is a part of the lesion position 503. Furthermore, the information about the representative lesion and the information about similar lesions are displayed in a manner that distinguishes between the thickness of the frame line, the line type, and the like. Note that the information about the lesion and the display manner are merely examples, and are not limited to these.

[0056] In step S608, the report generating unit 316 judges whether or not an image interpretation report for the designated representative lesion has been generated. If an image interpretation report has been generated (Yes in step S608), the process proceeds to step S622. If an image interpretation report has not been generated (No in step S608), the process proceeds to step S621.

[0057] In step S621, the report generator 316 generates an image interpretation report for the selected representative lesion, and also generates an image interpretation report for similar lesions. The image interpretation report is generated by estimating image findings from medical images of the lesion area using a CNN trained on medical images and image findings of the lesion area as training data, and applying the estimated image findings to a report text template. For similar lesions, the distribution is evaluated from the lesion positions of similar lesions, and a description of the distribution is generated using the template.

[0058] In step S622, the display unit 315 displays the image interpretation report 504-1 for the representative lesion and the image interpretation report 504-2 for the similar lesion together. Furthermore, the image interpretation report 504-1 for the representative lesion and the image interpretation report 504-2 for the similar lesion are displayed with different aspects such as the thickness and type of underlines.

[0059] In step S624, the interpretation state management unit 317 determines whether a finding confirmation operation has been detected, specifically, whether the finding confirmation button 541 has been clicked. If a finding confirmation operation has been detected (Yes in step S624), the process proceeds to step S631. If a finding confirmation operation has not been detected (No in step S624), the process proceeds to step S609.

[0060] In step S631, the interpretation status management unit 317 changes the interpretation status for the selected representative lesion to the "completed" status, and also changes the interpretation status for the similar lesions to the "completed" status.

[0061] In step S609, a control unit (not shown) determines whether termination such as shutdown of the information processing device 101 or termination of an application has been detected. If termination has been detected (Yes in step S609), the process is terminated, and if termination has not been detected (No in step S609), the process is repeated from S606.

[0062] As described above, in this embodiment, a similar lesion group is determined based on a predetermined criterion from among a plurality of acquired lesion candidates, a representative lesion is selected, and information on the similar lesion is also displayed together with information on the representative lesion. Furthermore, information on the representative lesion and information on the similar lesion are displayed in a manner that distinguishes them. In addition, the same type of processing as that for the representative lesion is also performed on the similar lesion. This makes it possible to interpret images while distinguishing between representative lesions and similar lesions, which has the effect of making interpretation more efficient when a large number of lesions are acquired.

[0063] As an example of the conventional technology, a diagnosis support device described in Patent Document 1 automatically detects features such as lesion position and lesion type and stores them in a diagnostic image database, and a doctor accesses the diagnostic image database to sequentially display images and lesion features to make a diagnosis. It also generates and outputs a provisional report creation template including image and lesion feature data, and the doctor creates a provisional report based on the provisional report creation template.

[0064] Also, for example, the medical information processing device and program described in Patent Document 2 detects multiple types of lesions in a medical image, and determines the priority of the lesion detection area based on a predetermined condition. Furthermore, display information of the detection area is generated so as to change the display form of the detected detection area according to the determined priority. The priority is determined based on whether the detection area is of a type of lesion specified by the user, whether it exists within an area specified by the user, the form, size, and position of the detection area, statistical information such as the probability of lesion occurrence, and the certainty of the lesion.

[0065] In the technology described in Patent Document 1, the creation of a provisional report makes it possible to efficiently create a report for each detected lesion, but when a large number of lesions are detected, it is necessary to confirm the report for each lesion. In the technology described in Patent Document 2, although the priority of the lesion is displayed, the doctor is required to check the condition of the lesion even for lesions with low priority. In other words, when a large number of lesions are detected, the work related to image interpretation, such as checking images of the lesions, filling out and confirming reports, increases. In response to this, the information processing device 101 according to the embodiment makes it possible to efficiently perform image interpretation when a large number of lesions are detected.

[0066] (Modification of the first embodiment) The representative lesion selection unit 314 may select the representative lesion based on any one or a combination of image findings such as the detected lesion size, overall shape, and marginal characteristics. That is, the representative lesion selection unit 314 may select the representative lesion based on any one of the size, shape, and marginal characteristics of the lesion candidate acquired by the lesion acquisition unit 312. Specifically, when the lesion size is used, the representative lesion selection unit 314 selects the lesion with the largest lesion size among the similar lesions as the representative lesion. When the overall shape is used, the representative lesion selection unit 314 selects the lesion with the most typical overall shape among the similar lesions, that is, the overall shape that occupies the majority of the similar lesions and has the highest likelihood, which is the likelihood of the findings estimated from the medical image of the lesion area, as the representative lesion. Similarly, when other image findings such as marginal characteristics are used, the representative lesion selection unit 314 selects the lesion with the most typical overall shape among the similar lesions and has the highest likelihood, which is the likelihood of the findings estimated from the medical image of the lesion area, as the representative lesion. For example, the representative lesion selection unit 314 selects the lesion with the highest likelihood of being estimated as a spherical lesion when the majority of similar lesions are spherical, and selects the lesion with the highest likelihood of being estimated as an irregular lesion when the majority of similar lesions are irregular. Similarly, the representative lesion selection unit 314 selects the lesion with the highest likelihood of being estimated as a smooth margin when the majority of similar lesions are smooth margin, and selects the lesion with the highest likelihood of being estimated as an irregular margin when the majority of similar lesions are irregular margin. In addition, when multiple selection criteria are combined, the representative lesion selection unit 314 selects the lesion with the largest sum of predetermined weights assigned to each item as the representative lesion. Here, the lesion size is acquired by the lesion acquisition unit 312. In addition, image findings such as the overall shape and margin characteristics are acquired by CNN or the like described in the group determination unit 313.

[0067] Furthermore, the group determination unit 313 may change the method of determining similar lesions based on the type of lesion acquired from the lesion acquisition unit 312. That is, the group determination unit 313 may change the method of determining groups based on the type of lesion candidate. For example, when the type of lesion is a pulmonary nodule, the group determination unit 313 determines the similarity based on the distance using the internal density, overall shape, marginal characteristics, etc. of the pulmonary nodule. Furthermore, when the type of lesion is a liver tumor, the group determination unit 313 determines the similarity using the internal density of the tumor, the time-dependent staining pattern in dynamic CT, the texture inside the tumor, etc. Furthermore, when the type of lesion is a brain tumor, the group determination unit 313 determines the similarity using the density around the lesion in a T2 weighted image or a FLAIR image, and the texture in a contrast-enhanced T1 weighted image.

[0068] The group determination unit 313 may also obtain multiple feature amounts from a lesion, and select a feature amount to be used for determining similarity based on the distribution of each feature amount. Here, the feature amount refers to image findings such as the internal density, overall shape, and marginal characteristics of the lesion, and Radiomics feature amount. The distribution of the feature amount refers to a histogram in which the number of lesions for each feature amount or for each range of feature amounts is plotted. In addition, in selecting the feature amount, the group determination unit 313 may select, for example, a feature amount having a singular value, specifically, a feature amount in which a lesion exists that is isolated from a peak that occupies a majority of the distribution in a histogram of the feature amount distribution. That is, the group determination unit 313 may obtain multiple feature amounts for each of the lesion candidates, evaluate the distribution of each feature amount, select a feature amount having a singular value based on the distribution, and determine the group based on the selected feature amount.

[0069] In addition, the group determination unit 313 may determine the lesion groups based on information other than the morphology, such as the anatomical location of the lesion, for example, the lung section, instead of determining the lesions as a group based on similar morphology. In this case, the similarity is regarded as the anatomical distance within or between the lung sections. Here, the anatomical distance is the length in terms of the bronchial connections.

[0070] In this embodiment, since the representative lesion is automatically selected from among similar lesions, in step S606 in Figure 6, from among the automatically selected representative lesions, the representative lesion specified based on a user operation is selected as the representative lesion to be displayed.

[0071] As described above, according to this modification, a similar lesion group is determined based on a predetermined criterion from among the acquired multiple lesion candidates, a representative lesion is selected, and information about the similar lesion is also displayed together with information about the representative lesion. Furthermore, information about the representative lesion and information about the similar lesion are displayed in a manner that distinguishes them. Furthermore, the same type of processing as that for the representative lesion is performed on the similar lesion. Furthermore, the representative lesion is automatically selected according to a predetermined condition. Furthermore, the determination method for the similar lesion can be automatically changed according to the type of lesion. Furthermore, the feature amount used for determining the similar lesion can be automatically selected based on the distribution of the feature amount. This makes it possible to interpret the image while distinguishing between the representative lesion and the similar lesion, and has the effect of making the interpretation more efficient when a large number of lesions are acquired. Furthermore, there is an effect that an appropriate similar lesion can be automatically selected according to the type of lesion without the user selecting the representative lesion. Furthermore, since the similar lesion can be determined so as to distinguish between a large number of similar lesions and a small number of specific lesions based on the histogram of the feature amount, there is an effect that an appropriate similar lesion can be determined for the detected lesion.

[0072] <Embodiment 2> The information processing device of this embodiment differs from the information processing device of the first embodiment in that a previous image of the medical image to be interpreted is acquired and used for processing. Here, the previous image is a medical image that has roughly the same shooting range as the medical image to be interpreted and was taken of the same patient on a date and time earlier than the target medical image. Note that the system configuration of the information processing device of this embodiment is the same as that of the first embodiment described using FIG. 1, the hardware configuration is the same as that of FIG. 2, the functional configuration is the same as that of FIG. 3, the user interface screen is the same as that of FIG. 4 and FIG. 5, and the processing flow is the same as that of FIG. 6, so that the description will be omitted.

[0073] The group determination unit 313 of this embodiment acquires information on changes from the lesion in the past image corresponding to each detected lesion, and determines similar lesions based on the information on the changes (hereinafter, also simply referred to as "changes"). That is, the group determination unit 313 acquires changes from past lesion candidates corresponding to the lesion candidates, and determines the group of the lesion candidates based on the acquired changes. The information on changes is information on changes in lesion size and information on changes in findings. Specifically, regarding changes in size, the group determination unit 313 determines that lesions whose lesion size difference between the target lesion and the lesion in the past image is within a predefined range, for example, less than 1 mm, 1 mm or more and less than 5 mm, 5 mm or more and less than 10 mm, and 10 mm or more, are groups of similar lesions. Regarding changes in findings, the group determination unit 313 determines that lesions whose predetermined findings have undergone a predetermined change, such as an increase in spicules, an increase in irregular margins, or the disappearance of ground-glass areas, are groups. Here, the past images are selected from those that have the same patient ID, examination site, and modality included in the DICOM tag as the medical image to be interpreted, and have an earlier examination date. For the lesions in the past images corresponding to each detected lesion, multiple landmarks that indicate anatomical image characteristics are defined in advance, and the lesion in the past image that has approximately the same positional relationship with the landmark as the target lesion is selected as the corresponding lesion. The size of the lesion (lesion size) is acquired from the lesion acquisition unit 312.

[0074] The group determination unit 313 may determine whether or not a past image exists, and if a past image does not exist, determine similar lesions without considering changes in findings over time, and if a past image exists, determine similar lesions taking changes in findings over time into consideration. In this case, if a past image exists, the group determination unit 313 may increase the weighting of changes from past lesion candidates when determining groups. For example, if a past image exists, the group determination unit 313 may increase the weighting of size changes when determining similar lesions. For example, if a past image does not exist, the group determination unit 313 determines groups of similar lesions based on the internal density, overall shape, and distance of the marginal properties of the lesion, but if a past image exists, the group determination unit 313 determines groups of similar lesions by weighting the distance for size changes more heavily than other distances. That is, the group determination unit 313 changes the method of determining groups of similar lesions depending on whether or not a past image exists.

[0075] As described above, according to this embodiment, a similar lesion group is determined based on a predetermined criterion from among a plurality of acquired lesion candidates, a representative lesion is selected, and information on the similar lesion is also displayed together with information on the representative lesion. Furthermore, information on the representative lesion and information on the similar lesion are displayed in a differentiated manner. Furthermore, the same type of processing as that for the representative lesion is performed on the similar lesion. Furthermore, a lesion having a similar change in size from the corresponding lesion in a past image is determined to be a similar lesion. This makes it possible to interpret images while distinguishing between the representative lesion and the similar lesion, which has the effect of making interpretation more efficient when a large number of lesions are acquired. Furthermore, since lesions having a similar change in size from the corresponding lesion in a past image can be interpreted together, it has the effect of making interpretation more efficient when observing the course of a large number of lesions.

[0076] (Modification of the second embodiment) When a past image exists, the display unit 315 may obtain the change in size of each of the lesion candidates with respect to the corresponding lesion candidate in the past image, and display the statistical value of the change in size for each group, or may display information about the lesion candidate whose change in size is unique within the group. For example, when a past image exists, the display unit 315 obtains the change in size of each of the similar lesions from the corresponding lesion in the past image, and calculates and displays the statistical value of the change in size within the similar lesion. Furthermore, when a lesion having a singular value exists within the similar lesion, the display unit 315 displays information that the lesion has a singular value. Specifically, when a representative lesion is designated, the display unit 315 displays the change in size of the representative lesion, and also displays the statistical value of the change in size of the similar lesion. The statistical value is, for example, the average value, maximum value, minimum value, median value, and standard deviation. Furthermore, a similar lesion whose absolute value of the difference from the average value is equal to or greater than a predetermined value is determined to be a similar lesion having a singular value, and is displayed in a manner further distinguished from other similar lesions.

[0077] Fig. 7 is a diagram showing an example of a user interface screen when a representative lesion is designated in the information processing device according to this embodiment. Various displays shown in Fig. 7 are displayed on the display 207 by the display unit 315, and various operations by the user are input via the keyboard 209 and the mouse 210.

[0078] 7, 503, 504-1, 504-2, and 541 are the lesion position, image interpretation report on representative lesion, image interpretation report on similar lesion, and the findings confirmation button, which have been described with reference to FIG. 5. Also, 701 shows an example of a screen area (hereinafter, referred to as "past comparison information") that displays information on the change in size from the corresponding lesion in a past image. Here, each screen shown in FIG. 7 is displayed at a predetermined position on the user interface screen 400.

[0079] The date of the past image to be compared, the size change of the representative lesion, and the average, maximum, and minimum values ​​of the size change of similar lesions are displayed in the past comparison information 701. In the example of Fig. 7, the date of the past image is "2022 / 2 / 2", the size change of the representative lesion is "+3 mm", the average value of the size change of similar lesions is "+2.5 mm", the maximum value is "+10 mm", and the minimum value is "-5.0 mm".

[0080] Furthermore, lesions with specific values ​​of size change among similar lesions including the representative lesion are distinguished and displayed in lesion position 503. For example, 731 is a lesion whose size has specifically increased, and 732 is a lesion whose size has specifically decreased, and these are displayed with "+" and "-", respectively, to distinguish them from other lesions.

[0081] As described above, in this modification, a similar lesion group is determined based on a predetermined criterion from among the acquired multiple lesion candidates, a representative lesion is selected, and information about the similar lesion is also displayed together with information about the representative lesion. Furthermore, information about the representative lesion and information about the similar lesion are displayed in a differentiated manner. Furthermore, the same type of processing as that for the representative lesion is performed on the similar lesion. Furthermore, within the similar lesion, statistics of the change in size from the corresponding lesion in a past image and specific lesions can be displayed. This makes it possible to interpret images while distinguishing between the representative lesion and the similar lesion, which has the effect of making interpretation more efficient when multiple lesions are acquired. Furthermore, it becomes easier to distinguish between lesions whose size change from the corresponding lesion in a past image is specific, which has the effect of making interpretation more efficient when observing the course of multiple lesions.

[0082] Each component of the device according to the above-mentioned embodiment is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or a part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0083] The information processing method described in the above embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. This program can also be recorded on a non-transitory computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, or a DVD, and executed by being read from the recording medium by a computer.

[0084] According to at least one of the embodiments described above, it is possible to improve the efficiency of image interpretation when a large number of lesions are detected.

[0085] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0086] 101: Information processing device 102:Case DB 103: LAN 201:Storage medium 202:ROM 203: CPU 204:RAM 205: LAN interface 206: Display Interface 207: Display 208: Input interface 209: Keyboard 210: Mouse 211: Internal bus 311: Medical image data acquisition unit 312: Lesion Acquisition Department 313: Group determination unit 314: Representative Lesion Selection Section 315: Display section 316: Report generation unit 317: Image interpretation status management unit

Claims

1. An acquisition means for acquiring a plurality of lesion candidates from a medical image; A determination means for determining a group of lesion candidates from among the acquired plurality of lesion candidates; a selection means for selecting a representative lesion candidate from among the lesion candidates forming the group; a display means for displaying information on the representative lesion candidate and information on lesion candidates other than the representative lesion candidate which are in the same group as the representative lesion candidate, in a distinguished manner; An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the determining means obtains a similarity between each of the acquired plurality of lesion candidates and other lesion candidates, and determines the group using the similarity.

3. The information processing apparatus according to claim 1 , further comprising a processing means for performing the same type of processing on the representative lesion candidate and lesion candidates other than the representative lesion candidate that form the same group as the representative lesion candidate.

4. The information processing apparatus according to claim 1 , wherein the selection means selects the representative lesion candidate in response to a user operation.

5. The information processing apparatus according to claim 1 , wherein the selection means selects the representative lesion candidate based on any one of a size, a shape, and a marginal property of the acquired lesion candidates.

6. The information processing apparatus according to claim 2 , wherein the display means changes a display mode of similar lesions based on the degree of similarity.

7. The information processing device according to claim 1 , wherein the display means displays a partial image, which is a part of the medical image and corresponds to the representative lesion candidate, together with partial images corresponding to lesion candidates other than the representative lesion candidate that form the same group as the representative lesion candidate.

8. The information processing device according to claim 1 , wherein the display means displays, together with information regarding the position of the representative lesion candidate, information regarding the positions of lesion candidates other than the representative lesion candidate that form the same group as the representative lesion candidate.

9. the processing means, in conjunction with the acquisition of findings for the representative lesion candidate, acquires findings for lesion candidates other than the representative lesion candidate which form the same group as the representative lesion candidate; The information processing apparatus according to claim 3 , wherein the display means displays findings for the representative lesion candidate together with findings for lesion candidates other than the representative lesion candidate which form the same group as the representative lesion candidate.

10. The information processing device according to claim 3 , wherein the processing means changes a state related to the interpretation of lesion candidates other than the representative lesion candidate that form the same group as the representative lesion candidate in accordance with a change in a state related to the interpretation of the representative lesion candidate.

11. the acquiring means acquires a type of the lesion candidate together with the acquired lesion candidate; The information processing apparatus according to claim 1 , wherein the determining means changes a method of determining the group based on a type of the lesion candidate.

12. 2. The information processing device according to claim 1, wherein the determination means acquires a plurality of features for each of the acquired lesion candidates, evaluates a distribution for each of the features, selects a feature having a singular value based on the distribution, and determines the group based on the selected feature.

13. The information processing apparatus according to claim 1 , wherein the determining means obtains a change from a past lesion candidate corresponding to the acquired lesion candidate, and determines the group based on the change.

14. The information processing apparatus according to claim 13 , wherein the determining means increases a weighting for the change when determining the group when a previous image of the medical image exists.

15. The information processing device according to claim 13 or 14, wherein the change is a change in size of the lesion candidate.

16. The information processing device according to claim 13 or 14, wherein the change is a change in findings for the lesion candidate.

17. the display means displays information about lesion candidates other than the representative lesion candidate that form the same group as the representative lesion candidate in an order based on the similarity; The information processing apparatus according to claim 2 , wherein the determining means further determines the group based on a similarity corresponding to a position designated by a user operation on the displayed information on the lesion candidate.

18. The information processing apparatus according to claim 1 , wherein the display means displays the number of lesion candidates as information about lesion candidates other than the representative lesion candidate that form the same group as the representative lesion candidate.

19. the medical image is composed of a plurality of cross-sectional images; The information processing apparatus according to claim 7 , wherein the display means receives an operation on the partial image and changes a position of the cross section to be displayed in the partial image.

20. The information processing device according to claim 7, wherein the display means, when displaying the partial images, accepts a specification of either enlarging or reducing each of the partial images based on the size of each of the acquired lesion candidates, applying the same magnification to multiple lesion candidates, or applying the same magnification as the display of the medical image.

21. 2. The information processing device according to claim 1, wherein, when a previous image corresponding to the medical image exists, the display means acquires, for each of the acquired plurality of lesion candidates, a change in size relative to the corresponding lesion candidate in the previous image, and displays statistics regarding the change for each of the groups, or displays information regarding the lesion candidate whose change is specific within the group.

22. Obtaining multiple lesion candidates from medical images; determining a group of lesion candidates from the acquired plurality of lesion candidates; selecting a representative lesion candidate from among the lesion candidates forming the group; Information on the representative lesion candidate and information on lesion candidates other than the representative lesion candidate which form the same group as the representative lesion candidate are displayed together in a distinguished manner.

2. An information processing method comprising:

23. Obtaining multiple lesion candidates from medical images; determining a group of lesion candidates from the acquired plurality of lesion candidates; selecting a representative lesion candidate from among the lesion candidates forming the group; Information on the representative lesion candidate and information on lesion candidates other than the representative lesion candidate which form the same group as the representative lesion candidate are displayed together in a distinguished manner. A program that causes a computer to execute each process.

Images