Information processing device, information processing method, and program

Abstract

To provide an information processing device, information processing method and program which allow a user to efficiently grasp another lesion related to a lesion of interest.SOLUTION: An information processing device according to the present invention comprises: a detection result acquisition unit 313 which acquires a result of detecting a lesion from medical image data; an instruction reception unit 314 which receives an instruction of designating a lesion of interest on the basis of the detection result; a related lesion information acquisition unit 315 which acquires information about a related lesion that is a lesion related to the designated lesion of interest and is a lesion in a different type from the lesion of interest; and a display control unit 316 which displays the information about the related lesion on a display unit.SELECTED DRAWING: Figure 3

Description

[[Technical Field]]

[0001] The disclosure of the present specification relates to an information processing apparatus, an information processing method, and a program that display other lesions related to a lesion focused on by a user. [[Background Art]]

[0002] CADe (Computer-Aided Detection), which analyzes medical images with a computer and detects candidate lesions that are abnormalities associated with diseases, is known. In addition, with the development of AI (Artificial Intelligence) technology, the types of lesions that can be detected have been expanding.

[0003] The medical diagnosis support apparatus described in Patent Document 1 divides medical image data into micro-regions, calculates a feature amount for each micro-region, and displays image data with different display densities or display colors superimposed on the medical image data according to the feature amount. [[Prior Art Documents]] [[Patent Documents]]

[0004] [[Patent Document 1]] Japanese Unexamined Patent Publication No. Hei 7-37056 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0005] However, although Patent Document 1 discloses that each lesion is displayed in different display modes according to the feature amount, no consideration is given to displaying other lesions related to the lesion focused on by a user such as a physician. [[Means for Solving the Problem]]

[0007] An information processing apparatus according to the present invention includes: a detection result acquisition unit that acquires a result of detecting lesions from medical image data; and an instruction reception unit that receives an instruction to designate a lesion of interest based on the lesion detection result; For designated lesions of interest, a table is used that defines the type of lesion according to the relationship between the primary tumor and metastatic lesions, the relationship of complications, and the assessment of the risk of progression.Related to the lesion of interest and lesions of a different type from the lesion of interest The system includes a related lesion information acquisition unit that acquires related lesion information indicating related lesions, and a display control unit that displays the related lesion information on a display unit. The related lesion information acquisition unit further acquires information on the operating status, which is information regarding whether or not a detection process for detecting the related lesions indicated by the related lesion information acquired by the related lesion information acquisition unit is performed, or whether or not the detection process can be performed. The display control unit then displays the operating status information and the related lesion information on the display unit in association with each other. [Effects of the Invention]

[0008] According to the present invention, it is possible to efficiently identify other lesions related to a lesion that the user is interested in. [Brief explanation of the drawing]

[0009] [Figure 1] This diagram shows the configuration of the information processing system according to Embodiments 1 to 3. [Figure 2] This figure shows the hardware configuration of the information processing device according to Embodiments 1 to 3. [Figure 3] This figure shows the functional configuration of the information processing device according to Embodiment 1. [Figure 4] This figure shows an example of a user interface screen for an information processing device according to Embodiment 1. [Figure 5] This is a flowchart showing the processing of the information processing device according to Embodiment 1. [Figure 6] This diagram shows the functional configuration of the information processing device according to Embodiment 2. [Figure 7] This figure shows an example of a user interface screen for an information processing device according to Embodiment 2. [Figure 8] This is a flowchart showing the processing of the information processing device according to Embodiment 2. [Figure 9A] This figure shows an example of a user interface screen for an information processing device according to Embodiment 3. [Figure 9B]This figure shows an example of a user interface screen for an information processing device according to Embodiment 3. [Figure 10] This is a flowchart showing the processing of the information processing device according to Embodiment 3. [Modes for carrying out the invention]

[0010] The present invention will be described in detail below with reference to the attached drawings, based on its preferred embodiments. Unless otherwise specified, items described in other embodiments will be given the same numbering and their descriptions will be omitted. Furthermore, the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations. [Examples]

[0011] <Embodiment 1> Embodiment 1 describes an information processing device that displays medical image data, such as X-ray CT (Computed Tomography) image data and MRI (Magnetic Resonance Imaging) image data.

[0012] The information processing device of this embodiment includes a lesion detection unit (e.g., CADe) that detects lesions in medical image data, and displays the detection result to the user when a candidate lesion (hereinafter, "candidate lesion" will be referred to as "lesion") is detected. Furthermore, when a user such as a physician specifies the detected lesion, the device determines the type of lesion associated with that lesion and also displays the CADe's detection result for that associated lesion type. Examples of lesion types that CADe detects include pulmonary nodules, chest wall masses, peritoneal masses, liver masses, pancreatic masses, kidney masses, colorectal masses, reticular opacities, honeycomb lung, bronchiectasis, pleurisy, pleural effusion, tenosynovitis, bone erosion, osteitis, pancreatic hypertrophy, and pancreatic necrosis.

[0013] (System Configuration) Figure 1 is a diagram showing the configuration of an information processing system including the information processing device of this embodiment.

[0014] In FIG. 1, the information processing system includes a case database (hereinafter referred to as case DB) 102, an information processing apparatus 101, and a LAN (Local Area Network) 103.

[0015] The case DB 102 stores medical image data captured by a medical image capturing apparatus such as a CT apparatus. The case DB 102 further has a database function of providing medical image data to the information processing apparatus 101 via the LAN 103. Specifically, the case DB 102 of the present embodiment is a known PACS (Picture Archiving and Communication Systems).

[0016] (Hardware Configuration) FIG. 2 is a diagram showing the hardware configuration of the information processing apparatus according to the present embodiment.

[0017] In FIG. 2, the information processing apparatus 101 includes a storage medium 201, a ROM (Read Only Memory) 202, a CPU (Central Processing Unit) 203, and a RAM (Random Access Memory) 204. The information processing apparatus 101 further includes a LAN interface 205, an input interface 208, a display interface 206, and an internal bus 211.

[0018] The storage medium 201 is a storage medium such as an HDD (Hard Disk Drive) that stores the OS (Operating System), processing programs for performing various processes according to this embodiment, and various information. The ROM 202 stores programs for initializing the hardware and starting the OS, such as the BIOS (Basic Input Output System). The CPU 203 performs calculations when executing the BIOS, OS, and processing programs. The RAM 204 temporarily stores information when the CPU 203 executes programs. The LAN interface 205 is an interface for communication via LAN 103 that complies with standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.3ab. The display 207 is a display that shows the user interface screen, and the display interface 206 converts the screen information to be displayed on the display 207 into signals and outputs them to the display 207. The keyboard 209 is a keyboard for key input, and the mouse 210 is a mouse for specifying coordinate positions on the screen and inputting button operations, and the input interface 208 receives signals from the keyboard 209 and the mouse 210. The internal bus 211 transmits signals when communication occurs between blocks.

[0019] (Functional Configuration) Figure 3 is a diagram showing the functional configuration of the information processing device 101 in this embodiment.

[0020] In Figure 3, the information processing device 101 consists of an image data acquisition unit 311, a lesion detection unit 312, a detection result acquisition unit 313, an instruction reception unit 314, a related lesion information acquisition unit 315, and a display control unit 316.

[0021] In Figure 3, the case database 102 stores medical image data 321-i (i=1,2,3,...) and provides the medical image data 321-i (i=1,2,3,...) to the information processing device 101 via the LAN 103. The medical image data 321-i (i=1,2,3,...) is, for example, a DICOM file.

[0022] The image data acquisition unit 311 acquires medical image data 321-i (i=1,2,3,...) to be examined from the case database 102 via the LAN 103. In this embodiment, the acquisition of medical image data 321-i (i=1,2,3,...) follows DICOM (Digital Imaging and Communications in Medicine).

[0023] The lesion detection unit 312 detects multiple types of lesions from the acquired medical image data 321-i (i=1,2,3,...). For lesion detection, a detector trained with a Convolutional Neural Network (CNN), a type of deep learning, is used. For training the detector, a set of medical image data and ground truth data indicating the lesion region in the medical image data is used as training data. The training data medical image data is input to the CNN, and the parameters of the CNN are adjusted so that the error between the output value of the CNN and the data indicating the lesion region is small. The system may be configured to detect one lesion with one CNN, or to detect multiple lesions with one CNN. Here, a CNN is described as an example of a detector, but a detector based on machine learning or deep learning, or a detector that detects lesions using feature-based image processing technology, may also be used.

[0024] The detection result acquisition unit 313 acquires the lesion detection results from the lesion detection unit 312. Here, the number of lesions acquired by the detection result acquisition unit 313 may be one or more, depending on the number of lesions detected by the detector. The detection results acquired by the detection result acquisition unit 313 include information indicating the type of detected lesion and information that can identify its location. The information indicating the type of lesion is, for example, an ID (identifier) ​​uniquely assigned to each type of lesion. The information that can identify the location of the lesion is, for example, coordinate information or a mask image that can be displayed overlaid on medical image data. The information that can identify the location of the lesion may differ for each type of lesion.

[0025] The instruction receiving unit 314 receives a designation from the user for at least one lesion from the lesion detection results acquired by the detection result acquisition unit 313. The lesion detection results acquired by the detection result acquisition unit 313 are displayed in a list on the display unit 316. The instruction receiving unit 314 accepts instructions for designating a lesion from the user, such as by a left click operation of the mouse 210. The instruction receiving unit 314 may also accept instructions from the user to change the highlight position of the detection result using the arrow keys or TAB key on the keyboard 209, and to designate a lesion using a combination of the Enter key or space key.

[0026] The related lesion information acquisition unit 315 identifies related lesions that are related to the lesion of interest specified by the user in the instruction receiving unit 314, but are of a different type from the lesion of interest. The related lesion information acquisition unit 315, for example, maintains the relationship between the lesion type and the related lesion type as information in a table format, and identifies related lesions based on this table. If there are multiple lesions specified by the user in the instruction receiving unit 314, the unit takes the OR or AND operation of the related lesion types obtained from the table for each specified lesion type. Whether to use OR or AND may be specified in advance as setting information by the information processing device 101, selected at any time by the user via the instruction receiving unit 314, or selected by the information processing device 101 according to the combination of lesions specified. Alternatively, the unit may maintain information in a table format that includes combinations of multiple lesion types and their corresponding related lesion types, and use this information for identification. Furthermore, the related lesion information acquisition unit 315 may identify related lesions from combinations of multiple lesion types based on set rules. The tables and rules used by the related lesion information acquisition unit 315 to acquire related lesions may be created in advance based on medical knowledge. Medical knowledge includes, for example, the relationship between primary and metastatic lesions, the relationship between complications, and the definition of lesion types related to assessing the risk of severe complications. Regarding the relationship between primary and metastatic lesions, for example, for a lung nodule suspected of being primary lung cancer, related lesions include tumors in the chest wall, peritoneum, liver, and pancreas, which are considered possible sites of metastasis from primary lung cancer. Similarly, for a lung nodule suspected of being metastatic lung cancer, related lesions include tumors in the colon, kidney, and mammary gland, which are considered possible primary lesions. Furthermore, reticular opacities in the lung may be associated with rheumatoid arthritis, and to differentiate this, tenosynovitis, bone erosion, and osteitis are considered related lesions. In the case of pancreatitis, pancreatic hypertrophy and pancreatic necrosis are considered related lesions to assess the risk of severe complications.

[0027] The related lesion information acquisition unit 315 further acquires information regarding the operational status of the detection process for detecting a specified lesion. The operational status acquired by the related lesion information acquisition unit 315 is information indicating whether or not the lesion detection process (lesion detection process) by the lesion detection unit 312 has been performed for each type of lesion. The operational status acquired by the related lesion information acquisition unit 315 may also include whether or not the lesion detection process has been performed, i.e., the state in which the lesion detection process has been started but has not yet finished. By acquiring the operational status of the detection process for detecting a specified lesion, the related lesion information acquisition unit 315 can determine whether or not the specified lesion has been detected and whether or not a related lesion has been identified. For example, if the detection process for detecting a specified lesion is operational, it can be determined whether or not a related lesion has been detected as a result of the detection process. If the detection process for detecting a specified lesion is not operational, it can be determined that the detection process has not been performed. As will be described later, by also displaying the reason why the detection process is not operational, it is possible to introduce a detector that performs the detection process or to instruct the processing and editing of medical data in order to perform the detection process.

[0028] Furthermore, if the lesion detection unit 312 is performing lesion detection processing, the related lesion status acquisition unit 315 may also acquire the progress and remaining time of the detection processing. In addition, the related lesion status acquisition unit 315 may acquire the operational status for all lesion types, or it may acquire only the operational status of the detection processing for related lesions associated with the lesion specified by the user.

[0029] The display control unit 316 causes the related lesion information acquired by the related lesion information acquisition unit 315 to be displayed on the display unit. The related lesion information that the display control unit 316 displays on the display unit (display 207) is displayed in association with, for example, the detection result of the related lesion and the operating status of the detection process.

[0030] Furthermore, the display control unit 316 also displays the medical image data 321-i (i=1,2,3,...) acquired by the image data acquisition unit 311 and the detection results of related lesions acquired by the related lesion information acquisition unit 315 on the display unit (display 207). The display control unit 316 displays the detection results of related lesions that are related to the lesion specified by the user in a way that distinguishes them from the detection results of other lesions. In this embodiment, the display control unit 316 will be described in a way that separates the display areas for related lesions and other lesions, but for example, it may also be displayed by changing the color of the text or background, or by changing the icon image. In addition, the display control unit 316 will also display the operating status of the detection process in a way that distinguishes whether or not it is operating for related lesions. In this embodiment, the display control unit 316 will be described in a way that separates the display area in a way similar to the detection results, but it may also be displayed by changing the color of the text or background, or by changing the icon image.

[0031] (User interface screen) Figure 4 shows an example of a user interface screen displayed on the display unit by the display control unit 316 of the information processing device 101 in this embodiment. The user interface screen is displayed on the display unit (display 207), and various operations by the user are input via the keyboard 209 and mouse 210.

[0032] In Figure 4, the user interface screen 400 consists of a medical image data display area 401, a lesion detection result display area 402, and a related lesion detection result display area 403.

[0033] The display control unit 316 displays the medical image data acquired by the image data acquisition unit 311 in the medical image data display area 401. The medical image data display area 401 also changes the WL / WW (Window Level / Window Width), slice position, magnification, etc. of the image in response to operations by the keyboard 209 or mouse 210. The display control unit 316 also displays annotations 411 indicating the location of a specified lesion in the medical image data display area 401 based on the lesion detection results acquired by the detection result acquisition unit 313. The display control unit 316 may also display an overlay image in the medical image data display area 401 that highlights the lesion region corresponding to the location of the lesion.

[0034] The display control unit 316 causes the display unit to display the lesion detection results (lesion detection results) 421-i (i=1,2,3,4,...) acquired by the detection result acquisition unit 313 in the lesion detection result display area 402. The display unit (display 207) displays only the lesions detected as lesion detection results 421-i (i=1,2,3,4,...) in the lesion detection result display area 402. Here, lesion detection result 421-1 indicates that a lesion of "lesion type 1-1" has been detected. In addition, in the lesion detection result display area 402, the detection result of a lesion can be specified by left-clicking the mouse 210 on the lesion detection result. The display control unit 316 causes the display unit to highlight the specified detection result, for example, as shown in lesion detection result 421-2, by highlighting the border and background. The display control unit 316 also updates the annotation 411 based on the detection position of the corresponding lesion according to the specified lesion detection result. Furthermore, the display control unit 316 updates the contents of the related lesion detection result display area 403 based on the related lesion detection result obtained by the related lesion information acquisition unit 315.

[0035] The display control unit 316 displays the detection results of related lesions (related lesion detection results) 431-i (i=1,2,3,4,...) acquired by the related lesion information acquisition unit 315 in the related lesion detection result display area 403. The display control unit 316 displays the type of lesion of the related lesion as the related lesion detection result 431-i (i=1,2,3,4,...). In addition, the display control unit 316 displays in the related lesion detection result display area 403 that no related lesions were detected, as shown by related lesion detection result 431-1. The display control unit 316 also displays the operating status of the lesion detection process acquired by the related lesion information acquisition unit 315 in addition to the related lesion detection result 431-i (i=1,2,3,4,...). Related lesion detection result 431-3 indicates that lesion detection has not been performed as the operating status.

[0036] Related lesion detection results 431-1 and 432-2 are displayed, allowing the user to understand the detection results for related lesions that are related to the target lesion but of a different type from the target lesion. Additionally, related lesion detection result 431-3 is displayed, allowing the user to check the status, such as whether detection processing for related lesions has been performed.

[0037] (Processing flow) Figure 5 is a flowchart showing the processing of the information processing device 101 in this embodiment. The report generation process in the information processing device 101 is started after the information processing device is started, based on instructions from another system or user. When starting the process, the case to be processed is specified.

[0038] In step S501, the image data acquisition unit 311 acquires medical image data 321-i (i=1,2,3,...) of the case specified at startup from the case database 102 via the LAN 103, and proceeds to the next step.

[0039] In step S502, the lesion detection unit 312 detects lesions from the medical image data 321-i (i=1,2,3,...) acquired in step S501. Here, the lesion detection unit 312 performs detection processing using trained detectors generated based on machine learning or deep learning. The detectors to be detected may also be pre-configured. After performing the lesion detection processing on the medical image data, the lesion detection unit 312 proceeds to the next step.

[0040] In step S503, the display control unit 316 displays the medical image data 321-i (i=1,2,3,...) acquired by the image data acquisition unit 311 in step S501 in the medical image data display area 401 of the user interface screen 400. It also changes the WL / WW, slice position, magnification, etc. of the displayed image based on the operation of the keyboard 209 and mouse 210.

[0041] In step S504, the detection result acquisition unit 313 acquires the lesion detection result from the lesion detection unit 312. The lesion detection result includes information indicating the type of lesion detected and information indicating its position within the medical image data 321-i (i=1,2,3,...).

[0042] In step S505, the display control unit 316 displays the lesion detection result 421-i (i=1,2,3,4,...) in the lesion detection result display area 402 of the user interface screen 400, based on the lesion detection result acquired in step S504.

[0043] In step S506, the instruction receiving unit 314 receives an instruction from the user to specify a lesion to focus on, based on the input from the keyboard 209 or mouse 210. If a lesion is specified in step S506 (Yes in step S506), the process proceeds to step S511. If no lesion of interest to the user is detected by the instruction receiving unit 314 (No in step S506), the process proceeds to step S507.

[0044] In step S507, the OS (not shown) detects the end of processing by the information processing device 101. Processing termination can occur through OS shutdown, power-off, window closing, or process cessation. If processing termination is detected (Yes in step S507), processing is terminated; otherwise, if not detected (No in step S507), the process from step S503 is repeated.

[0045] In step S511, the related lesion information acquisition unit 315, based on the user's instruction to specify a lesion of interest in the lesion detection results detected in step S506, refers to a table defining the relationships between lesion types and identifies related lesions that are related to the lesion in question.

[0046] In step S512, the related lesion information acquisition unit 315 acquires the detection results for the related lesions identified in step S511 from the detection result acquisition unit 313.

[0047] In step S513, the display unit 316 displays the related lesion detection result 431-i (i=1,2,3,...) in the related lesion detection result display area 403 of the user interface screen 400, based on the related lesion detection result obtained in step S512 by the related lesion information acquisition unit 315.

[0048] In step S514, the related lesion information acquisition unit 315 acquires the operational status of the lesion detection process for the related lesion determined in step S511. In this embodiment, the operational status includes information indicating whether or not lesion detection is performed.

[0049] In step S515, the display control unit 316 displays the operating status of the related lesion detection in the related lesion detection result 431-i (i=1,2,3,...) in the related lesion detection result display area 403 of the user interface screen 400, based on the operating status of the related lesion detection acquired in step S514. After the processing in step S515 is completed, the process returns to step S507.

[0050] As described above, the information processing device 101 of the present invention includes a detection result acquisition unit 313 that acquires the results of detecting lesions from medical image data, an instruction reception unit 314 that receives instructions to designate a lesion of interest for the detected lesion, a related lesion information acquisition unit 315 that acquires information on related lesions that are related to the designated lesion of interest but of a different type from the lesion of interest, and a display control unit 316 that displays the related lesion information on a display unit such as a display 207. With this configuration, when a user designates a lesion of interest for the lesion detected by the lesion detection unit via the instruction reception unit 314, the related lesion information acquisition unit 315 automatically determines the type of lesion related to the lesion of interest for the user, and the display control unit 316 displays the detection results of the related lesions on the display unit. Therefore, it is possible to efficiently grasp other lesions related to the lesion of interest for the user, such as a doctor. Furthermore, even if the number of lesions to be detected increases, it becomes easier to find whether or not there are detection results for other related lesions. Furthermore, the display control unit 316 displays the operating status of the lesion detection process on the display unit, making it easier to distinguish whether the lesion was not detected because CADE was executed or because CADE was not executed.

[0051] (Modified form of Embodiment 1) The information processing device 101 may be an image processing workstation, an electronic medical record system, an integrated viewer that displays information from multiple types of devices, or a device that acquires medical images, such as an ultrasound diagnostic device.

[0052] Furthermore, the lesion detection unit 312 may be located on another device connected to the information processing device 101 via a network, such as an image processing server. The lesion detection unit 312 may also detect lesions at the time medical image data 321-i (i=1,2,3,...) is captured, at the time it is saved to the case database 102, or during other background processing, and store the detection results in a storage device such as the case database 102. In this case, the detection result acquisition unit 313 acquires the detection results from the storage device.

[0053] Furthermore, the lesion detection unit 312 may use methods other than CNN, such as SVM (Support Vector Machine), which is a type of machine learning, to detect lesions as a detector.

[0054] Furthermore, the detection result acquisition unit 313 may acquire other information related to the detection result, such as whether or not a lesion was detected, the location of the detected lesion, and, if the detection process is in progress, the progress of the process and the time remaining until the process is completed.

[0055] Furthermore, the related lesion information acquisition unit 315 may extract medical knowledge by processing past radiology reports, papers, and clinical guidelines using language processing, and create tables and rules to be used in determining related lesions.

[0056] <Embodiment 2> In addition to the information processing device of Embodiment 1, the information processing device 101 of this embodiment acquires and displays information regarding the feasibility of performing related lesion detection if the operation status of related lesion detection has not been performed. Furthermore, the information processing device 101 accepts an instruction to perform lesion detection if the detection process has not been performed and is feasible. Note that the system configuration of the information processing device of this embodiment is the same as that of Embodiment 1, which was described using Figure 1 and the hardware configuration using Figure 2, so a detailed explanation is omitted.

[0057] (Functional block) Figure 6 shows the functional configuration of the information processing device of this embodiment. Functional blocks identical to those described in Embodiment 1 using Figure 3 are given the same numbers, and their descriptions are omitted.

[0058] In Figure 6, the instruction receiving unit 614 in the information processing device 101 of this embodiment is characterized by receiving instructions to perform detection processing. The instruction receiving unit 614 receives further instructions from the user according to the operating status acquired by the related lesion information acquisition unit 315. In this embodiment, if the operating status regarding the detection processing of a predetermined lesion acquired by the related lesion information acquisition unit 315 is not performed, and the operating status regarding whether the detection processing can be performed is performed, the instruction receiving unit 614 receives an instruction from the user to the lesion detection unit 312 to perform the detection processing of the lesion. When the instruction receiving unit 614 receives an instruction from the user to perform additional lesion detection processing, it causes the lesion detection unit 312 to perform additional lesion detection processing. In this embodiment, the additional lesion detection processing is performed in response to the user's instructions at the instruction receiving unit 614, but the related lesion information acquisition unit 315 may perform the additional detection processing without user confirmation.

[0059] (User interface screen) Figure 7 shows an example of a user interface screen displayed by the display unit 316 in the information processing device 101 of this embodiment. Note that parts identical to the user interface of Embodiment 1 described using Figure 4 are given the same numbers, and their descriptions are omitted.

[0060] In addition to the user interface screen 400 described in Embodiment 1, the user interface screen 700 displays information regarding whether the detection process can be performed in the related lesion detection result 431-i (i=1,2,3,...) when the detection process is not being performed. The display unit 316 also displays a lesion detection instruction confirmation window 404, which is linked to the instruction reception unit 314.

[0061] The lesion detection instruction confirmation window 404 is a window for the instruction receiving unit 614 to receive instructions from the user to perform the lesion detection process. The display control unit 316 displays the lesion detection result (related lesion detection result 431-4 in Figure 7) on the display unit of the lesion detection instruction confirmation window 404, where the information regarding whether the detection process is performed is "not performed" and the operational status regarding whether the detection process can be performed is "can be performed". When the instruction receiving unit 614 receives an instruction from the user to specify "yes" in the lesion detection instruction confirmation window 404, it performs the lesion detection process, and when it receives an instruction to specify "no", it does not perform the lesion detection process. The specified instruction receiving unit 314 detects the user's specified instruction based on input from the mouse 210 or keyboard 209.

[0062] (Processing flow) Figure 8 shows the processing flow of the information processing device 101 of this embodiment. Steps identical to those in the processing flow of Embodiment 1 described using Figure 5 are given the same numbers, and their descriptions are omitted.

[0063] In step S516, the related lesion information acquisition unit 315 determines, via the instruction reception unit 614, whether the operation status of the detection process for the lesion detection result specified based on user operation is either not performed or can be performed. If the related lesion information acquisition unit 315 determines that the operation status is either not performed or can be performed, it proceeds to the process in step S521; otherwise, it returns to the process in step S507. In this embodiment, the operation information acquired by the related lesion information acquisition unit 315 includes information indicating whether detection has been performed and information indicating whether detection can be performed.

[0064] In step S521, the instruction receiving unit 614 receives an instruction to execute a detection process for a lesion that corresponds to the lesion detection result that the related lesion information acquisition unit 315 determined to be neither performable nor performable in step S516.

[0065] In this embodiment, when the instruction receiving unit 614 receives an instruction, the display unit 316 displays the lesion detection instruction confirmation window 404 shown in Figure 7 and receives confirmation instructions from the user. When the instruction receiving unit 614 receives an instruction from the user to select "yes" in response to the lesion detection instruction confirmation window 404, it instructs the lesion detection unit 312 to detect a lesion. When it detects an instruction to select "no", it does not issue an instruction to detect a lesion and terminates the processing of this step.

[0066] As described above, according to this embodiment, when a user specifies a lesion detected by the lesion detection unit 312, the related lesion information acquisition unit 315 automatically determines the type of related lesions related to the lesion of interest to the user, and the display unit 316 displays the detection results of the related lesions. Therefore, even when the number of lesions to be detected increases, it becomes easier for the user to find out whether or not there are detection results for other lesions related to the lesion of interest.

[0067] Furthermore, the display control unit 316 displays the operating status of the lesion detection process on the display unit, making it easier to distinguish whether the lesion was not detected because CADE was executed or because CADE was not executed.

[0068] Furthermore, if lesion detection has not yet been performed but is possible, the instruction receiving unit 614 will receive instructions from the user to perform further lesion detection processing, making it easier to perform the unperformed lesion detection processing.

[0069] (Modified version of Embodiment 2) In this embodiment, the instruction receiving unit 614 receives an instruction for lesion detection processing, and the lesion detection unit 312 performs additional lesion detection based on the user's instruction. However, even without receiving a user instruction from the instruction receiving unit 614, the related lesion information acquisition unit 315 may instruct the lesion detection unit 312 to perform any unexecuted detection processing.

[0070] <Embodiment 3> In this embodiment, the information processing device 101, in addition to the information processing device of Embodiment 2, has a display control unit 316 that displays information on the reason for the non-operation of related lesion detection on the display unit when the operation status of related lesion detection is not being performed. Furthermore, if the reason for non-operation is that the detection processing function has not been introduced, the related lesion information acquisition unit 315 receives an instruction to introduce the detection processing function via the instruction reception unit 614, and if the medical image data is outside the conditions for performing the detection processing, it receives an instruction to acquire medical image data that satisfies the conditions for performing the detection. Note that the system configuration including the information processing device 101 of this embodiment is the same as that of Embodiment 1, which was described using Figure 1, the hardware configuration is the same as that of Embodiment 1, which was described using Figure 2, and the functional blocks are the same as those of Embodiment 2, which was described using Figure 6, so their explanation is omitted.

[0071] (User interface screen) Figures 9A and 9B show examples of user interface screens displayed by the display unit 316 of the information processing device 101 in this embodiment. Note that parts identical to the user interfaces of Embodiments 1 and 2 described using Figure 4 are given the same numbers, and their descriptions are omitted.

[0072] The user interface screen 900 displayed on the display unit by the display control unit 316, in addition to the user interface screen 700 described in Embodiment 2, displays the reason if the operational status indicates that the operation is not possible, and the instruction receiving unit 614 receives additional instructions from the user. Furthermore, the user interface screen 900 displayed on the display unit by the display control unit 316 includes a lesion detection introduction instruction confirmation window 405 and an operation condition image acquisition instruction confirmation window 406.

[0073] Furthermore, in the user interface screen 900 that the display control unit 316 displays on the display unit, related lesion detection result 431-5 indicates that the reason for not performing the procedure is that the detection processing function is "not implemented," and related lesion detection result 431-6 indicates that the reason for not performing the procedure is that the medical image data targeted for lesion detection processing is "outside the application conditions" of the detection processing.

[0074] The lesion detection introduction instruction confirmation window 405, displayed on the display unit 207 or other display unit by the display control unit 316, is a window for the instruction reception unit 614 to confirm with the user the instruction to introduce lesion detection. The display control unit 316 displays the lesion detection introduction instruction confirmation window 405 on the display unit when the user specifies a lesion for which the detection processing function has not been introduced (related lesion detection result 431-5 in Figure 9A) as the reason for not performing the procedure. If the user specifies "yes" in the lesion detection introduction instruction confirmation window 405, the instruction reception unit 614 instructs the lesion detection unit 312 to introduce the lesion detection processing function, and does not instruct the user to introduce the function if the user specifies "no". The instruction reception unit 614 detects the user's specified instruction based on input from the mouse 210 or keyboard 209. Here, the instruction by the instruction reception unit 614 to introduce the detection processing function includes software installation and input of a key to activate the software.

[0075] The execution condition image acquisition instruction confirmation window 406, displayed on the display unit by the display control unit 316, is a window for the instruction reception unit 614 to confirm the instruction from the user to acquire medical image data that satisfies the execution conditions of the lesion detection process. The execution condition image acquisition instruction confirmation window 406 is displayed when the user specifies a lesion (related lesion detection result 431-6 in Figure 9B) for which the medical image data is outside the application conditions of the lesion detection process. If the user specifies "yes" in the execution condition image acquisition instruction confirmation window 406, the instruction reception unit 614 will have the image data acquisition unit 311 perform the acquisition process, and if the user specifies "no", it will not issue an instruction to acquire the image. The instruction reception unit 614 detects the user's specification based on input from the mouse 210 or keyboard 209. The execution condition image acquisition instruction confirmation window 406 also includes a user interface for adding or changing the specified conditions. Here, the application conditions for the lesion detection process include the imaging modality, reconstruction function, contrast conditions, time phase, imaging range, etc. Furthermore, instructions for acquisition include image reconstruction based on conditions, and ordering photography through the ordering system.

[0076] (Processing flow) Figure 10 shows the processing flow of the information processing device 101 of this embodiment. Steps identical to those in the processing flow of Embodiment 1 described using Figure 5 and the steps in Embodiment 2 described using Figure 8 are given the same numbers, and their descriptions are omitted.

[0077] In step S517, the related lesion information acquisition unit 315 determines whether the detection process for the specified lesion is operational or not, and whether the reason for non-operation is that the detection function has not been implemented. If the determination by the related lesion information acquisition unit 315 indicates that the detection process is not operational and the reason for non-operation is that the detection process function has not been implemented, the process proceeds to step S531. If the detection process is not operational and the reason for non-operation is not that the function has not been implemented, the process proceeds to step S517. Here, the operational information acquired by the related lesion information acquisition unit 315 in this embodiment includes information indicating whether detection is performed, information indicating whether detection can be performed, and information indicating the reason for non-operation.

[0078] In step S531, the instruction receiving unit 614 instructs the lesion detection unit 312 to implement the lesion detection function as instructed by the user. In this embodiment, when the instruction receiving unit 614 implements the instruction for the lesion detection unit 312 to implement the detection function, the display control unit 316 displays the lesion detection implementation instruction confirmation window 405 shown in Figure 9A on a display unit such as the display 207, and receives an instruction from the user to input confirmation. If the instruction receiving unit 614 detects an instruction of "yes" in the lesion detection implementation instruction confirmation window 405, it instructs the lesion detection unit 312 to implement the lesion detection processing function. If it detects an instruction of "no", it does not instruct the lesion detection processing function to be implemented and terminates the processing of this step.

[0079] In step S518, the related lesion information acquisition unit 315 determines whether the detection process for the specified lesion is operational or not based on user operation, and whether the reason for non-operation is that the conditions for performing the lesion detection process are not met. If the related lesion information acquisition unit 315 determines that the lesion detection process is not operational and the reason for non-operation is that the conditions for performing the lesion detection process are not met, it proceeds to step S541. If the lesion detection process is not operational and the reason for non-operation is not that the conditions for performing the lesion detection process are not met, it returns to step S507.

[0080] In step S541, if the related lesion information acquisition unit 315 determines in step S517 that the procedure is not to be performed and the reason for not being to be performed is that it does not meet the conditions for performing the procedure, it instructs the image data acquisition unit 311 to acquire medical image data that satisfies the conditions for performing the lesion detection corresponding to the lesion detection result. In this embodiment, the display control unit 316 displays the implementation condition image acquisition instruction confirmation window 406 shown in Figure 9B on a display unit such as the display 207, and obtains input from the user to confirm the performance of the process of acquiring medical image data that satisfies the conditions for performing the procedure. If the instruction reception unit 614 detects a "yes" designation in the implementation condition image acquisition instruction confirmation window 406, it instructs the image data acquisition unit 311 to acquire medical image data that satisfies the conditions for performing the procedure. If it detects a "no" designation, it does not instruct the acquisition of medical image data and terminates the processing of this step. The image data acquisition unit 311 may generate the medical image data that satisfies the conditions for performing the procedure from already acquired medical image data, acquire it from an external source, or send an order for acquisition to an ordering system, etc.

[0081] As described above, according to this embodiment, when a user specifies a lesion detected by the lesion detection unit 312, the related lesion information acquisition unit 315 automatically determines the type of related lesion, and the display unit 316 displays the detection results of related lesions related to the lesion of interest to the user. Therefore, even if the number of lesions to be detected increases, it becomes easier for the user to find out whether or not there are detection results for other lesions related to the lesion of interest.

[0082] Furthermore, the display control unit 316 displays the operating status of the lesion detection process on the display unit, making it easier to distinguish whether the lesion was not detected because CADE was executed or because CADE was not executed.

[0083] Furthermore, the related lesion information acquisition unit 315 instructs the lesion detection process to be performed if the lesion detection process has not yet been performed but is possible, thus facilitating the performance of unperformed lesion detection.

[0084] Furthermore, the related lesion information acquisition unit 315 will instruct the introduction of the lesion detection process if the lesion detection process is not performed and the reason for not performing it is that the detection process function has not been introduced, thus making it easier to introduce the necessary lesion detection.

[0085] Furthermore, if the acquisition of related lesion information 315 is not performed and the reason for not performing it is that the medical image data does not meet the performance conditions, it will instruct the acquisition of medical image data that meets the performance conditions, making it easier to acquire the necessary medical image data.

[0086] (Other examples) Furthermore, the present invention can also be realized by performing the following process: that is, supplying software (program) that realizes the functions of the above-described embodiment to a system or device via a network or various storage media, and having the computer (or CPU or MPU, etc.) of that system or device read and execute the program.

Claims

1. A detection result acquisition unit that acquires the results of detecting lesions from medical image data, Based on the detection results, an instruction receiving unit receives instructions for designating a lesion of interest, A related lesion information acquisition unit acquires related lesion information indicating related lesions that are related to the designated target lesion and are of a different type from the target lesion, using a table that defines the types of lesions according to the relationship between the primary lesion and metastatic lesions, the relationship of complications, and the relationship for evaluating the risk of severe complications for the designated target lesion. It has a display control unit that displays information about the related lesions on the display unit, The related lesion information acquisition unit further acquires information regarding the operating status, which is information regarding whether or not a detection process for detecting the related lesion indicated by the related lesion information acquired by the related lesion information acquisition unit is performed, or whether or not the detection process can be performed. The information processing apparatus is characterized in that the display control unit displays information relating to the operating status and information relating to the associated lesions on the display unit.

2. The information processing apparatus according to claim 1, characterized in that the display control unit displays a message to receive an instruction to perform the detection process when the detection process is feasible.

3. A detection result acquisition step that obtains the result of detecting a lesion from medical image data, Based on the detection results, an instruction reception step is performed to receive instructions for designating a lesion of interest, A related lesion information acquisition step involves obtaining related lesion information, which indicates related lesions that are related to the designated lesion of interest and of a different type from the lesion of interest, using a table that defines the types of lesions according to the relationship between the primary lesion and metastatic lesions, the relationship of complications, and the relationship for evaluating the risk of severe complications, for the designated lesion of interest; The system includes a display control step that causes the information of the related lesions to be displayed on the display unit, In the step of acquiring related lesion information, further information is acquired regarding the operational status, which is information regarding whether or not a detection process for detecting the related lesion indicated by the related lesion information acquired in the step is performed, or whether or not the detection process can be performed. The information processing method is characterized in that, in the display control step, the information relating to the operating state and the information relating to the related lesion are displayed on the display unit in association.

4. A program for executing the information processing method described in claim 3 using a computer.

Images