Information processing apparatus, information processing method, and non-transitory storage medium

Abstract

This invention provides an information processing apparatus, an information processing method, and a non-transitory storage medium. According to one aspect of the invention, the information processing apparatus includes: a detection result acquisition unit configured to acquire results of detecting lesions from medical image data; an instruction receiving unit configured to receive an instruction for specifying a lesion of interest based on the detection results; a related lesion information acquisition unit configured to acquire information about related lesions associated with the specified lesion of interest, wherein the related lesions are of a different type than the lesion of interest; and a display control unit configured to control a display unit to display information about the related lesions.

Description

Technical Field

[0001] This disclosure relates to an information processing apparatus, an information processing method, and a non-transitory storage medium for displaying information about other lesions related to a lesion of interest to a user. Background Technology

[0002] Computer-aided detection (CADe) is known to use computers to analyze medical images to detect candidate lesions that are abnormalities associated with diseases. With advancements in artificial intelligence (AI) technology, the number of lesions of interest being detected is currently increasing.

[0003] The medical diagnostic support device discussed in Japanese Patent Application Publication No. 7-37056 divides medical image data into microregions, calculates the characteristic quantities of each microregion, and overlays image data with different display concentrations or display colors onto the medical image data to display the medical image data based on the characteristic quantities.

[0004] However, the technology discussed in Japanese Patent Application Publication No. 7-37056 discusses the use of different display forms for each lesion part according to the characteristic quantity, but does not consider displaying other lesions related to the lesions of concern to users such as doctors. Summary of the Invention

[0005] According to one aspect of the present invention, an information processing apparatus is provided, comprising: a detection result acquisition unit configured to acquire the result of detecting a lesion from medical image data; an instruction receiving unit configured to receive an instruction for specifying a lesion of interest based on the detection result; a related lesion information acquisition unit configured to acquire information about related lesions associated with the specified lesion of interest, wherein the related lesions are of a different type than the lesion of interest; and a display control unit configured to control a display unit to display information about the related lesions.

[0006] Other features of the invention will become clear from the following description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0007] Figure 1 The construction of an information processing system according to the first exemplary embodiment to the third exemplary embodiment is shown.

[0008] Figure 2 The hardware structure of an information processing apparatus according to the first exemplary embodiment to the third exemplary embodiment is shown.

[0009] Figure 3 The functional structure of an information processing apparatus according to a first exemplary embodiment is shown.

[0010] Figure 4An example of a user interface screen of an information processing apparatus according to a first exemplary embodiment is shown.

[0011] Figure 5 This is a flowchart illustrating the processing of an information processing apparatus according to a first exemplary embodiment.

[0012] Figure 6 The functional structure of an information processing apparatus according to a second exemplary embodiment is shown.

[0013] Figure 7 An example of a user interface screen of an information processing apparatus according to a second exemplary embodiment is shown.

[0014] Figure 8 This is a flowchart illustrating the processing of an information processing apparatus according to a second exemplary embodiment.

[0015] Figure 9A An example of a user interface screen of an information processing apparatus according to a third exemplary embodiment is shown.

[0016] Figure 9B Another example of a user interface screen of an information processing apparatus according to a third exemplary embodiment is shown.

[0017] Figure 10 This is a flowchart illustrating the processing of an information processing apparatus according to a third exemplary embodiment. Detailed Implementation

[0018] The present invention will now be described with reference to the accompanying drawings based on exemplary embodiments. Items described in other exemplary embodiments are assigned the same reference numerals, and redundant descriptions thereof will be omitted unless otherwise stated. The constructions described in the following exemplary embodiments should be considered illustrative, and the invention is not limited to the constructions shown. The various embodiments of the invention described below can be implemented individually, or, where necessary or where a combination of elements or features from various embodiments is advantageous in a single embodiment, can be implemented as a combination of multiple embodiments or features thereof.

[0019] The first exemplary embodiment will now be described with regard to an information processing apparatus that displays medical image data (such as X-ray computed tomography (CT) image data and magnetic resonance imaging (MRI) image data).

[0020] According to a first exemplary embodiment, the information processing apparatus includes a lesion detection unit, such as computer-aided detection (CADe), for detecting lesions in medical image data. When the lesion detection unit detects a candidate lesion (hereinafter referred to as a lesion candidate), the information processing apparatus displays the detection results to a user. When a user, such as a doctor, specifies a detected lesion, the information processing apparatus determines the type of lesion associated with that lesion and displays the CADE detection results for the relevant lesion type. Examples of lesion types to be detected by CADE include: pulmonary nodules, chest wall masses, peritoneal masses, liver masses, pancreatic masses, kidney masses, colonic masses, reticular formation, honeycomb lung, bronchiectasis, pleurisy, pleural effusion, tenosynovitis, bone erosion, osteitis, pancreatic hypertrophy, and pancreatic necrosis.

[0021] (System Structure)

[0022] Figure 1 The construction of an information processing system including an information processing apparatus according to this exemplary embodiment is shown.

[0023] Reference Figure 1 The information processing system includes a medical case database (hereinafter referred to as medical case DB) 102, an information processing device 101, and a local area network (LAN) 103.

[0024] The medical case database 102 stores medical image data captured by a medical imaging device such as a computed tomography (CT) scanner. The medical case database 102 also includes database functionality for providing the medical image data to the information processing device 101 via a LAN 103. More specifically, the medical case database 102 according to this exemplary embodiment is a known Picture Archiving and Communication System (PACS).

[0025] (Hardware Structure)

[0026] Figure 2 The hardware configuration of the information processing apparatus 101 according to this exemplary embodiment is shown.

[0027] Reference Figure 2 The information processing device 101 includes a storage medium 201, a read-only memory (ROM) 202, a central processing unit (CPU) 203, and a random access memory (RAM) 204.

[0028] The information processing device 101 also includes a LAN interface 205, an input interface 208, a display interface 206, and an internal bus 211.

[0029] Storage medium 201 is, for example, a hard disk drive (HDD), which stores the operating system (OS), various information, and processing programs for performing various processes according to this exemplary embodiment. ROM 202 stores programs such as the Basic Input / Output System (BIOS) for initializing hardware and starting the OS. CPU 203 performs computational processing while executing the BIOS, OS, and processing programs. RAM 204 temporarily stores information used when CPU 203 executes programs. An IEEE 802.3ab compliant LAN interface 205 enables information processing device 101 to communicate via LAN 103. Display 207 displays a user interface screen. Display interface 206 converts screen information to be displayed on display 207 into signals and outputs the signals to display 207. Keyboard 209 performs key input. Mouse 210 specifies coordinate positions on the screen and inputs button operations. Input interface 208 receives signals from keyboard 209 and mouse 210. Internal bus 211 transmits signals in inter-block communication.

[0030] (Functional Structure)

[0031] Figure 3 The functional configuration of the information processing apparatus 101 according to this exemplary embodiment is shown.

[0032] Reference Figure 3 The information processing device 101 includes an image data acquisition unit 311, a lesion detection unit 312, a detection result acquisition unit 313, an instruction receiving unit 314, a related lesion information acquisition unit 315, and a display control unit 316.

[0033] Reference Figure 3 The medical case database 102 stores medical image data 321-i (i = 1, 2, 3, ...) and provides the medical image data 321-i (i = 1, 2, 3, ...) to the information processing device 101 via LAN 103. The medical image data 321-i (i = 1, 2, 3, ...) is, for example, a medical digital imaging and communication (DICOM) file.

[0034] Image data acquisition unit 311 acquires medical image data 321-i (i = 1, 2, 3, ...) to be examined from medical case DB 102 via LAN 103. According to this exemplary embodiment, the acquisition of medical image data 321-i (i = 1, 2, 3, ...) conforms to DICOM.

[0035] The lesion detection unit 312 detects multiple types of lesions from acquired medical image data 321-i (i = 1, 2, 3, ...). The detector is used for lesion detection, and a convolutional neural network (CNN), as one of the deep learning models, has been trained. During the detector's training, a set of medical image data and ground truth data indicating lesion regions in the medical image data are used as teacher data. The medical image data of the teacher data is input into the CNN. The CNN parameters are adjusted to minimize the difference between the CNN's output value and the data indicating lesion regions. The lesion detection unit 312 can be configured to detect one lesion using a CNN, or to detect multiple lesions using a CNN. The above description of a CNN as an example of a detector can be a detector based on machine learning and deep learning, or it can be a detector that detects lesions using image processing techniques based on feature quantities.

[0036] The detection result acquisition unit 313 acquires the results of lesion detection performed by the lesion detection unit 312. Depending on the number of lesions detected by the detector, the detection result acquisition unit 313 can acquire one or more lesions. The detection results acquired by the detection result acquisition unit 313 contain information about the type of the detected lesion and information about the location of the lesion. Information about the type of the detected lesion may include a unique identifier (ID) assigned to each lesion type. Information about the location of the lesion may be, for example, a mask image that can be displayed by overlaying coordinate information and medical image data. The location information for identifying the lesion may differ for each lesion type.

[0037] The instruction receiving unit 314 receives from the user a designation of at least one lesion from the lesion detection results acquired by the detection result acquisition unit 313. The lesion detection results acquired by the detection result acquisition unit 313 are displayed in a list on the display control unit 316. The instruction receiving unit 314 receives the lesion designation instruction via a left-click operation of the mouse 210 by the user. The instruction receiving unit 314 can receive user instructions for changing the highlighted position in the detection results via the arrow keys and TAB key on the keyboard 209, and can receive designation instructions via a combination of the ENTER key and the space bar.

[0038] The related lesion information acquisition unit 315 identifies related lesions as those associated with the lesions of interest specified by the user via the instruction receiving unit 314, and these lesions are of a different type than the lesions of interest. For example, the related lesion information acquisition unit 315 maintains the relationship between lesion types and related lesion types in a tabular format and identifies related lesions based on this table. When the user specifies multiple lesions, the instruction receiving unit 314 performs an OR (or operation) or AND (and operation) on the related lesion types obtained from the table for each specified lesion type. The use of OR or AND can be pre-selected by the information processing device 101 as setting information, can be selected by the user via the instruction receiving unit 314 as required, or can be selected by the information processing device 101 based on the combination of specified lesions. The related lesion information acquisition unit 315 can maintain the combination of multiple lesion types and corresponding related lesion types in a tabular format and identify related lesions using this information. The related lesion information acquisition unit 315 can also identify related lesions from combinations of multiple lesion types based on set rules. The table and rules used by the related lesion information acquisition unit 315 to acquire related lesions can be pre-generated based on medical knowledge. Examples of medical knowledge include the relationship between primary and metastatic lesions, the relationship between complications, and the relationship between the types of lesions that define the severity risk assessment. Examples of the relationship between primary and metastatic lesions are described below. For a lung nodule suspected to be primary lung cancer, relevant lesions include tumor masses in the chest wall, peritoneum, liver, and pancreas that are considered metastatic sites of primary lung cancer. For a lung nodule suspected to be metastatic lung cancer, relevant lesions include tumor masses in the large intestine, kidney, and breast that are considered primary sites. For pulmonary reticular formation that may be a complication of rheumatoid arthritis, relevant lesions for differentiation include tenosynovitis, bone erosion, and osteitis. For pancreatitis, relevant lesions for determining severity risk include pancreatic hypertrophy and pancreatic necrosis.

[0039] The relevant lesion information acquisition unit 315 acquires information about the operational status of the detection process used to detect a specified lesion. The operational status acquired by the relevant lesion information acquisition unit 315 is information about whether the lesion detection unit 312 has performed lesion detection processing on each lesion type. The operational status acquired by the relevant lesion information acquisition unit 315 can include the implementation of lesion detection processing, i.e., the state where lesion detection processing has started but not yet ended. The relevant lesion information acquisition unit 315 acquires the operational status of the detection process used to detect a specified lesion, detects the specified lesion, and determines whether the relevant lesion has been identified. For example, if the detection process used to detect a specified lesion is in operation, the relevant lesion information acquisition unit 315 can determine whether the relevant lesion has been detected as a detection processing result. If the detection process used to detect a specified lesion is not in operation, the relevant lesion information acquisition unit 315 can determine that no detection processing has been performed. Since the reason for the detection processing not being performed is displayed (as described below), the user can introduce a detector to perform the detection processing and issue instructions to process and edit medical data for detection processing.

[0040] When the lesion detection unit 312 is currently performing lesion detection processing, the related lesion information acquisition unit 315 can simultaneously acquire the progress and remaining time of the detection processing. The related lesion information acquisition unit 315 can acquire the operation status of all lesion types, or it can acquire only the operation status of the detection processing of related lesions specified by the user. The related lesion information acquisition unit 315 can acquire the results of the related lesions. In this case, the display control unit 316 (described below) controls the display unit to display the detection results of the related lesions.

[0041] The display control unit 316 controls the display unit to display information about the relevant lesions acquired by the relevant lesion information acquisition unit 315. For example, the display control unit 316 controls the display unit (display 207) to display information about the relevant lesions, for example, by displaying the results of the relevant lesion detection and the operation status of the detection process in a correlated manner.

[0042] The display control unit 316 also displays on the display unit (display 207) the medical image data 321-i (i = 1, 2, 3, ...) acquired by the image data acquisition unit 311 and the results of related lesion detection acquired by the related lesion information acquisition unit 315. The display control unit 316 displays the detection results of related lesions associated with lesions specified by the user, so as to distinguish them from the detection results of other lesions. This exemplary embodiment will now be described primarily in the form of the display control unit 316 displaying related lesions and other lesions in different display areas. For example, different characters, different background colors, or different icon images can be used to display these lesions. The display control unit 316 controls the display unit to display the operation state of the detection process, making it possible to distinguish the operation state of related lesions. The display control unit 316 can also display medical image data, detection results, and results of related lesions in a recognizable manner. Similar to the detection results, this exemplary embodiment will now be described primarily in the form of the display control unit 316 controlling the display unit to display in different display areas. However, for example, different characters, different background colors, or different icon images can be used for display.

[0043] (User interface screen)

[0044] Figure 4 An example of a user interface screen displayed on a display unit by the display control unit 316 of the information processing apparatus 101 according to this exemplary embodiment is shown. The user interface screen is displayed on the display unit (display 207), and various user operations are input via the keyboard 209 and the mouse 210.

[0045] Reference Figure 4 The user interface screen 400 includes a medical image data display area 401, a lesion detection result display area 402, and a related lesion detection result display area 403.

[0046] The display control unit 316 controls the display unit to display medical image data acquired by the image data acquisition unit 311 in the medical image data display area 401. In the medical image data display area 401, the user changes the window level / width (WL / WW), slice position, and magnification of the image in response to operations via the keyboard 209 and mouse 210. The display control unit 316 also controls the display unit to display annotations 411 indicating the location of specified lesions in the medical image data display area 401 based on lesion detection results acquired by the detection result acquisition unit 313. The display control unit 316 can also display overlapping images in the medical image data display area 401, highlighting the lesion area corresponding to the lesion location.

[0047] The display control unit 316 controls the display unit to display the lesion detection results (lesion detection results) 421-i (i = 1, 2, 3, 4, ...) acquired by the detection result acquisition unit 313 in the lesion detection result display area 402. The display unit (display 207) only displays the lesions detected as lesion detection results 421-i (i = 1, 2, 3, 4, ...) in the lesion detection result display area 402. The lesion detection result 421-1 indicates that the lesion "lesion type 1-1" has been detected. In the lesion detection result display area 402, the user can specify the lesion detection result by left-clicking the lesion detection result with the mouse 210. For example, the display control unit 316 controls the display unit to highlight the outline and background of the specified detection result, as shown in lesion detection result 421-2. The display control unit 316 also updates the annotation 411 based on the corresponding lesion detection position and the specified lesion detection result. The display control unit 316 also updates the content of the relevant lesion detection result display area 403 based on the relevant lesion detection results acquired by the relevant lesion information acquisition unit 315.

[0048] The display control unit 316 controls the display unit to display the results of relevant lesion detection (related lesion detection results) 431-i (i = 1, 2, 3, 4, ...) acquired by the relevant lesion information acquisition unit 315 in the relevant lesion detection result display area 403. The display control unit 316 displays the lesion type of the relevant lesion as the relevant lesion detection result 431-i (i = 1, 2, 3, 4, ...). When no relevant lesion is detected, the display control unit 316 controls the display unit to display "No relevant lesion detected" in the relevant lesion detection result display area 403, as shown in the relevant lesion detection result 431-1. The display control unit 316 also controls the display unit to display the operation status of the lesion detection processing acquired by the relevant lesion information acquisition unit 315 in the relevant lesion detection results 431-i (i = 1, 2, 3, 4, ...). The relevant lesion detection result 431-3 indicates that no lesion detection was performed as the operation status.

[0049] The display of related lesion detection results 431-1 and 431-2 allows the user to understand the detection results of related lesions that are related to the lesion of interest to the user, even though the related lesions are of a different type than the lesion of interest. The display of related lesion detection result 431-3 allows the user to confirm that the treatment used to detect the related lesion was not implemented.

[0050] (Processing flow)

[0051] Figure 5This is a flowchart illustrating the processing of the information processing apparatus 101 according to this exemplary embodiment. After the information processing apparatus 101 is started, it begins report generation processing based on instructions from other systems or users. When processing begins, the medical case to be processed is specified.

[0052] In step S501, the image data acquisition unit 311 acquires medical image data 321-i (i = 1, 2, 3, ...) of the medical case specified at startup from the medical case DB 102 via LAN 103. Then, the process proceeds to the next step.

[0053] In step S502, the lesion detection unit 312 detects lesions from the medical image data 321-i (i = 1, 2, 3, ...) acquired in step S501. The lesion detection unit 312 performs detection processing using a detector that has been learned and generated based on machine learning and deep learning. The detector used for detection processing can be preset. When the lesion detection unit 312 performs lesion detection processing on the medical image data, the processing proceeds to the next step.

[0054] In step S503, the display control unit 316 controls the display unit to display the medical image data 321-i (i = 1, 2, 3, ...) acquired by the image data acquisition unit 311 in step S501 in the medical image data display area 401 of the user interface screen 400. The user responds to operations via the keyboard 209 and mouse 210 to change the WL / WW, slice position, and magnification of the image to be displayed.

[0055] In step S504, the detection result acquisition unit 313 acquires the lesion detection result from the lesion detection unit 312. The lesion detection result includes information about the type of lesion detected and information about the location of the lesion in the medical image data 321-i (i = 1, 2, 3, ...).

[0056] In step S505, the display control unit 316 controls the display unit to display the lesion detection result 421-i (i = 1, 2, 3, 4, ...) in the lesion detection result display area 402 of the user interface screen 400 based on the lesion detection result obtained in step S504.

[0057] In step S506, the instruction receiving unit 314 receives an instruction to designate a lesion of interest for the user based on input from the keyboard 209 and the mouse 210. When a designation of a lesion of interest for the user is detected ("Yes" in step S506), the process proceeds to step S511. On the other hand, when no designation of a lesion of interest for the user is detected ("No" in step S506), the process proceeds to step S507.

[0058] In step S507, the OS (not shown) detects the end of the information processing device 101's processing. The end processing includes OS shutdown, power-off, window closing, and processing deactivation. When end processing is detected ("Yes" in step S507), the processing exits the flowchart. Conversely, when no end processing is detected ("No" in step S507), the processing returns to step S503. Then, the information processing device 101 repeats the processing.

[0059] In step S511, the relevant lesion information acquisition unit 315, based on the user's instruction to specify the lesion of interest in the detected lesion detection results in step S506, identifies the related lesions as lesions associated with the lesion by referring to a table defining the relationship between lesion types.

[0060] In step S512, the relevant lesion information acquisition unit 315 acquires the detection results of the relevant lesions identified in step S511 from the detection result acquisition unit 313.

[0061] In step S513, the display control unit 316 displays the relevant lesion detection results 431-i (i = 1, 2, 3, ...) in the relevant lesion detection result display area 403 of the user interface screen 400 based on the relevant lesion detection results obtained by the relevant lesion information acquisition unit 315 in step S512.

[0062] In step S514, the relevant lesion information acquisition unit 315 acquires the operation status of the relevant lesion detection and processing determined in step S511. According to this exemplary embodiment, the operation status includes information regarding the implementation of the lesion detection and processing.

[0063] In step S515, the display control unit 316, based on the operation status of the relevant lesion detection obtained in step S514, controls the display unit to display the operation status in the relevant lesion detection results 431-i (i = 1, 2, 3, ...) in the relevant lesion detection result display area 403 of the user interface screen 400. When the processing in step S515 is completed, the processing returns to step S507.

[0064] As described above, the information processing apparatus 101 of an exemplary embodiment of the present invention includes a display control unit 316, which controls a display unit to display the results of detecting at least one lesion from medical image data and information on related lesions associated with a lesion of interest selected for the at least one lesion in response to a user's instruction. The information processing apparatus 101 of an exemplary embodiment of the present invention includes: a detection result acquisition unit 313, which acquires the results of detecting a lesion from medical image data; an instruction receiving unit 314, which receives an instruction to designate a lesion of interest for the detected lesion; a related lesion information acquisition unit 315, which acquires information on related lesions that are associated with the designated lesion of interest, and these related lesions are of a different type than the lesion of interest; and a display control unit 316, which controls a display unit, such as a display 207, to display the information on the related lesions. In this configuration, when a user designates a lesion of interest for a lesion detected by the lesion detection unit 312 via the instruction receiving unit 314, the related lesion information acquisition unit 315 automatically determines the type of lesion associated with the user's lesion of interest, and the display control unit 316 displays the results of the related lesion detection on the display unit. This enables efficient monitoring of other lesions associated with a user's lesion of interest, such as a doctor's. Even as the number of lesions of interest increases, users can easily identify whether other related lesions are detected. Furthermore, the display control unit 316 controls the display unit to show the operational status of the lesion detection processing. This configuration makes it easier to distinguish between a state where no lesions were detected during CADE and a state where no lesions were detected without CADE.

[0065] (Variations of the first exemplary embodiment)

[0066] The information processing device 101 may be an image processing workstation, a comprehensive viewer for displaying electronic charts and information from various types of devices, and a device for capturing medical images (such as an ultrasound diagnostic device).

[0067] The lesion detection unit 312 may reside on other devices (such as an image processing server connected to the information processing device 101 via a network). The lesion detection unit 312 may detect lesions while capturing medical image data 321-i (i = 1, 2, 3, ...), while storing the data in the medical case database 102, or during other background processing, and store the detection results in a storage device such as the medical case database 102. In this case, the detection result acquisition unit 313 acquires the detection results from the storage device.

[0068] The lesion detection unit 312, which acts as a detector, can detect lesions by using methods other than CNNs, such as support vector machines (SVM) as a type of machine learning technique.

[0069] The detection result acquisition unit 313 can acquire whether a lesion has been detected, the location of the detected lesion, and other information related to the detection result (such as the progress of processing and the remaining time before the end of processing).

[0070] The relevant lesion information acquisition unit 315 also extracts medical knowledge by performing language processing on past X-ray image interpretation reports, papers and medical care guidelines to generate tables and rules for identifying relevant lesions.

[0071] The information processing apparatus 101 according to the second exemplary embodiment includes the information processing apparatus 101 according to the first exemplary embodiment, and further includes the following functions: when the operation status of the relevant lesion detection is "not implemented", acquiring and displaying information about the feasibility of the detection process. When the detection process is not implemented and is feasible, the information processing apparatus 101 receives an instruction to implement the lesion detection. In this exemplary embodiment, the system configuration of the information processing apparatus 101 is the same as described above. Figure 1 The system configuration described according to the first exemplary embodiment is similar, and the hardware configuration is the same as described above. Figure 2 The hardware construction described according to the first exemplary embodiment is similar, and its redundant description will be omitted.

[0072] (Function block)

[0073] Figure 6 The functional configuration of the information processing apparatus 101 according to this exemplary embodiment is shown. (Referring to the above references) Figure 3 Functional blocks described according to the first exemplary embodiment are assigned the same reference numerals, and their redundant descriptions will be omitted.

[0074] Reference Figure 6 According to this exemplary embodiment, the instruction receiving unit 614 of the information processing apparatus 101 receives an instruction for performing detection processing. The instruction receiving unit 614 receives additional instructions from the user based on the operation status acquired by the relevant lesion information acquisition unit 315. According to this exemplary embodiment, when the operation status of the processing for detecting a predetermined lesion acquired by the relevant lesion information acquisition unit 315 is "not implemented" and the operation status of the feasibility of the detection processing is "feasible," the instruction receiving unit 614 receives an instruction from the user for processing of the lesion detected by the lesion detection unit 312. Upon receiving an instruction from the user for performing additional lesion detection processing, the instruction receiving unit 614 instructs the lesion detection unit 312 to perform the additional lesion detection processing. According to this exemplary embodiment, the additional lesion detection processing is performed when the instruction receiving unit 614 receives a user instruction. However, the relevant lesion information acquisition unit 315 may perform additional detection processing without receiving confirmation input from the user.

[0075] (User interface screen)

[0076] Figure 7 An example of a user interface screen displayed by the display control unit 316 in the information processing apparatus 101 according to this exemplary embodiment is shown. (Referring to the above...) Figure 4 The same elements of the user interface described according to the first exemplary embodiment are assigned the same reference numerals, and redundant descriptions thereof will be omitted.

[0077] The user interface screen 700 displays information from the user interface screen 400 according to the first exemplary embodiment, and when the operation status of the detection process is "not implemented," information regarding the feasibility of the detection process is also displayed in the relevant lesion detection results 431-i (i = 1, 2, 3, ...). The display control unit 316 displays the lesion detection instruction check window 404 associated with the instruction receiving unit 614.

[0078] The instruction receiving unit 614 uses the lesion detection instruction check window 404 to receive instructions from the user for implementing lesion detection processing. The display control unit 316 controls the display unit to display the lesion detection instruction check window 404, and the lesion detection results when the operation status of the detection processing related to the implementation of the detection processing is "not implemented" and the operation status of the detection processing related to the feasibility of the detection processing is "feasible". Figure 7 The relevant lesion detection results (431-4) are processed. When the instruction receiving unit 614 receives a user instruction specifying "yes" in the lesion detection instruction check window 404, lesion detection processing is performed. On the other hand, when the instruction receiving unit 614 receives a user instruction specifying "no", lesion detection processing is not performed. The instruction receiving unit 614 detects the user's specified instruction based on input from the mouse 210 and the keyboard 209.

[0079] (Processing flow)

[0080] Figure 8 This is a flowchart illustrating the processing of the information processing apparatus 101 according to this exemplary embodiment. (Referring to the above...) Figure 5 The same steps in the flowchart described according to the first exemplary embodiment are assigned the same reference numerals, and redundant descriptions thereof will be omitted.

[0081] In step S516, the relevant lesion information acquisition unit 315 determines whether the operation status of the detection process for the lesion specified by the instruction receiving unit 614 through user operation is "not implemented" or "feasible". When the operation status is "not implemented" and "feasible" ("yes" in step S516), the process proceeds to step S521. On the other hand, when the operation status is "not implemented" and "not feasible" ("no" in step S516), the process returns to step S507. According to this exemplary embodiment, the operation information acquired by the relevant lesion information acquisition unit 315 includes information about the implementation of the detection process and information about the feasibility of the detection process.

[0082] In step S521, the instruction receiving unit 614 receives an instruction for performing lesion detection processing that was determined by the relevant lesion information acquisition unit 315 to be "not implemented" and "feasible" in step S516.

[0083] According to this exemplary embodiment, when the instruction receiving unit 614 receives an instruction, the display control unit 316 displays... Figure 7 The lesion detection instruction check window 404 is shown and receives confirmation input from the user. When the instruction receiving unit 614 receives a "yes" confirmation from the user in the lesion detection instruction check window 404, it instructs the lesion detection unit 312 to perform lesion detection. On the other hand, when the instruction receiving unit 614 receives a "no" confirmation, it ends the processing of this step without instructing the lesion detection unit 312 to perform lesion detection.

[0084] As described above, according to this exemplary embodiment, when a user specifies a lesion detected by the lesion detection unit 312, the related lesion information acquisition unit 315 automatically determines the type of related lesions associated with the lesion of interest to the user, and the display control unit 316 displays the results of the related lesion detection. Therefore, even when the number of detected lesions of interest increases, the user can easily discover whether there are detection results for other lesions related to the lesion of interest to the user.

[0085] Furthermore, the display control unit 316 controls the display unit to display the operational status of the lesion detection and processing. This makes it easier for the user to distinguish between a state where no lesion was detected during CADE and a state where no lesion was detected when CADE was not performed.

[0086] When the lesion detection is "not implemented" or "feasible", the instruction receiving unit 614 receives an instruction from the user to perform additional lesion detection processing, making it easier to implement the lesion detection processing that was not implemented.

[0087] (Variations of the second exemplary embodiment)

[0088] The above description focuses on the scenario where the instruction receiving unit 614 receives instructions for lesion detection processing and the lesion detection unit 312 performs additional lesion detection based on the user instructions. However, the instruction receiving unit 614 is not necessarily required to receive user instructions. In this case, the relevant lesion information acquisition unit 315 may instruct the lesion detection unit 312 to perform detection processing that has not yet been implemented.

[0089] The information processing apparatus 101 according to the third exemplary embodiment includes the information processing apparatus 101 according to the second exemplary embodiment, and further includes a display control unit 316 with the following functions: when the operation status of the relevant lesion detection is "infeasible", the control display unit displays information about the reason for the infeasibility. When the detection process is infeasible because the lesion detection processing function is not installed, the relevant lesion information acquisition unit 315 receives an instruction to install the lesion detection processing function via the instruction receiving unit 614. When the detection process is infeasible because the medical image data does not meet the implementation conditions of the detection process, the relevant lesion information acquisition unit 315 receives an instruction to acquire medical image data that meets the implementation conditions. The system structure and hardware structure of the information processing apparatus 101 according to this exemplary embodiment are similar to those described above. Figure 1 and Figure 2 The system configuration and hardware configuration of the information processing apparatus 101 according to the first exemplary embodiment are described, and the functional blocks according to this exemplary embodiment are similar to those described above. Figure 6 The functional blocks described according to the second exemplary embodiment will be omitted, and their redundant descriptions will be omitted.

[0090] (User interface screen)

[0091] Figure 9A and Figure 9B This is an example of a user interface screen displayed by the display control unit 316 of the information processing device 101 according to this exemplary embodiment. (Referring to the above references) Figure 4 and Figure 7 The same elements of the user interface described according to the first exemplary embodiment and the second exemplary embodiment are assigned the same reference numerals, and their redundant descriptions will be omitted.

[0092] The user interface screen 900 displayed on the display unit by the display control unit 316 includes the user interface screen 700 according to the second exemplary embodiment. Furthermore, when the information regarding the operating status (feasibility of the detection process) is "not feasible," the user interface screen 900 displays the reason for the infeasibility. In this case, the instruction receiving unit 614 receives additional instructions from the user. The user interface screen 900 displayed on the display unit by the display control unit 316 includes a lesion detection installation instruction check window 405 and an implementation condition image acquisition instruction check window 406.

[0093] In the user interface screen 900 displayed on the display unit by the display control unit 316, the relevant lesion detection result 431-5 indicates that the detection processing is not feasible because the lesion detection processing function is "not installed", and the relevant lesion detection result 431-6 indicates that the detection processing is not feasible because the medical image data to be processed by lesion detection is "not applicable" for the detection processing.

[0094] The lesion detection installation instruction check window 405, displayed by the display control unit 316 on a display unit such as the display 207, is used by the instruction receiving unit 614 to check whether the user has issued an instruction to install the lesion detection and treatment function. The display control unit 316 controls the display unit to detect and treat lesions that are infeasible due to the lack of installation of the lesion detection and treatment function, as specified by the user. Figure 9A When the relevant lesion detection results (431-5) are displayed, the lesion detection installation instruction check window 405 is shown. When the user specifies "Yes" in the lesion detection installation instruction check window 405, the instruction receiving unit 614 instructs the lesion detection unit 312 to install the lesion detection processing function. On the other hand, when the user specifies "No", the instruction receiving unit 614 does not instruct the lesion detection unit 312 to install the lesion detection processing function. The instruction receiving unit 614 detects the user's specified instruction based on input from the mouse 210 and the keyboard 209. The instruction to install the lesion detection processing function via the instruction receiving unit 614 is, for example, key input for software installation and software startup.

[0095] The implementation condition image acquisition instruction check window 406, displayed on the display unit by the display control unit 316, is used by the instruction receiving unit 614 to check whether the user has issued an instruction to acquire medical image data that meets the implementation conditions for lesion detection and treatment. When the user specifies that the detection and treatment of a lesion is not feasible because the medical image data does not meet the applicable conditions for lesion detection and treatment (…), the instruction is checked. Figure 9B When the relevant lesion detection results (431-6) are displayed, the implementation condition image acquisition instruction check window 406 is displayed. When the user specifies "Yes" in the implementation condition image acquisition instruction check window 406, the instruction receiving unit 614 instructs the image data acquisition unit 311 to acquire medical image data. On the other hand, when the user specifies "No", the instruction receiving unit 614 does not instruct the image data acquisition unit 311 to acquire medical image data. The instruction receiving unit 614 detects user-specified instructions based on input from the mouse 210 and keyboard 209. The implementation condition image acquisition instruction check window 406 includes a user interface for issuing instructions to add and change specified conditions. Applicable conditions for lesion detection processing include the mode of image acquisition, reconstruction function, contrast conditions, temporal phase, and imaging range. Instructions for acquiring medical image data include condition-based image reconstruction and imaging commands to the ordering system.

[0096] (Processing flow)

[0097] Figure 10 The processing flow of the information processing apparatus 101 according to this exemplary embodiment is shown. Referring to the above, respectively... Figure 5 and Figure 8 The same steps in the flowcharts described according to the first and second exemplary embodiments are assigned the same reference numerals, and their redundant descriptions will be omitted.

[0098] In step S517, the relevant lesion information acquisition unit 315 determines whether the operation status of the detection processing of the lesion specified by the user is "infeasible" due to the lack of installation of the lesion detection processing function. When the detection processing is infeasible due to the lack of installation of the lesion detection processing function ("Yes" in step S517), the process proceeds to step S531. On the other hand, when the detection processing is not infeasible due to the lack of installation of the lesion detection processing function ("No" in step S517), the process proceeds to step S518. According to this exemplary embodiment, the operation information acquired by the relevant lesion information acquisition unit 315 includes information about the implementation of the detection processing, information about the feasibility of the detection processing, and the reason why the detection processing is infeasible.

[0099] In step S531, the CPU 203 instructs the lesion detection unit 312 via the instruction receiving unit 614 to install the lesion detection processing function specified by the user. According to this exemplary embodiment, when the instruction receiving unit 614 instructs the lesion detection unit 312 to install the lesion detection processing function, the display control unit 316 controls the display unit of the display 207 to display... Figure 9A The lesion detection installation instruction check window 405 is shown. Then, the instruction receiving unit 614 receives confirmation input from the user. When a specified "Yes" is received in the lesion detection installation instruction check window 405, the instruction receiving unit 614 instructs the lesion detection unit 312 to install the lesion detection processing function. On the other hand, when a specified "No" is received, the instruction receiving unit 614 ends the processing of this step without instructing the lesion detection unit 312 to install the lesion detection processing function.

[0100] In step S518, the relevant lesion information acquisition unit 315 determines whether the operation status of the detection processing of the lesion specified by the user is "infeasible" because the medical image data does not meet the implementation conditions of the lesion detection processing. When the lesion detection processing is infeasible because the medical image data does not meet the implementation conditions of the lesion detection processing ("Yes" in step S518), the process proceeds to step S541. On the other hand, when the lesion detection processing is not infeasible because the medical image data does not meet the implementation conditions of the lesion detection processing ("No" in step S518), the process returns to step S507.

[0101] In step S541 (when the relevant lesion information acquisition unit 315 determines in step S518 that the implementation of the detection process is not feasible due to the medical image data not meeting the implementation conditions), the relevant lesion information acquisition unit 315 instructs the image data acquisition unit 311 to acquire medical image data that meets the implementation conditions of the lesion detection process corresponding to the lesion detection result. According to this exemplary embodiment, the display control unit 316 controls the display unit of the display 207 to display... Figure 9B The implementation condition image acquisition instruction check window 406 is shown, and the user receives confirmation input from the user regarding the implementation of the process of acquiring medical image data that meets the implementation conditions. When the instruction receiving unit 614 detects a specified "Yes" in the implementation condition image acquisition instruction check window 406, the relevant lesion information acquisition unit 315 instructs the image data acquisition unit 311 to acquire medical image data that meets the implementation conditions. On the other hand, when the instruction receiving unit 614 detects a specified "No", the relevant lesion information acquisition unit 315 ends the processing of this step without instructing the image data acquisition unit 311 to acquire medical image data. The image data acquisition unit 311 can generate medical image data that meets the implementation conditions based on the acquired medical image data, acquire medical image data from an external source, or send an acquisition command to the ordering system.

[0102] As described above, according to this exemplary embodiment, when a user specifies a lesion detected by the lesion detection unit 312, the related lesion information acquisition unit 315 automatically determines the type of the related lesion, and the display control unit 316 displays the detection results of related lesions associated with the user's lesion of interest. Therefore, even when the number of detected lesions of interest increases, the user can easily discover whether there are detection results of other lesions related to the user's lesion of interest.

[0103] The display control unit 316 controls the display unit to display the operational status of lesion detection and processing. This makes it easier for the user to distinguish between a state where no lesion was detected during CADE and a state where no lesion was detected without CADE.

[0104] When the lesion detection processing is "not implemented" or "feasible", the relevant lesion information acquisition unit 315 issues a lesion detection processing instruction, making it easier to implement the lesion detection that has not been implemented.

[0105] When the lesion detection and processing function is not feasible due to the lack of installation of the detection and processing function, the relevant lesion information acquisition unit 315 issues an instruction to install the lesion detection and processing function, making it easier to install the required lesion detection and processing function.

[0106] When lesion detection and processing is not feasible due to medical image data not meeting the implementation conditions, the relevant lesion information acquisition unit 315 issues an instruction to acquire medical image data that meets the implementation conditions, making it easier to acquire the required medical image data.

[0107] (Other exemplary embodiments)

[0108] The present invention can also be implemented by performing the following process. More specifically, software (programs) for implementing the functions of the exemplary embodiments described above are supplied to the system or device via a network or various types of storage media, and the computer (CPU or microprocessor unit (MPU) of the system or device reads and executes the program.

[0109] Other embodiments

[0110] Embodiments of the invention can also be implemented by a computer that reads and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (also more fully referred to as a "non-transitory computer-readable storage medium") to perform one or more functions in the above embodiments, and / or includes one or more circuits (e.g., application-specific integrated circuits (ASICs)) for performing one or more functions in the above embodiments. Furthermore, embodiments of the invention can be implemented using a method by which the computer of the system or device, for example, reads and executes the computer-executable instructions from the storage medium to perform one or more functions in the above embodiments, and / or controls the one or more circuits to perform one or more functions in the above embodiments. The computer may include one or more processors (e.g., central processing unit (CPU), microprocessor unit (MPU)) and may include separate computers or a network of separate processors to read and execute the computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, a hard disk, random access memory (RAM), read-only memory (ROM), the memory of a distributed computing system, or an optical disc (such as a compact disc (CD), a digital versatile optical disc (DVD), or a Blu-ray disc (BD)). TMOne or more of the following: flash memory devices and memory cards.

[0111] The embodiments of the present invention can also be implemented by providing software (programs) that perform the functions of the above embodiments to a system or device via a network or various storage media, and the computer or central processing unit (CPU) or microprocessor unit (MPU) of the system or device reads out and executes the program.

[0112] Although the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The appended claims should be interpreted in the broadest possible sense to encompass all such variations and equivalent structures and functions.

Claims

1. An information processing apparatus, comprising: The detection result acquisition unit is configured to acquire the results of lesion detection from medical image data; The instruction receiving unit is configured to receive instructions for specifying lesions of interest based on detection results; The relevant lesion information acquisition unit is configured to acquire (i) relevant lesion information about the relevant lesion and (ii) operation status information, wherein the relevant lesion is related to the lesion of interest and is of a different type from the lesion of interest, and the operation status information includes information about whether detection and treatment for the relevant lesion has been implemented; as well as The display control unit is configured to control the display unit to display the relevant lesion information and the operation status information in a correlated manner.

2. The information processing apparatus according to claim 1, wherein The operational status information also includes information regarding the feasibility of detection and treatment for the relevant lesions.

3. The information processing device according to claim 1, wherein The relevant lesion information acquisition unit refers to a table that associates the types of lesions.

4. The information processing apparatus according to claim 2, wherein If the detection process is feasible, the instruction receiving unit receives an instruction for implementing the detection process.

5. The information processing apparatus according to claim 2, wherein If the detection process is not feasible, the display control unit controls the display unit to display information about the reason why the detection process is not feasible.

6. The information processing apparatus according to claim 5, wherein Information regarding the reasons why the detection and treatment are not feasible includes information indicating that a detector for detecting and treating the relevant lesions is not installed.

7. The information processing apparatus according to claim 6, wherein The instruction receiving unit receives instructions for installing any detectors that are not currently installed.

8. The information processing apparatus according to claim 5, wherein Information regarding the reasons why the detection process is not feasible includes information indicating that the medical image data does not meet the conditions for implementing the detection process.

9. The information processing apparatus according to claim 8, wherein If the medical image data does not meet the implementation conditions for the detection and processing of the relevant lesion, the instruction receiving unit receives an instruction for acquiring medical image data that meets the implementation conditions.

10. The information processing apparatus according to claim 1, wherein The relevant lesion information acquisition unit also acquires the results of the detection processing, and The display control unit controls the display unit to display the results of the detection process.

11. The information processing apparatus according to claim 1, wherein The display control unit displays the medical image data, the detection results, the relevant lesion information, and the operation status information in a recognizable manner.

12. An information processing method, comprising: The result acquisition step obtains the results of lesion detection from medical image data; In the instruction receiving step, an instruction is received to specify the lesion of interest based on the test results; In the relevant lesion information acquisition step, (i) relevant lesion information about the relevant lesion and (ii) operation status information are acquired, wherein the relevant lesion is related to the lesion of interest and is of a different type than the lesion of interest, and the operation status information includes information about whether detection and treatment for the relevant lesion have been implemented; as well as In the display control step, the display unit is controlled to display the relevant lesion information and the operation status information in a correlated manner.

13. A non-transitory computer-readable storage medium storing a program, which, when executed on a computer, causes the computer to perform the information processing method according to claim 12.

14. A stored-program computer program product, which, when executed on a computer, causes the computer to perform the information processing method according to claim 12.

15. An information processing apparatus, comprising: A display control unit is configured to control a display unit to display the results of detecting at least one lesion from medical image data, and to control the display unit to display in a correlated manner (i) relevant lesion information and (ii) operation status information, the relevant lesion being related to a lesion of interest selected for the at least one lesion in response to an instruction from a user, the operation status information including information on whether detection processing for the relevant lesion has been performed.

Images