Information processing device, diagnostic device, medical system, program, and information processing method

Abstract

This system makes it easier to distinguish between clinical support information obtained through computer processing and diagnostic support information provided by technicians, and provides this information to physicians in an easily understandable manner. [Solution] The information processing device comprises a control unit for controlling the capture of medical images, a display unit for displaying at least medical images, a first information generation unit for generating first diagnostic support information for medical images by predetermined computer processing, an input unit for inputting second diagnostic support information regarding medical images, and an output unit for outputting the first diagnostic support information and the second diagnostic support information to an external device in an identifiable manner.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing apparatus, a diagnostic apparatus, a medical system, a program, and an information processing method. [Background Art]

[0002] In recent years, in order to reduce the burden on doctors, STAT image reporting, in which a radiological technologist promptly notifies a doctor (attending physician) of images showing findings of highly urgent diseases related to life prognosis after imaging, has attracted attention. Through STAT image reporting, the captured images are immediately reported to the doctor, enabling rapid and appropriate treatment.

[0003] For example, Patent Document 1 describes that when a life-threatening disease is detected from a medical image captured of a patient, contact information and hospitalization status of a doctor or the like are acquired. Then, an information processing apparatus that notifies information of an available doctor or the like together with the detection result of the disease is described.

[0004] Furthermore, recently, a computer-aided detection / diagnosis (CAD) system, which analyzes medical images using a computer, presents lesion detection regions detected through image analysis to doctors, and seeks the doctor's judgment, has been put into practical use. In diagnostic support using CAD, diagnostic support information is generated by performing computer processing such as AI (Artificial Intelligence) analysis on medical images, and is provided to doctors. [Prior Art Documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent No. 7405546 [Summary of the Invention] [Problem to be Solved by the Invention]

[0006] When CAD-assisted diagnosis is used, both CAD-assisted diagnostic information and technologist-assisted diagnostic information coexist for a single medical image. In this case, there is a risk that the physician may confuse the two. For example, if CAD information generated by AI analysis includes comments and suggestions in the same way as a human engineer would, it becomes difficult to distinguish it from information provided by a human engineer. This distinction becomes even more difficult if the CAD information uses detailed, human-like expressions. Conversely, there are cases where engineers only add annotations and do not include comments or other text. If engineers perform mechanical processing such as CAD, it may become even more difficult to distinguish between the two.

[0007] In this regard, the device described in Patent Document 1 can promptly contact doctors and other medical professionals when a highly urgent illness is detected. However, it cannot distinguish whether the diagnostic support information for medical images is provided by a technician or by CAD.

[0008] This disclosure is made in view of the problems in the prior art described above, and aims to provide physicians with clinical support information obtained by computer processing and diagnostic support information obtained by technicians in an easily distinguishable manner. [Means for solving the problem]

[0009] To solve the above problems, the information processing device disclosed herein is: A control unit that controls the acquisition of medical images, At least a display unit for displaying the medical image, A first information generation unit generates first diagnostic support information from the medical image by predetermined computer processing, An input unit for inputting second diagnostic support information regarding the aforementioned medical image, An output unit that outputs the first diagnostic support information and the second diagnostic support information to an external device in a distinguishable manner, It is equipped with.

[0010] Furthermore, the diagnostic device described herein is An acquisition unit that acquires first diagnostic support information generated by predetermined computer processing on a medical image, and second diagnostic support information which is input with respect to the medical image and is identifiable as the first diagnostic support information, At least a display unit for displaying the medical image, It is equipped with.

[0011] Furthermore, the medical system disclosed herein is A medical system having the information processing device described in claim 1, An acquisition unit that acquires the first diagnostic support information and the second diagnostic support information, The diagnostic device further comprises a second display unit for displaying at least medical images.

[0012] Furthermore, the program disclosed herein is In the computer of a device equipped with a display unit, A shooting control function that controls the acquisition of medical images, At least a display control function that causes the medical image to be displayed on the display unit, A generation function that generates first diagnostic support information from the aforementioned medical image through predetermined computer processing, An input reception function that accepts input of second diagnostic support information related to the aforementioned medical image, An output function that outputs the first diagnostic support information and the second diagnostic support information to an external device in an identifiable manner, Make it run.

[0013] Furthermore, the information processing method disclosed herein is: A shooting control step that controls the acquisition of medical images, At least a display control step of displaying the medical image on the display unit, A generation step of generating first diagnostic support information from the medical image by predetermined computer processing, An input acceptance step for receiving input of second diagnostic support information related to the aforementioned medical image, an information output step of outputting said first diagnosis support information and said second diagnosis support information to an external device in a distinguishable manner.

Effect of the Invention

[0014] According to the present disclosure, it becomes possible to provide medical diagnosis support information obtained through computer processing and diagnosis support information provided by a technologist to a physician in an easily distinguishable manner.

Brief Description of Drawings

[0015] [Figure 1] It is a system configuration diagram of a medical system. [Figure 2] It is a block diagram showing the functional configuration of a console. [Figure 3] It is a block diagram showing the functional configuration of a diagnostic apparatus. [Figure 4] It is a diagram showing a display example of a display screen of a console on which a medical image is displayed. [Figure 5] It is a diagram showing a display example of a display screen of a console on which a medical image and first diagnosis support information are displayed. [Figure 6] It is a diagram showing a display example of a display screen of a console on which a medical image and second diagnosis support information are displayed. [Figure 7] It is a diagram showing a display example of a display screen of a console on which a medical image and second diagnosis support information are displayed. [Figure 8] It is a diagram showing a display example of a display screen of a diagnostic apparatus on which a medical image superimposed with first diagnosis support information and a medical image superimposed with second diagnosis support information are displayed in parallel. [Figure 9] It is a diagram showing a display example of a display screen of a diagnostic apparatus on which a medical image superimposed with first diagnosis support information and second diagnosis support information is displayed. [Figure 10] It is a diagram showing an example of a display screen of a diagnostic apparatus that enables selection of a display method for first diagnosis support information and second diagnosis support information to be superimposed on a medical image. [Figure 11]This figure shows an example of the display screen of a diagnostic device when the option to overlay the first diagnostic support information onto a medical image is selected. [Figure 12] This figure shows an example of a display screen of a diagnostic device, where the overlapping region between the first diagnostic support information and the first diagnostic support information is shown on a medical image. [Figure 13] This figure shows an example of the display screen of a diagnostic device when a change is made in the display of second diagnostic support information superimposed on a medical image. [Figure 14] This flowchart shows the information processing performed by the console and diagnostic device to support the diagnosis. [Modes for carrying out the invention]

[0016] Hereinafter, an embodiment of the information processing device, diagnostic device, medical system, program, and information processing method relating to this disclosure will be described. However, the scope of the invention is not limited to the illustrated example.

[0017] [Medical System Configuration] Figure 1 shows the system configuration of the medical system 100 according to this embodiment. As shown in Figure 1, the medical system 100 consists of a modality 1, a console 2 which is an information processing device, and a diagnostic device 4. Each device is capable of data communication via a communication network N. In the medical system 100, medical images captured and generated in modality 1 can be displayed on console 2. Each device constituting the medical system 100 conforms to HL7 (Health Level Seven) and DICOM (Digital Image and Communications in Medicine) standards, and communication between devices is conducted in accordance with HL7 and DICOM. The number and types of modality 1 devices and the number of diagnostic devices 4 are not particularly limited.

[0018] Modality 1 involves imaging the patient (subject) and generating medical image data. Imaging is performed based on examination order information transmitted from a RIS (Radiology Information System) or similar system (not shown). Examples of Modality 1 include CR (Computed Radiography), DR (Digital Radiography), CT (Computed Tomography), MRI (Magnetic Resonance Imaging), US (Ultra Sonography), NM (Nuclear Medicine), and ES (Endoscopy). Medical images generated by Modality 1 may include, for example, X-ray images, CT images, MRI images, ultrasound images, mammography images, angiography images, and endoscopic images.

[0019] Modality 1 attaches image attribute information to a medical image. This image attribute information includes patient ID, patient name, date of birth, gender, date and time of shooting, image ID, modality, body part, direction, etc. Image data of medical images generated by Modality 1 can be displayed on the display unit 24 of Console 2, as described later. The image data of medical images is also transmitted to and stored on a medical image management server (not shown). The medical image management server stores and manages the image data of medical images generated in Modality 1. Examples of medical image management servers include PACS (Picture Archiving and Communication System).

[0020] [Console configuration as an information processing device] Console 2 is an imaging control device that controls imaging in Modality 1. Console 2 is operated by a technician, such as a radiologic technologist, who performs the imaging. Console 2 outputs shooting conditions, image reading conditions, etc., to Modality 1. It also acquires image data of medical images captured by Modality 1 and stores it appropriately in a medical image management server, etc. Furthermore, in this embodiment, console 2 is an information processing device that attaches diagnostic support information to medical images.

[0021] Figure 2 shows the functional configuration of console 2. Console 2 is comprised of a control unit 21, a storage unit 22, an operation unit 23, a display unit 24, a communication unit 25, and the other components are connected by a bus 26.

[0022] The control unit 21 consists of a CPU (Central Processing Unit), RAM (Random Access Memory), etc., and comprehensively controls the processing operations of each part of the console 2. The storage unit 22 is composed of, for example, an HDD or a non-volatile semiconductor memory. The storage unit 22 stores various processing programs, parameters and files necessary for the execution of those programs, and so on. The CPU of the control unit 21 reads various processing programs stored in the memory unit 22, loads them into RAM, and performs various processing in cooperation with those programs.

[0023] For example, as shown in Figure 2, the memory unit 22 of this embodiment stores an image acquisition control processing program 221. The CPU of the control unit 21 first works in cooperation with this program to control the acquisition of medical images by modality 1. Furthermore, as shown in Figure 2, the storage unit 22 of this embodiment stores a diagnostic support processing program 222. The CPU of the control unit 21 first works in cooperation with this program to control the display unit 24 to display at least the medical image generated by modality 1.

[0024] Furthermore, the control unit 21 in this embodiment functions as a first information generation unit (hereinafter simply referred to as the generation unit) that generates first diagnostic support information. As a generation unit, the control unit 21 performs predetermined computer processing on medical images to generate first diagnostic support information. The function as a generation unit is realized by software processing through the cooperation of the diagnostic support processing program 222 stored in the storage unit 22 and the CPU of the control unit 21. The predetermined computer processing is image diagnosis and image analysis, including the detection of candidate lesion locations (abnormal locations) by CAD (Computer Aided Diagnosis). Image analysis by CAD is a concept that broadly includes various processes, but in this embodiment, for example, it is an analysis process on medical images using AI (Artificial Intelligence).

[0025] The first diagnostic support information is the result of inference by computer processing (AI analysis in this embodiment). In the AI ​​analysis, deep learning is introduced to extract areas from the image that have a high degree of likelihood of being lesions as suspected lesion areas (abnormal areas). The first diagnostic support information may include a numerical value indicating the likelihood of lesions derived by the AI ​​for each area, as a confidence score. The first diagnostic support information may include, for example, image information such as annotations and marks superimposed on medical images, and textual information such as comments and suggestions. The control unit 21 attaches tag information to the first diagnostic support information to indicate that it is diagnostic support information (CAD information) generated by computer processing.

[0026] Furthermore, the control unit 21 of this embodiment, in cooperation with the diagnostic support processing program 222, receives the input information when the second diagnostic support information is input from the operation unit 23. The second set of diagnostic support information is input information entered by a technician operating console 2. This second set of diagnostic support information is related to medical images, and is, for example, information related to the first set of diagnostic support information derived by AI analysis. The second type of diagnostic support information includes, for example, image information such as annotations and marks superimposed on medical images, and textual information such as comments and suggestions. The control unit 21 attaches tag information to the second diagnostic support information to indicate that it is diagnostic support information entered by a technician.

[0027] Furthermore, the control unit 21 of this embodiment, in cooperation with the diagnostic support processing program 222, controls the communication unit 25 and functions as an output unit that outputs diagnostic support information to an external device. The "external device" referred to here is, for example, the diagnostic device 4 described later, a medical image management server (PACS), an image review device, etc.

[0028] The control unit 21, acting as an output unit, outputs the first diagnostic support information and the second diagnostic support information to an external device in a identifiable manner. The control unit 21 outputs the second diagnostic support information in association with the first diagnostic support information. Furthermore, the first and second diagnostic support information are associated (linked) with medical images and output to the external device.

[0029] The method for outputting the first diagnostic support information and the second diagnostic support information in an "identifiable" manner is not particularly limited. If there is first diagnostic support information and second diagnostic support information associated with a certain medical image, for example, the control unit 21 attaches tag information to each diagnostic support information as described above. This makes it possible to identify whether it is the first or second diagnostic support information. The tag information may also include the display mode (described later) when displaying the first and second diagnostic support information on the display unit of an external device.

[0030] The operation unit 23 includes, for example, a keyboard equipped with cursor keys, character / number input keys, and various function keys, and a pointing device such as a mouse. The operation unit 23 outputs operation signals input by key operations on the keyboard or mouse operations to the control unit 21. The user (technician) inputs second diagnostic support information by operating the operation unit 23. Alternatively, the operation unit 23 may be configured as a touch panel stacked on the display unit 24. In this case, operation signals corresponding to the position of touch operations by the user's (technician's) finger, etc., are output to the control unit 21.

[0031] The display unit 24 is configured with a monitor such as an LCD (Liquid Crystal Display). It displays various screens according to the instructions of the display signals input from the control unit 21. Figures 4 to 7 show examples of the display screen 24a of the display unit 24. In this embodiment, the display screen 24a of the console 2 displays at least a medical image 241, as shown in Figure 4, for example. The display screen 24a also includes an order information field 242 that displays information about the patient being examined, a tool display field 243 that displays various tools, and so on. Furthermore, the display screen 24a is provided with various operation buttons 245 that are operated by the user (technician).

[0032] The display unit 24 may display, for example, the medical image 241 and the first diagnostic support information, as shown in Figure 5. In this case, the medical image 241 and the first diagnostic support information are displayed on the same screen. Figure 5 shows an example in which triangular marks M1 to M4 are superimposed on the medical image 241 as the first diagnostic support information.

[0033] The user (technician) can view and confirm the medical image 241 and the first diagnostic support information on the same screen of the display unit 24. Then, while referring to the medical image 241 and the first diagnostic support information, the user inputs second diagnostic support information, such as marks and comments, for the medical image 241 from the operation unit 23 or the like. Figure 6 shows an example where circular markers m1 and m2 are superimposed on medical image 241 as a second diagnostic support information. Figure 7 shows the markers m1 and m2 superimposed with an arrow S indicating a suspected lesion as another second diagnostic support information. Furthermore, in the illustrated example, a comment C describing the suspected lesion is superimposed on medical image 241.

[0034] The communication unit 25 is composed of a network interface and the like. The communication unit 25 transmits and receives data with external devices connected via the communication network N. For example, the communication unit 25 receives medical images obtained by photographing a patient using modality 1. The communication unit 25 also transmits and receives medical images and analysis results with external devices such as a medical image management server and a diagnostic device 4 as appropriate, in accordance with the control of the control unit 21. That is, the first diagnostic support information generated by the control unit 21 of the console 2 and the second diagnostic support information input by the operation unit 23 and received by the control unit 21 are transmitted to and stored in external devices such as a medical image management server and a diagnostic device 4 as appropriate via the communication unit 25.

[0035] [Diagnostic device configuration] Figure 3 shows the functional configuration of the diagnostic device 4. Diagnostic device 4 receives medical images generated by modality 1, as well as first and second diagnostic support information attached to the medical images in console 2. The information received by diagnostic device 4 becomes accessible to the physician on the device. Diagnostic device 4 is a terminal device operated by the physician, who is the user. Diagnostic device 4 is, for example, a radiologist's image interpretation terminal or a reference terminal used by a practicing physician (clinician). As shown in Figure 3, the diagnostic device 4 is composed of a control unit 41, a storage unit 42, an operation unit 43, a display unit 44, a communication unit 45, etc., and each unit is connected by a bus 46.

[0036] The control unit 41 consists of a CPU, RAM, etc., and comprehensively controls the processing operations of each part of the diagnostic device 4. Specifically, the CPU reads various processing programs stored in the memory unit 42, loads them into the RAM, and performs various processing operations in cooperation with those programs. The storage unit 42 is composed of an HDD or non-volatile semiconductor memory, and stores various processing programs, parameters and files necessary for the execution of those programs.

[0037] For example, as shown in Figure 3, the storage unit 42 of this embodiment stores a diagnostic support processing program 421. The CPU of the control unit 41 first functions as an acquisition unit that acquires the aforementioned first diagnostic support information and second diagnostic support information in cooperation with the program. As mentioned above, the first diagnostic support information is the inference result of AI analysis generated by predetermined computer processing (AI analysis) on medical images. The second diagnostic support information is information input regarding medical images that is identifiable from the first diagnostic support information.

[0038] Furthermore, the control unit 41 may function as a third information generation unit (hereinafter simply referred to as a generation unit) in cooperation with the diagnostic support processing program 421. As a generation unit, the control unit 41 generates third diagnostic support information from the first diagnostic support information and the second diagnostic support information based on predetermined conditions.

[0039] Here, the "third diagnostic support information" is generated from the first and second diagnostic support information, but it is different from both the first and second. For example, if the first and second diagnostic support information are marks of the same color but differ only in shape, the "third diagnostic support information" may be the first and second information with different colors added to each. The "third diagnostic support information" may also be information in which the first and second diagnostic support information, distinguished by different colors, are superimposed on a single medical image (see Figure 9).

[0040] Furthermore, the "third diagnostic support information" may be, for example, a numerical representation of the reliability of the likelihood of a lesion derived by AI. In addition, the "third diagnostic support information" may be the overlapping area of ​​the parts detected as suspected lesion areas in both the first and second diagnostic support information (see Figure 10). The degree of overlap of the parts detected as suspected lesion areas in the first and second diagnostic support information is calculated, for example, by the control unit 41 as a generation unit. The image displayed in the display unit 44, described later, does not have to be the first or second diagnostic support information itself, but may be a "third diagnostic support information" or the like generated based on them.

[0041] The operation unit 43 also includes, for example, a keyboard equipped with cursor keys, character / number input keys, and various function keys, and a pointing device such as a mouse. The operation unit 43 outputs operation signals input by key operations on the keyboard or mouse operations to the control unit 41. Alternatively, the operation unit 43 may be composed of a touch panel stacked on the display unit 44. In this case, operation signals corresponding to the position of touch operations by the user's finger, etc., are output to the control unit 41.

[0042] The display unit 44 is equipped with a monitor such as an LCD (Liquid Crystal Display) and displays various screens according to the instructions of the display signals input from the control unit 41. Figures 8 to 11 show examples of the display screen 44a of the display unit 44. In this embodiment, the display unit 44 of the diagnostic device 4 is a "second display unit" that displays at least the medical image 441 (see Figure 10). The display unit 44 may display the medical image 441 and the first diagnostic support information (see Figure 11). In this case, it is preferable to display the medical image 441 and the first diagnostic support information on the same screen. Alternatively, as shown in Figure 8, the medical image 441, the first diagnostic support information, and the second diagnostic support information may be displayed on the same screen.

[0043] In this embodiment, the display unit 44 displays the first diagnostic support information and the second diagnostic support information in a distinguishable manner, in accordance with the control of the control unit 41. Here, "displaying in an identifiable manner" means, for example, displaying in different display modes. It is sufficient that the two pieces of information are identifiable; there are no particular limitations on how they are displayed in different display modes.

[0044] For example, in Figure 8, an image in which the first diagnostic support information is superimposed on medical image 441 and an image in which the second diagnostic support information is superimposed on medical image 441 are displayed on the same screen. In Figure 8 and the following explanation, the first diagnostic support information is also referred to as "CAD information." The second diagnostic support information is also referred to as "technician input information." As described above, the first and second diagnostic support information output from console 2 are associated with medical images. In addition, the control unit 21 of console 2 attaches tag information to each piece of information, indicating whether it is the first or second diagnostic support information. The control unit 41 displays an image with the first diagnostic support information superimposed on it in the CAD information field 451 based on the tag information (see Figure 8). It also displays an image with the second diagnostic support information superimposed on it in the technician input information field 452. This prevents the user (physician) from misunderstanding or becoming confused, even when the first and second diagnostic support information are displayed side by side on the same display screen 44a.

[0045] "Displaying in an identifiable manner" includes, for example, changing at least one of the following: text, color, size, font, and shape. More specifically, it includes, for example, changing the color, size, shape, etc. of marks, etc., between the first and second diagnostic support information. It also includes changing the color, size, and font of text information such as comments and suggestions between the first and second diagnostic support information.

[0046] Furthermore, the display unit 44 may display either the first diagnostic support information or the second diagnostic support information based on certain conditions (predetermined conditions). For example, the operation unit 43 of the diagnostic device 4 may input a request to refer to the first diagnostic support information. In this case, the control unit 41 will display only the first diagnostic support information superimposed on the medical image 441 on the display unit 44. Alternatively, the operation unit 43 of the diagnostic device 4 may input a request to refer to the second diagnostic support information. In this case, the control unit 41 will display only the second diagnostic support information superimposed on the medical image 441 on the display unit 44. By superimposing either one of the two pieces of information onto the medical image 441, the relevant information can be emphasized.

[0047] For example, as shown in Figure 10, the display screen of the display unit 44 may be provided with tabs or buttons 445 for first diagnostic support information and tabs or buttons 445 for second diagnostic support information. In this case, when a user (doctor) selects either tab or button 445 with a pointing device such as a mouse, the selected information may be superimposed on the medical image 441. Alternatively, a button for superimposing both the first and second diagnostic support information may be provided to instruct the user to superimpose both information.

[0048] The example shown in Figure 10 illustrates the case where CAD information is selected. In this case, as shown in Figure 11, only the CAD information is superimposed on the medical image 441 and displayed on the display screen 44a. If the user (doctor) selects to superimpose both pieces of information, both the first and second diagnostic support information are superimposed on the medical image 441 and displayed, as shown in Figure 9, for example. Even when the two pieces of information are superimposed in this way, in this embodiment, the shape and color of the marks representing each piece of information are different. Therefore, the user (doctor) can correctly distinguish between the two.

[0049] Furthermore, the display unit 44 may display a single medical image 441 in which both the first and second diagnostic support information are superimposed on the medical image 441. In this case, as shown in Figure 9, the first and second diagnostic support information are represented on a single medical image 441. This allows the user (doctor) to view both pieces of information without having to switch pages or perform other actions.

[0050] Furthermore, the display unit 44 may be configured to display the differences between the first diagnostic support information and the second diagnostic support information in a way that allows for identification. For example, parts common to both the first and second diagnostic support information may be displayed in black. Also, information present only in the first diagnostic support information may be displayed in green, and information present only in the second diagnostic support information may be displayed in red. This makes it easy for the user (physician) to see the common and different parts of the two diagnostic support information. For example, in the above example, marks M2 and m1 in Figure 9 are black. Also, marks M1, M3, and M4 are green, and mark m1 is red.

[0051] Technicians may not be aware of diagnostic support information provided by CAD (AI), such as AI analysis. Furthermore, technicians may be concerned with aspects other than those determined by the AI. It is sometimes necessary to know whether technicians are concerned with CAD (AI) diagnostic support information, whether they are not concerned with it, or whether they are concerned with aspects other than those determined by the CAD (AI). Therefore, the differences between the first and second diagnostic support information are displayed in an easily identifiable manner. This allows users (physicians) to easily grasp these discrepancies between CAD (AI) judgments and the technician's judgment. For example, in the example shown in Figure 9, marks M2 and m2 are both extracted as suspected lesion areas. It can also be seen that the technician was not concerned with marks M1, M3, and M4. On the other hand, although mark m1 was not detected by the AI ​​analysis, it can be seen that the technician considered it a suspected lesion area. By overlaying the information onto a single medical image 441, the user (physician) can easily confirm the differences and commonalities between the first and second diagnostic support information.

[0052] Furthermore, the control unit 41, acting as a generation unit, may generate a third diagnostic support information from the first diagnostic support information and the second diagnostic support information. In this case, the third diagnostic support information may be displayed on the display screen 44a of the display unit 44. In this case, the first diagnostic support information, the second diagnostic support information, and the third diagnostic support information may be displayed while being switched as appropriate by the user (physician).

[0053] For example, if the first and second diagnostic support information points to approximately the same location, the overlap of the first and second diagnostic support information may be displayed on the display screen 44a, as shown in Figure 12. For example, in Figure 12, the first diagnostic support information (CAD information), mark M5, is shown with a dashed line. The second diagnostic support information (technician input information), mark m3, is shown with a solid line. The positions of mark M5 and mark m3 are slightly offset, and in Figure 12, the overlapping region Ar1 where the two overlap is shown with diagonal shading. As shown in Figure 12, if the positions of the first and second diagnostic support information are misaligned or only partially overlap, the range and area of ​​the overlapping region Ar1 may be shown as third diagnostic support information. The range and area of ​​the overlapping region Ar1 are calculated by the control unit 41, which acts as the generation unit.

[0054] Furthermore, as shown in Figure 8, the first diagnostic support information and the second diagnostic support information may be displayed side by side on the display screen 44a of the display unit 44. In this case, the user (physician) may operate to display either one in full screen. In addition, the medical image with the first diagnostic support information superimposed and the medical image with the second diagnostic support information superimposed may be switched and displayed as appropriate. Display switching operations can be performed, for example, by scrolling or clicking (selecting). Alternatively, if a mouse is used as the operation unit 43, the screen may be switched by operating the scroll wheel (or mouse wheel). Alternatively, one image selected by the user (doctor) may be displayed across the entire display screen 44a, while other images may be overlaid on the bottom or sides of the display screen 44a (for example, stacked display). In the case of stacked display, selecting an image overlaid on the bottom or sides of the display screen 44a by tapping it may cause that image to be displayed across the entire display screen 44a.

[0055] Furthermore, for example, on the display screen 24a of console 2 (see Figures 6 and 7), the user (technician) may broadly circle the suspected lesion area with a circular mark m4. Then, a mark m5 and comment C, etc., may be added to a point (range) within that circle to indicate a particularly noteworthy area. In this case, the display unit 44 of the diagnostic device 4 may display a dynamic (changing) display as shown in Figure 13. That is, as shown on the left side of Figure 13, when the user (doctor) moves the mouse pointer, cursor, etc., close to the mark m4, the mark m4 may shrink to the mark m5, which is the area of ​​interest, as shown on the right side of the figure. At this time, comments C, etc., may also appear on the display screen 44a. By displaying such dynamic (action) displays, the areas of particular interest can be clearly shown to the user (doctor).

[0056] Furthermore, the first and second diagnostic support information may be presented to the user (physician) using methods such as tooltips. In this case, for example, when the user (physician) moves the mouse pointer or cursor close to the circular marks m1 to m5, comments or supplementary explanations may be displayed around them. The display mechanism and specifications implemented in the display unit 44 of the diagnostic device 4 are set in the console 2. Specifically, for example, these display modes can be set in the tag information attached to the first and second diagnostic support information.

[0057] The communication unit 45 of the diagnostic device 4 is configured with a network interface and the like, and is capable of sending and receiving information with various devices connected via the communication network N. Devices with which the diagnostic device 4 communicates via the communication unit 45 include, for example, the console 2 and the medical image management server (PACS). The diagnostic device 4 can receive medical images and diagnostic support information from the console 2 and the medical image management server (PACS) via the communication unit 45.

[0058] [Operation in medical systems] Next, with reference to Figure 14, the operation of the medical system 100 will be described. Figure 14 is a flowchart showing the processing performed by the console 2, which is an information processing device, and the diagnostic device 4, which constitute the medical system 100 of this embodiment. As shown in Figure 14, first, when a shooting order is received from a physician, console 2 controls the corresponding modality according to the order and takes an image of the patient (subject). Then, console 2 acquires the captured medical image (step S1). The acquired medical image may be displayed on the display unit 24 of console 2.

[0059] As shown in Figure 4, the display screen 24a of the display unit 24 is equipped with medical images 241, an order information field 242, a tool display field 243, and various operation buttons 245. For example, in Figure 4, operation buttons 245 include an image analysis button for instructing image analysis by computer processing, and an exit button for ending the processing. The configuration and display content of the display screen 24a are not limited to the illustrated example.

[0060] When a medical image is acquired and the user (technician) operating console 2 presses the image analysis button, the control unit 21, acting as a generation unit, performs AI-based image analysis on the medical image and generates first diagnostic support information (step S2). Alternatively, even without the user (technician) pressing any buttons, the control unit 21 may automatically perform AI-based image analysis once a medical image is acquired.

[0061] The control unit 21 attaches tag information to the first diagnostic support information indicating that it is CAD information resulting from computer processing (AI analysis). Then, the first diagnostic support information is associated with a medical image (step S3). The medical image and the first diagnostic support information associated with it are stored in the storage unit 22, etc.

[0062] Once the AI-based image analysis is complete, the control unit 21 displays the medical image and the first diagnostic support information on the display unit 24 (step S4). Specifically, as shown in Figure 5, a display screen 24a in which the image of the first diagnostic support information is superimposed on the medical image 241 is displayed on the display unit 24. In Figure 5, an analysis result display button is provided as an operation button 245 on the display screen 24a. The analysis results may be automatically displayed on the display screen 24a even if the operation button 245 is not operated.

[0063] The example shown in Figure 5 illustrates a case where the first diagnostic support information consists of triangular marks placed on areas suspected of being lesions. In the illustrated example, four suspected lesion areas, marked M1 to M4, are detected by AI. The user (technician) can view the first diagnostic support information, which is CAD information, on the display unit 24. The user (technician) inputs their own opinion on the medical image and the first diagnostic support information via the operation unit 23, etc. (see Figure 6). The opinion entered by the user (technician) is received by the control unit 21 as second diagnostic support information (step S5). Figure 6 shows an example where the second diagnostic support information consists of a circular mark indicating a suspected lesion. In the example, two suspected lesion locations, marked m1 and m2, are identified by the user (technician). Furthermore, the analysis results from the AI ​​and the input results from the engineers may be incorporated into subsequent machine learning processes as data.

[0064] The control unit 21 attaches tag information to the second diagnostic support information indicating that it represents the technician's opinion. Then, it associates the second diagnostic support information with the medical image and the first diagnostic support information (step S6). The medical image and the first diagnostic support information associated with it may be stored in the storage unit 22 or the like. The tag information attached to the first and second diagnostic support information includes the following: that is, the display correspondence when displaying the first and second diagnostic support information is associated with each other.

[0065] The control unit 21 then outputs the medical image and the first and second diagnostic support information associated with it to an external device in an identifiable manner (step S7). In this embodiment, information output from console 2 is acquired by diagnostic device 4, which is an external device (step S11). The output from console 2 consists of medical images and the first and second diagnostic support information associated with them. Note that medical images and diagnostic support information are not limited to being output directly from console 2 to diagnostic device 4. For example, medical images and diagnostic support information may be output from console 2 to a medical image management server, which is an external device. In this case, diagnostic device 4 may acquire medical images and diagnostic support information from the medical image management server. In this case, diagnostic device 4 sends a request to acquire medical images, etc., to the medical image management server via the communication unit 45. As a result, medical images and the information associated with them are sent from the medical image management server to diagnostic device 4.

[0066] The control unit 41 of the diagnostic device 4 acquires a medical image and the first and second diagnostic support information associated with it. Then, based on the tag information, the control unit 41 displays the first and second diagnostic support information on the display unit 44 in an identifiable manner (step S12).

[0067] In this embodiment, for example, as shown in Figure 8, the first diagnostic support information and the second diagnostic support information have different mark shapes, colors, etc., making them distinguishable from each other. Therefore, even when the first and second diagnostic support information are attached to a single medical image, the user (doctor) viewing the image can easily determine which information corresponds to which. Note that the first and second diagnostic support information can only be distinguishable from each other and are not limited to the illustrated examples.

[0068] As described above, the information processing device in this embodiment is a console 2 equipped with a control unit 21 that controls the acquisition of medical images by modality 1. The console 2 also includes at least a display unit 24 for displaying medical images and an operation unit 23 as an input unit for inputting second diagnostic support information regarding the medical images. The control unit 21 also functions as a first information generation unit (generation unit) that generates first diagnostic support information from the medical images by predetermined computer processing (e.g., AI processing). The control unit 21 also functions as an output unit that outputs the first diagnostic support information and the second diagnostic support information to an external device in an identifiable manner.

[0069] In some cases, a single medical image may contain both CAD information generated by AI processing and information entered by a technician. Depending on the content and nature of the diagnostic support information, it may be difficult to determine whether it is CAD information or information entered by a technician. For example, in ultrasound and mammography imaging, the technician's skill is highly valued. Therefore, a system where the technician verifies whether the results of computer processing (e.g., AI processing) are appropriate is conceivable. In such a system, it is desirable that physicians be able to identify which technician-input information is more reliable. Furthermore, technicians may not be concerned with CAD information, or they may be preoccupied with other aspects. If it is not possible to distinguish between CAD information and technician-input information, doctors will not be able to reflect these technician judgments in their diagnoses. In this embodiment, the console 2 outputs both types of information to the diagnostic device 4, etc., in a way that allows for identification. Therefore, a physician viewing the diagnostic support information can determine whether it is CAD information or technician input information without misunderstanding. This enables the deriving of a correct diagnosis from the diagnostic support information.

[0070] The display unit 24 of console 2 displays medical images and first diagnostic support information on the same screen. Therefore, the technician can input second diagnostic support information while confirming the first diagnostic support information.

[0071] Furthermore, the diagnostic device 4, which acquires both CAD information and technician input information, displays on the display unit 44 whether the information is CAD information or technician input information, making it identifiable. This allows physicians performing diagnoses to correctly interpret the diagnostic support information and utilize it to their advantage.

[0072] The first and second diagnostic support information can be displayed side-by-side on a single screen, or they can be switched between. In this case, it is easy to compare the two and identify similarities and differences. The first and second diagnostic support information can be superimposed on a single medical image. In this case, the two can be easily compared, which can be useful for diagnosis.

[0073] Furthermore, the diagnostic device 4 may generate third diagnostic support information from the first diagnostic support information and the second diagnostic support information. In this case, the display unit 44 may display the third diagnostic support information. This allows the physician performing the diagnosis to be provided with more specific information, and increases the amount of information available for making decisions during the diagnosis.

[0074] The descriptions in the above embodiments are examples of information processing devices, diagnostic devices, medical systems, programs, and information processing methods related to this disclosure, and are not limited thereto. The detailed configuration and detailed operation of each device constituting the system can also be modified as appropriate without departing from the spirit of this disclosure.

[0075] For example, in the above embodiment, an example was given in which the control unit 21 of the console 2, which is an information processing device, obtains first diagnostic support information, which is CAD information, through AI analysis. However, the predetermined computer processing that generates the first diagnostic support information (CAD-based diagnostic support information) is not limited to AI analysis.

[0076] For example, the generation of diagnostic support information using CAD may also be done by rule-based algorithms that take into account differences in concentration, shape characteristics, etc. In this case, the detection of suspected lesions (abnormal areas) is performed according to rules based on specific patterns or criteria. For example, abnormalities are detected based on physical characteristics such as differences in density and shape within the image, edge detection, size abnormalities, and positional relationships. Furthermore, the generation of diagnostic support information using CAD may also be performed by thresholding. In this case, areas where brightness or density exceeds a certain standard are detected as suspected lesions (abnormal areas) based on threshold settings. For example, a nodule exceeding a certain size could be judged as abnormal. Furthermore, the generation of diagnostic support information using CAD may also be achieved through shape analysis and basic statistical methods. In this case, a simple statistical analysis is used to capture characteristics by utilizing differences in shape and density. This makes it easier to identify suspected lesions (abnormal areas).

[0077] The following advantages exist for processing methods that do not use AI, as described above. First, without using AI, the computational resources and time required for image processing are reduced. Therefore, processing can be done efficiently. Furthermore, because the criteria and rules for processing are clear, the results tend to be easy to understand and interpret by doctors.

[0078] On the other hand, processing without using AI has the following limitations: In other words, it lacks the ability to recognize complex patterns like deep learning in AI. This increases the likelihood of oversights and false positives. Furthermore, because the rules and standards are fixed, there are limitations in detecting new symptoms or subtle abnormalities. However, non-AI-based processing methods, such as the rule-based methods described above, remain useful for diagnostic assistance and use under specific conditions. Therefore, this disclosure does not exclude these methods from the predetermined computer processing that generates the first diagnostic support information.

[0079] Furthermore, although the above embodiment illustrates an example where a chest image is used as the medical image, the imaging site is not limited to the chest. In addition, the medical image to be processed may be multiple slice images obtained by CT, multi-frame images (video), or images obtained by 3D conversion of images obtained by CT or MRI.

[0080] Furthermore, the programs for executing each process in each device may be stored on a portable recording medium. Alternatively, a carrier wave may be used as the medium for providing the program data via a communication line. [Explanation of Symbols]

[0081] 1 Modality 2. Console (information processing device) 21 Control Unit 22 Memory section 23 Control section 24 Display 24a display screen 4. Diagnostic equipment 41 Control Unit 42 Storage section 43 Operation section 44 Display section 44a display screen 100 Medical Systems N Communication Network

Claims

1. A control unit that controls the acquisition of medical images, At least a display unit for displaying the medical image, A first information generation unit generates first diagnostic support information from the medical image by predetermined computer processing, An input unit for inputting second diagnostic support information regarding the aforementioned medical image, An output unit that outputs the first diagnostic support information and the second diagnostic support information to an external device in a distinguishable manner, Equipped with, An information processing device characterized by the following:

2. The display unit displays the medical image and the first diagnostic support information on the same screen. The second diagnostic support information is information relating to the first diagnostic support information. The information processing apparatus according to feature 1.

3. An acquisition unit that acquires first diagnostic support information generated by predetermined computer processing on a medical image, and second diagnostic support information which is input with respect to the medical image and is identifiable as the first diagnostic support information, At least a display unit for displaying the medical image, Equipped with, A diagnostic device characterized by the following features.

4. The display unit displays the first diagnostic support information and the second diagnostic support information in a way that allows them to be identified. The diagnostic device according to feature 3.

5. The display unit displays either the first diagnostic support information or the second diagnostic support information based on predetermined conditions. The diagnostic device according to feature 3.

6. The system includes a third information generation unit that generates a third diagnostic support information from the first diagnostic support information and the second diagnostic support information based on predetermined conditions, The display unit displays the third diagnostic support information generated by the third information generation unit. The diagnostic device according to feature 3.

7. The display unit displays the difference between the first diagnostic support information and the second diagnostic support information in a way that allows for identification. The diagnostic device according to any one of claims 3 to 6.

8. The display unit displays the first diagnostic support information and the second diagnostic support information side by side on the display screen. The diagnostic device according to feature 4.

9. The display unit displays, side by side, a medical image in which the first diagnostic support information is superimposed on the medical image, and a medical image in which the second diagnostic support information is superimposed on the medical image, or switches between displaying them. The diagnostic device according to feature 4.

10. The display unit displays a medical image in which the first diagnostic support information and the second diagnostic support information are superimposed on the medical image. The diagnostic device according to feature 4.

11. The display unit displays the first diagnostic support information and the second diagnostic support information in a distinguishable manner by changing at least one of the characters, color, size, font, and shape of the first diagnostic support information and the second diagnostic support information. The diagnostic device according to claim 4, characterized in that...

12. A medical system having the information processing device described in claim 1, An acquisition unit that acquires the first diagnostic support information and the second diagnostic support information, The diagnostic device further comprises a second display unit for displaying at least medical images, A medical system characterized by the following.

13. The second display unit further displays the first diagnostic support information and the second diagnostic support information in a distinguishable manner. The medical system according to claim 12, characterized by the features described herein.

14. The second display unit displays either the first diagnostic support information or the second diagnostic support information based on predetermined conditions. The medical system according to claim 12, characterized by the features described herein.

15. The second display unit displays a third diagnostic support information derived from the first diagnostic support information and the second diagnostic support information based on predetermined conditions. The medical system according to claim 12, characterized in that it is a medical system.

16. In the computer of a device equipped with a display unit, A shooting control function that controls the acquisition of medical images, At least a display control function that causes the medical image to be displayed on the display unit, A generation function that generates first diagnostic support information from the aforementioned medical image through predetermined computer processing, An input reception function that accepts input of second diagnostic support information related to the aforementioned medical image, An output function that outputs the first diagnostic support information and the second diagnostic support information to an external device in an identifiable manner, To execute A program characterized by the following features.

17. A shooting control step that controls the acquisition of medical images, At least a display control step of displaying the medical image on the display unit, A generation step of generating first diagnostic support information from the medical image by predetermined computer processing, An input acceptance step for receiving input of second diagnostic support information relating to the aforementioned medical image, The system includes an information output step of outputting the first diagnostic support information and the second diagnostic support information to an external device in a way that allows them to be identified, An information processing method characterized by the following:

Images