Medical imaging device, medical imaging method, and medical imaging program

Abstract

This medical image device performs control so as to: acquire a medical image, information representing multiple interest regions included in the medical image, and the attribute of each of the multiple interest regions; select at least one interest region among the multiple interest regions; and display information pertaining to an interest region other than the selected interest region on the basis of the attribute of the selected interest region.

Description

Technical Field

[0005] ,

[0001] The present disclosure relates to a medical imaging device, a medical imaging method, and a medical imaging program.

Background Art

[0002] International Publication No. 2020 / 209382 discloses a technique for detecting a plurality of findings representing characteristics of abnormal shadows included in a medical image, identifying at least one finding to be used for creating a radiology report from the detected findings, and creating a radiology report using the identified findings.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, in the technique described in International Publication No. 2020 / 209382, when a medical image includes a large number of regions of interest, in order to create a medical document such as a radiology report, a doctor has to specify each region of interest, which is time-consuming for the doctor. That is, in the technique described in International Publication No. 2020 / 209382, when a medical image includes a large number of regions of interest, it may not be possible to appropriately support the creation of a medical document.

[0004] The present disclosure has been made in view of the above circumstances, and an object thereof is to provide a medical imaging device, a medical imaging method, and a medical imaging program that can appropriately support the creation of a medical document even when a medical image includes a large number of regions of interest.

Means for Solving the Problems

[0005] The medical imaging device of the present disclosure is a medical imaging device including at least one processor. The processor acquires a medical image, information representing a plurality of regions of interest included in the medical image, and attributes of each of the plurality of regions of interest, selects at least one region of interest from the plurality of regions of interest, and performs control to display information regarding regions of interest other than the selected region of interest based on the attributes of the selected region of interest.

[0006] Furthermore, the medical imaging device of this disclosure may have a processor that controls the display of information about regions of interest with the same attributes as the selected region of interest.

[0007] Furthermore, the medical imaging device of this disclosure may have a processor that controls the display of information about a region of interest with attributes different from those of the selected region of interest.

[0008] Furthermore, the medical imaging device of this disclosure may perform control such that the processor displays information about regions of interest with attributes different from those of the selected region of interest if the number of regions of interest with the same attributes as the selected region of interest is greater than or equal to a threshold.

[0009] Furthermore, the medical imaging device of this disclosure may have a region of interest that includes a lesion, an attribute that includes whether the lesion is benign or malignant, and the processor may perform control to display information about a region of interest that includes a malignant lesion when the selected region of interest includes a benign lesion and the number of regions of interest that include benign lesions is greater than or equal to a threshold.

[0010] Furthermore, the medical imaging device of this disclosure may have a processor that performs control to highlight a region of interest as a control to display information related to the region of interest.

[0011] Furthermore, the medical imaging device of this disclosure may have a processor that, as a control for displaying information about regions of interest, displays information indicating that there are regions of interest with attributes different from those of the selected region of interest.

[0012] Furthermore, the medical imaging device of this disclosure may have a processor that displays additional information indicating that the attributes of regions of interest other than the selected region of interest are different from those previously detected.

[0013] Furthermore, the medical imaging method of this disclosure involves a processor in a medical imaging device that acquires a medical image, information representing multiple regions of interest contained in the medical image, and the attributes of each of the multiple regions of interest, selects at least one region of interest from among the multiple regions of interest, and performs a process to control the display of information about regions of interest other than the selected region of interest based on the attributes of the selected region of interest.

[0014] Furthermore, the medical image program of this disclosure is intended to cause a processor in a medical image device to perform a process that acquires a medical image, information representing multiple regions of interest contained in the medical image, and the attributes of each of the multiple regions of interest, selects at least one region of interest from among the multiple regions of interest, and displays information about the regions of interest other than the selected region based on the attributes of the selected region of interest.

[0015] Furthermore, the medical imaging apparatus of this disclosure is a medical imaging apparatus comprising at least one processor, the processor acquires a medical image, information representing a plurality of regions of interest contained in the medical image, and the attributes of each of the plurality of regions of interest, selects at least one region of interest from the plurality of regions of interest, and generates a finding statement for a region of interest with the same attributes as the selected region of interest.

[0016] Furthermore, the medical imaging method of this disclosure involves a processor in a medical imaging device that acquires a medical image, information representing multiple regions of interest contained in the medical image, and the attributes of each of the multiple regions of interest, selects at least one region of interest from among the multiple regions of interest, and generates a finding statement for a region of interest with the same attributes as the selected region of interest.

[0017] Furthermore, the medical image program of this disclosure is intended to cause a processor in a medical image device to perform the following processes: acquire a medical image, information representing multiple regions of interest contained in the medical image, and the attributes of each of the multiple regions of interest; select at least one region of interest from among the multiple regions of interest; and generate a finding statement for a region of interest with the same attributes as the selected region of interest. [Effects of the Invention]

[0018] According to the present disclosure, even when a medical image includes a large number of regions of interest, the creation of a medical document can be appropriately supported.

Brief Description of the Drawings

[0019] [Figure 1] It is a block diagram showing a schematic configuration of a medical information system. [Figure 2] It is a block diagram showing an example of a hardware configuration of a medical imaging device. [Figure 3] It is a block diagram showing an example of a functional configuration of a medical imaging device according to the first and second embodiments. [Figure 4] It is a diagram for explaining a process of extracting a lesion. [Figure 5] It is a diagram for explaining a process of deriving a name of a lesion. [Figure 6] It is a diagram showing an example of a screen in which a lesion is highlighted. [Figure 7] It is a flowchart showing an example of a lesion display process according to the first embodiment. [Figure 8] It is a diagram for explaining a process of deriving an attribute of a lesion. [Figure 9] It is a diagram showing an example of a screen in which a lesion is highlighted. [Figure 10] It is a flowchart showing an example of a lesion display process according to the second embodiment. [Figure 11] It is a block diagram showing an example of a functional configuration of a medical imaging device according to the third embodiment. [Figure 12] It is a diagram for explaining a process of deriving a name and a finding of a lesion. [Figure 13] It is a diagram for explaining a process of generating a finding sentence. [Figure 14] It is a flowchart showing an example of a finding sentence generation process according to the third embodiment. [Figure 15] It is a diagram showing an example of a screen in which a lesion according to a modification is highlighted. [Modes for carrying out the invention]

[0020] Hereinafter, with reference to the drawings, examples of embodiments for carrying out the technology of this disclosure will be described in detail.

[0021] [First Embodiment] First, with reference to Figure 1, the configuration of Medical Information System 1, to which the medical imaging device relating to the disclosed technology is applied, will be explained. Medical Information System 1 is a system for taking images of the diagnostic target area of ​​a subject and storing the medical images obtained from the images, based on examination orders from physicians in clinical departments using a known ordering system. Medical Information System 1 is also a system for radiologists to interpret medical images and create interpretation reports, and for physicians in the requesting clinical departments to view interpretation reports and perform detailed observations of the medical images being interpreted.

[0022] As shown in Figure 1, the medical information system 1 according to this embodiment includes multiple imaging devices 2, multiple image interpretation workstations (WS) 3 which are image interpretation terminals, clinical department WS4, image server 5, image database (DataBase: DB) 6, image interpretation report server 7, and image interpretation report DB8. The imaging devices 2, image interpretation WS3, clinical department WS4, image server 5, and image interpretation report server 7 are connected to each other via a wired or wireless network 9, enabling them to communicate with one another. In addition, image DB6 is connected to image server 5, and image interpretation report DB8 is connected to image interpretation report server 7.

[0023] The imaging device 2 is a device that generates a medical image representing the diagnostic target area of ​​a subject by imaging that area. The imaging device 2 may be, for example, a simple X-ray imaging device, an endoscope, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, or a PET (Positron Emission Tomography) device. The medical image generated by the imaging device 2 is transmitted to and stored in the image server 5.

[0024] The Department WS4 is a computer used by physicians in a clinical department for detailed observation of medical images, viewing of image interpretation reports, and creation of electronic medical records. In the Department WS4, the creation of patient electronic medical records, requests for image viewing from the image server 5, and the display of medical images received from the image server 5 are performed by executing software programs for each process. In addition, the Department WS4 performs processes such as automatic detection or highlighting of disease-prone areas in medical images, requests for viewing of image interpretation reports from the image interpretation report server 7, and the display of image interpretation reports received from the image interpretation report server 7, by executing software programs for each process.

[0025] Image server 5 incorporates a software program that provides database management system (DBMS) functionality to a general-purpose computer. When image server 5 receives a request to register a medical image from imaging device 2, it formats the medical image into a database format and registers it in image DB6.

[0026] Image DB6 stores image data representing medical images acquired by imaging device 2, along with associated information. This associated information includes, for example, an image ID (identification) to identify individual medical images, a patient ID to identify the patient being studied, an examination ID to identify the examination content, and a unique ID (UID: unique identification) assigned to each medical image. It also includes information such as the examination date and time the medical image was generated, the type of imaging device used to acquire the image, patient information (e.g., patient's name, age, and gender), examination site (i.e., imaging site), imaging information (e.g., imaging protocol, imaging sequence, imaging method, imaging conditions, and whether contrast agent was used), and series number or acquisition number when multiple medical images are acquired in a single examination. Furthermore, when image server 5 receives a viewing request from image interpretation WS3 via network 9, it searches for medical images registered in image DB6 and sends the retrieved medical images to the requesting image interpretation WS3.

[0027] The image interpretation report server 7 incorporates a software program that provides DBMS functionality to a general-purpose computer. When the image interpretation report server 7 receives a registration request for an image interpretation report from the image interpretation WS3, it formats the image interpretation report into a database format and registers it in the image interpretation report database 8. Furthermore, when it receives a search request for an image interpretation report, it searches for that report in the image interpretation report database 8.

[0028] The image interpretation report DB8 stores image interpretation reports that include information such as an image ID to identify the medical image being interpreted, a radiologist ID to identify the radiologist who performed the interpretation, the name of the lesion, the location of the lesion, findings, and the confidence level of the findings.

[0029] Network 9 is a wired or wireless local area network that connects various devices within the hospital. If the image interpretation WS3 is installed in another hospital or clinic, Network 9 may be configured to connect the local area networks of each hospital via the Internet or a dedicated line. In either case, it is preferable that Network 9 be configured to enable high-speed transfer of medical images, such as through an optical network.

[0030] The image interpretation WS3 performs the following: requests to view medical images from the image server 5, various image processing on medical images received from the image server 5, display of medical images, analysis processing of medical images, highlighting of medical images based on the analysis results, and creation of image interpretation reports based on the analysis results. In addition, the image interpretation WS3 assists in the creation of image interpretation reports, requests registration and viewing of image interpretation reports from the image interpretation report server 7, and displays image interpretation reports received from the image interpretation report server 7. The image interpretation WS3 performs each of the above processes by executing software programs for each process. The image interpretation WS3 incorporates the medical imaging device 10, which will be described later, and since the processes other than those performed by the medical imaging device 10 are performed by well-known software programs, a detailed explanation is omitted here. Alternatively, instead of performing processes other than those performed by the medical imaging device 10 in the image interpretation WS3, a separate computer that performs such processes may be connected to the network 9, and the computer may perform the requested processes in response to processing requests from the image interpretation WS3. The following provides a detailed description of the medical imaging device 10 included in the image interpretation WS3.

[0031] Next, the hardware configuration of the medical imaging device 10 according to this embodiment will be described with reference to Figure 2. As shown in Figure 2, the medical imaging device 10 includes a CPU (Central Processing Unit) 20, a memory 21 as a temporary storage area, and a non-volatile storage unit 22. The medical imaging device 10 also includes a display 23 such as a liquid crystal display, input devices 24 such as a keyboard and mouse, and a network interface 25 connected to the network 9. The CPU 20, memory 21, storage unit 22, display 23, input devices 24, and network interface 25 are connected to the bus 27.

[0032] The storage unit 22 is implemented by an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, etc. The medical image program 30 is stored in the storage unit 22 as a storage medium. The CPU 20 reads the medical image program 30 from the storage unit 22, expands it into memory 21, and executes the expanded medical image program 30.

[0033] Next, with reference to Figure 3, the functional configuration of the medical imaging device 10 according to this embodiment will be described. As shown in Figure 3, the medical imaging device 10 includes an acquisition unit 40, an extraction unit 42, an analysis unit 44, a selection unit 46, and a display control unit 48. The CPU 20 executes the medical image program 30, thereby enabling the acquisition unit 40, the extraction unit 42, the analysis unit 44, the selection unit 46, and the display control unit 48 to function.

[0034] The acquisition unit 40 acquires the medical image to be diagnosed (hereinafter referred to as the "diagnostic image") from the image server 5 via the network interface 25. In the following explanation, the case in which the diagnostic image is a CT image of the liver will be used as an example.

[0035] The extraction unit 42 extracts the region containing the lesion from the diagnostic image acquired by the acquisition unit 40. Specifically, the extraction unit 42 extracts the region containing the lesion using a trained model M1 for detecting lesions from the diagnostic image. The region containing the lesion in the diagnostic image is an example of the region of interest related to the disclosed technology. The region of interest is not limited to the region containing the lesion, but may also be the region of an organ such as the lungs and liver, or the region of an anatomical structure such as the subsegments of the liver divided into S1 to S8.

[0036] The trained model M1 is constructed using a Convolutional Neural Network (CNN), for example, which takes a medical image as input and outputs the region containing the lesion within that medical image. The trained model M1 is a model trained using machine learning, for example, by using a large number of combinations of medical images containing lesions and information identifying the region where the lesion exists within that medical image as training data.

[0037] As an example, as shown in Figure 4, the extraction unit 42 inputs the image to be diagnosed into the trained model M1. The trained model M1 outputs information identifying the region in the input image to be diagnosed where a lesion exists. In the example in Figure 4, the area filled with diagonal lines indicates the lesion. The extraction unit 42 may extract the region containing the lesion using a known CAD (Computer-Aided Diagnosis) method, or it may extract a region specified by the user as the region containing the lesion.

[0038] The analysis unit 44 analyzes each lesion extracted by the extraction unit 42 and derives a name for the lesion as an example of the lesion's attributes. Specifically, the analysis unit 44 derives the name of the lesion using a pre-trained model M2 for deriving the name of the lesion. The pre-trained model M2 is, for example, a CNN that takes a medical image containing the lesion and information identifying the region in the medical image where the lesion exists as input, and outputs the name of the lesion. The pre-trained model M2 is a model trained by machine learning using, for example, a large number of combinations of medical images containing the lesion, information identifying the region in the medical image where the lesion exists, and the name of the lesion as training data.

[0039] As an example, as shown in Figure 5, the analysis unit 44 inputs the diagnostic target image and information identifying the region containing lesions extracted from the diagnostic target image by the extraction unit 42 into the trained model M2. The trained model M2 outputs the names of the lesions contained in the input diagnostic target image. Figure 5 shows an example where five lesions are named liver cysts and one lesion is named liver metastasis. Note that the attributes of the lesion are not limited to the name of the lesion, but may also include findings such as location, size, presence or absence of calcification, whether it is benign or malignant, and presence or absence of irregular margins. Furthermore, there may be multiple attributes for the lesion.

[0040] The selection unit 46 selects at least one lesion specified by the user from among the multiple lesions extracted by the extraction unit 42.

[0041] The display control unit 48 obtains information representing multiple lesions contained in the diagnostic target image extracted by the extraction unit 42 from the extraction unit 42. The display control unit 48 also obtains the attributes of each of the multiple lesions derived by the analysis unit 44 from the analysis unit 44. The display control unit 48 may also obtain the information representing multiple lesions contained in the diagnostic target image and the attributes of each of the multiple lesions from an external device such as a clinical department WS4. In this case, the extraction unit 42 and the analysis unit 44 would be provided by the external device.

[0042] The display control unit 48 controls the display 23 to display information representing multiple lesions extracted by the extraction unit 42. The user selects a lesion from among the multiple lesions displayed on the display 23 to be used for creating medical documents such as image interpretation reports. This selected lesion is then selected by the selection unit 46 described above.

[0043] Furthermore, the display control unit 48 controls the display 23 to display information about lesions other than the first lesion (hereinafter referred to as the "second lesion") based on the attributes of the lesion selected by the selection unit 46 (hereinafter referred to as the "first lesion"). In this embodiment, the display control unit 48 controls the display 23 to highlight the first lesion and any lesions of the second lesion that have the same name as the first lesion. As an example, as shown in Figure 6, the display control unit 48 controls the display to highlight the first lesion and any lesions of the first lesion that have the same name as the first lesion by surrounding them with a rectangular frame. In Figure 6, an example of highlighting is shown when one of the liver cyst lesions in Figure 5 (in the example in Figure 6, the lesion indicated by the arrow representing the mouse pointer) is specified by the user. In this way, the user can easily identify lesions with the same name as the lesion that the user has specified as the target for creating a medical document. This makes it easier for the user to create a findings document summarizing the findings of lesions with the same name.

[0044] The display control unit 48 may also control the display to make the name of the lesion identifiable by setting the color of the frame line to a color predetermined according to the name of the lesion. In addition, the display control unit 48 may also control the display to highlight the lesion by, for example, making the lesion blink, adding a predetermined mark, or drawing a line around the outer edge of the lesion area.

[0045] Next, the operation of the medical imaging device 10 according to this embodiment will be explained with reference to Figure 7. The CPU 20 executes the medical imaging program 30, which in turn executes the lesion display process shown in Figure 7. The lesion display process shown in Figure 7 is executed, for example, when a user inputs an instruction to start execution.

[0046] In step S10 of Figure 7, the acquisition unit 40 acquires the diagnostic target image from the image server 5 via the network interface 25. In step S12, the extraction unit 42 extracts the region containing the lesion from the diagnostic target image acquired in step S10, as described above. In step S14, the analysis unit 44 performs an analysis on each of the lesions extracted in step S12, as described above, and derives the name of the lesion.

[0047] In step S16, the display control unit 48 controls the display 23 to display information representing the multiple lesions extracted in step S12. The user specifies the lesion to be used to create a medical document such as an image interpretation report from among the multiple lesions displayed on the display 23. In step S18, the selection unit 46 selects at least one lesion specified by the user from among the multiple lesions.

[0048] In step S20, the display control unit 48 performs the operation described above to highlight the first lesion selected in step S18 and lesions with the same name as the first lesion on the display 23. When the processing in step S20 is completed, the lesion display process is completed.

[0049] As described above, according to this embodiment, even when a medical image contains numerous lesions, lesions with the same attributes as the lesion specified by the user are highlighted, making it easier for the user to create medical documents. Therefore, it is possible to appropriately support the creation of medical documents.

[0050] [Second Embodiment] A second embodiment of the disclosed technology will now be described. Note that the configuration of the medical information system 1 and the hardware configuration of the medical imaging device 10 according to this embodiment are the same as those of the first embodiment, and therefore will not be described.

[0051] Referring to Figure 3, the functional configuration of the medical imaging device 10 according to this embodiment will be described. Functional parts having the same functions as the medical imaging device 10 according to the first embodiment are denoted by the same reference numerals and their description is omitted. As shown in Figure 3, the medical imaging device 10 includes an acquisition unit 40, an extraction unit 42, an analysis unit 44A, a selection unit 46, and a display control unit 48A. The CPU 20 executes the medical image program 30, thereby enabling the acquisition unit 40, extraction unit 42, analysis unit 44A, selection unit 46, and display control unit 48A to function.

[0052] The analysis unit 44A analyzes each lesion extracted by the extraction unit 42 and derives whether the lesion is benign or malignant as an example of the lesion's attributes. Specifically, the analysis unit 44A uses a trained model M3 to derive whether the lesion is benign or malignant. The trained model M3 is composed of a CNN that takes a medical image containing the lesion and information identifying the region in the medical image where the lesion exists as input, and outputs whether the lesion is benign or malignant. The trained model M3 is a model trained by machine learning using, for example, a medical image containing the lesion and information identifying the region in the medical image where the lesion exists, and information indicating whether the lesion is benign or malignant as training data.

[0053] As an example, as shown in Figure 8, the analysis unit 44A inputs the diagnostic target image and information identifying the region where lesions are present, extracted from the diagnostic target image by the extraction unit 42, into the trained model M3. The trained model M3 outputs whether the lesions included in the input diagnostic target image are benign or malignant. Figure 8 shows an example where five lesions are benign and one lesion is malignant.

[0054] The display control unit 48A, similar to the display control unit 48 in the first embodiment, controls the display 23 to display information representing multiple lesions extracted by the extraction unit 42.

[0055] Furthermore, the display control unit 48A controls the display 23 to display information about the second lesions other than the first lesion, based on the attributes of the first lesion selected by the selection unit 46. In this embodiment, the display control unit 48A controls the display 23 to highlight lesions among the second lesions that have different attributes from those of the first lesion. The method of highlighting is the same as in the first embodiment, so a detailed explanation is omitted. As an example, as shown in Figure 9, lesions with different attributes from those specified by the user are highlighted by the control of the display control unit 48A. Note that Figure 9 shows an example of highlighting when one of the benign lesions in Figure 8 (in the example in Figure 9, the lesion indicated by the arrow representing the mouse pointer) is specified by the user.

[0056] Furthermore, the display control unit 48A may perform control to highlight lesions with attributes different from those of the first lesion on the display 23 if the number of lesions with the same attributes as the first lesion selected by the selection unit 46 is equal to or greater than the threshold TH. Specifically, the display control unit 48A may perform control to highlight malignant lesions on the display 23 if the first lesion selected by the selection unit 46 is a benign lesion and the number of benign lesions is equal to or greater than the threshold TH. This can suppress the oversight of malignant lesions by the user due to the presence of numerous benign lesions in the medical image.

[0057] Furthermore, the display control unit 48A may control the display 23 to display information indicating the presence of lesions with attributes different from those of the first lesion, if the number of lesions with the same attributes as the first lesion selected by the selection unit 46 is greater than or equal to the threshold TH. Specifically, the display control unit 48A may control the display 23 to display information indicating the presence of malignant lesions, if the first lesion selected by the selection unit 46 is a benign lesion and the number of benign lesions is greater than or equal to the threshold TH. As a result, the user can understand that malignant lesions are present in the medical image, thereby suppressing the oversight of malignant lesions by the user due to the presence of numerous benign lesions in the medical image.

[0058] Furthermore, the display control unit 48A may perform control to display information indicating that the attributes of a second lesion other than the first lesion selected by the selection unit 46 are different from those of a lesion detected in the past. Specifically, as an example shown in Figure 15, the display control unit 48A performs control to highlight on the display 23 any second lesions with attributes different from those of the first lesion, as described above. In this example, the display control unit 48A performs control to display information indicating that the attributes of a second lesion are different from those of a lesion detected in the past. In Figure 15, similar to Figure 9, one of the benign lesions is specified by the user, and an example is shown where a lesion with different attributes from the specified lesion, i.e., a malignant lesion, had benign attributes when it was detected last time. Also in Figure 15, an example is shown where text indicating that the lesion was benign in the previous examination is displayed as information indicating that the attributes are different. In this example, the user can understand that the lesion has changed from benign to malignant.

[0059] Next, the operation of the medical imaging device 10 according to this embodiment will be explained with reference to Figure 10. The CPU 20 executes the medical imaging program 30, which in turn executes the lesion display process shown in Figure 10. The lesion display process shown in Figure 10 is executed, for example, when a user inputs an instruction to start execution. Steps in Figure 10 that perform the same process as in Figure 7 are given the same step numbers and their explanations are omitted.

[0060] In step S14A of Figure 10, the analysis unit 44A performs an analysis on each of the lesions extracted in step S12, as described above, and derives whether the lesion is benign or malignant.

[0061] In step S20A, the display control unit 48A, as described above, controls the display 23 to highlight lesions among the second lesions that have different attributes from the first lesion selected in step S18. When the processing in step S20A is completed, the lesion display process is completed.

[0062] As described above, according to this embodiment, it is possible to appropriately support the creation of medical documents even when medical images contain a large number of regions of interest.

[0063] [Third Embodiment] A third embodiment of the disclosed technology will now be described. Note that the configuration of the medical information system 1 and the hardware configuration of the medical imaging device 10 according to this embodiment are the same as those of the first embodiment, and therefore will not be described.

[0064] Referring to Figure 11, the functional configuration of the medical imaging device 10 according to this embodiment will be described. Functional parts having the same functions as the medical imaging device 10 according to the first embodiment are denoted by the same reference numerals and their description is omitted. As shown in Figure 11, the medical imaging device 10 includes an acquisition unit 40, an extraction unit 42, an analysis unit 44B, a selection unit 46, a display control unit 48B, and a generation unit 50. The CPU 20 executes the medical image program 30, thereby enabling the acquisition unit 40, extraction unit 42, analysis unit 44B, selection unit 46, display control unit 48B, and generation unit 50 to function.

[0065] The analysis unit 44B analyzes each lesion extracted by the extraction unit 42 and derives the name of the lesion as an example of the lesion's attributes. Furthermore, the analysis unit 44B analyzes each lesion extracted by the extraction unit 42 and derives findings of the lesion. Below, to make the explanation easier to understand, an example in which size is applied as a finding will be described. An example of lesion size is the longest diameter of the lesion.

[0066] Specifically, the analysis unit 44B derives the name and findings of a lesion using a pre-trained model M4 for deriving the name and findings of the lesion. The pre-trained model M4 is composed of a CNN that takes, for example, a medical image containing a lesion and information identifying the region in the medical image in which the lesion is located as input, and outputs the name and findings of the lesion. The pre-trained model M4 is a model trained by machine learning using, for example, a large number of combinations of medical images containing a lesion, information identifying the region in the medical image in which the lesion is located, and the name and findings of the lesion as training data.

[0067] As an example, as shown in Figure 12, the analysis unit 44B inputs the diagnostic target image and information identifying the region containing lesions extracted from the diagnostic target image by the extraction unit 42 into the trained model M4. The trained model M4 outputs the names and findings of the lesions contained in the input diagnostic target image. Figure 12 shows an example where five lesions are named hepatic cysts and one lesion is named liver metastasis. Figure 12 also shows the derived size for each lesion.

[0068] As an example, as shown in Figure 13, the generation unit 50 generates a findings statement that summarizes the findings of lesions with the same name as the lesion selected by the selection unit 46. In Figure 13, one of the five liver cyst lesions (indicated by the arrow representing the mouse pointer in the example in Figure 13) is specified by the user, and an findings statement summarizing the findings of the five liver cysts with the same name as the specified lesion is generated.

[0069] For example, the generation unit 50 generates a report by inputting the name of the lesion selected by the selection unit 46 and the findings of lesions with the same name into a recurrent neural network that has been trained to generate text from input words.

[0070] The display control unit 48B, similar to the display control unit 48 in the first embodiment, controls the display 23 to show information representing multiple lesions extracted by the extraction unit 42. The display control unit 48B also controls the display 23 to show the findings text generated by the generation unit 50.

[0071] Next, the operation of the medical imaging device 10 according to this embodiment will be explained with reference to Figure 14. The CPU 20 executes the medical imaging program 30, which in turn executes the findings statement generation process shown in Figure 14. The findings statement generation process shown in Figure 14 is executed, for example, when a user inputs an instruction to start execution. Steps in Figure 14 that perform the same process as in Figure 7 are given the same step numbers and their explanations are omitted.

[0072] In step S14B of Figure 14, the analysis unit 44B performs an analysis on each of the lesions extracted in step S12, as described above, and derives the name and findings of the lesion.

[0073] In step S22, the generation unit 50 generates a findings statement summarizing the findings of lesions with the same name as the lesion selected in step S18, as described above. In step S24, the display control unit 48B controls the display 23 to show the findings statement generated in step S22. When the processing in step S24 is completed, the findings statement generation process ends.

[0074] As described above, according to this embodiment, it is possible to appropriately support the creation of medical documents even when medical images contain a large number of regions of interest.

[0075] In each of the above embodiments, the hardware structure of the processing unit that performs various processes, such as the various functional parts of the medical imaging device 10, can be the following types of processors. These processors include, as mentioned above, a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), which has a circuit configuration that can be changed after manufacturing, and an ASIC (Application Specific Integrated Circuit), which has a circuit configuration specifically designed to perform a particular process. It includes a dedicated electrical circuit, etc.

[0076] A single processing unit may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, multiple processing units may be composed of a single processor.

[0077] Examples of configuring multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software combine to form a single processor, which then functions as multiple processing units, as exemplified by client and server computers. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as exemplified by System on Chip (SoC). Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned processors.

[0078] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits, which are combinations of circuit elements such as semiconductor devices.

[0079] Furthermore, although the above embodiments describe a configuration in which the medical image program 30 is pre-stored (installed) in the storage unit 22, the invention is not limited thereto. The medical image program 30 may be provided in the form of a recording medium such as a CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), or USB (Universal Serial Bus) memory. Alternatively, the medical image program 30 may be provided in the form of a download from an external device via a network.

[0080] The disclosures of Japanese Patent Application No. 2021-065375, filed on April 7, 2021, and Japanese Patent Application No. 2021-208525, filed on December 22, 2021, are incorporated herein by reference in their entirety. Furthermore, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if the incorporation of each individual document, patent application, and technical standard were specifically and individually noted.

Claims

1. A medical imaging device comprising at least one processor, The aforementioned processor, A medical image, information representing multiple regions of interest contained in the medical image, and the attributes of each of the multiple regions of interest are obtained. Select at least one of the aforementioned multiple areas of interest, If the number of regions of interest with the same attributes as the selected region of interest exceeds a threshold, the system will display information about regions of interest with different attributes than the selected region of interest. Medical imaging equipment.

2. The aforementioned region of interest is the region containing the lesion. The aforementioned attributes include whether the lesion is benign or malignant. The aforementioned processor, If the selected region of interest includes a benign lesion, and the number of regions of interest containing benign lesions is equal to or greater than the threshold, control is performed to display information about regions of interest containing malignant lesions. The medical imaging apparatus according to claim 1.

3. The aforementioned processor, As a control for displaying information related to the aforementioned region of interest, a control is performed to highlight the aforementioned region of interest. A medical imaging apparatus according to claim 1 or claim 2.

4. The aforementioned processor, As a control for displaying information about the aforementioned region of interest, the control displays information indicating that there are regions of interest with attributes different from the selected region of interest. A medical imaging apparatus according to any one of claims 1 to 3.

5. The aforementioned processor, For regions of interest other than the selected region of interest, if their attributes differ from those detected in the past, control is implemented to display additional information indicating these attribute differences. A medical imaging apparatus according to any one of claims 1 to 4.

6. A medical image, information representing multiple regions of interest contained in the medical image, and the attributes of each of the multiple regions of interest are obtained. Select at least one of the aforementioned multiple areas of interest, If the number of regions of interest with the same attributes as the selected region of interest exceeds a threshold, the system will display information about regions of interest with different attributes than the selected region of interest. A medical imaging method in which processing is performed by a processor installed in a medical imaging device.

7. A medical image, information representing multiple regions of interest contained in the medical image, and the attributes of each of the multiple regions of interest are obtained. Select at least one of the aforementioned multiple areas of interest, If the number of regions of interest with the same attributes as the selected region of interest exceeds a threshold, the system will display information about regions of interest with different attributes than the selected region of interest. A medical image program that causes the processor in a medical imaging device to perform the processing.

8. The aforementioned region of interest is the region containing the lesion. The medical imaging apparatus according to claim 1.

Images