Diagnosis explanation assistance device, diagnosis explanation assistance method, and recording medium

Abstract

In this diagnosis explanation assistance device, an acquisition means acquires a target image to be the subject of explanation assistance. A line drawing creation means creates a line drawing from the target image by using a first machine learning model trained by learning data including case images and correct answer data constituted of line drawings of the case images. An output means outputs the target image and a line drawing of the target image. The diagnosis explanation assistance device is able to assist decision making of users in the medical field.

Description

Diagnostic explanation support device, diagnostic explanation support method, and recording medium

[0001] The present disclosure relates to techniques for aiding in diagnostic interpretation.

[0002] In endoscopic diagnosis or pathological diagnosis, an explanation of the diagnosis result may be required. In response to this, Patent Document 1 discloses a method for presenting the diagnosis result regarding a disease along with the reason for the diagnosis.

[0003] Japanese Patent Application Laid-Open No. 2020-089710

[0004] When explaining diagnostic results, photographs of the area on which the diagnosis is based are sometimes used, but it is difficult to explain characteristic parts in photographs. Furthermore, medical books on diagnostic standards often use line drawings that highlight characteristic parts.

[0005] One object of the present disclosure is to provide a diagnosis explanation support device that generates a line drawing that reflects a case.

[0006] In one aspect of the present disclosure, a diagnosis explanation support device includes: an acquisition means for acquiring a target image that is a target of explanation support; a line drawing creation means for creating a line drawing from the target image using a first machine learning model trained with training data including case images and ground truth data that is a line drawing of the case images; and an output means for outputting the target image and the line drawing of the target image.

[0007] In another aspect of the present disclosure, a pathology diagnosis support method includes: acquiring a target image that is a target for explanation support; creating a line drawing from the target image using a first machine learning model trained with training data including case images and ground truth data that is a line drawing of the case images; and outputting the target image and the line drawing of the target image.

[0008] In yet another aspect of the present disclosure, a recording medium records a program that causes a computer to execute a process of acquiring a target image that is the subject of explanation assistance, creating a line drawing from the target image using a first machine learning model trained with training data including case images and correct answer data that is a line drawing of the case image, and outputting the target image and the line drawing of the target image.

[0009] According to the present disclosure, it is possible to provide a diagnosis explanation support device that generates a line drawing that reflects a case.

[0010] 1 is a diagram conceptually illustrating a diagnosis explanation support device according to the present disclosure. FIG. 2 is a block diagram illustrating a hardware configuration of a diagnosis explanation support device according to the present disclosure. FIG. 3 is a block diagram illustrating a functional configuration of a diagnosis explanation support device according to the present disclosure. FIG. 4 is a flowchart of a diagnosis explanation support process. FIG. 5 is a block diagram illustrating a functional configuration of another diagnosis explanation support device according to the present disclosure. FIG. 6 is a diagram for explaining method 1. FIG. 7 is a diagram illustrating an example of display data by a line drawing comparison unit. FIG. 8 is a diagram for explaining method 2. FIG. 9 is a diagram illustrating another example of display data by a line drawing comparison unit. FIG. 10 is a block diagram illustrating a functional configuration of another diagnosis explanation support device according to the present disclosure. FIG. 11 is a diagram illustrating an example of line drawing division. FIG. 12 is an example of display data by a line drawing similarity calculation unit. FIG. 13 is a block diagram illustrating a functional configuration of another diagnosis explanation support device according to the present disclosure. FIG. 14 is a flowchart of processing by another diagnosis explanation support device according to the present disclosure.

[0011] Preferred embodiments of the present disclosure will be described below with reference to the drawings. First Embodiment Overall Configuration Fig. 1 is a diagram conceptually illustrating a diagnosis explanation support device. The diagnosis explanation support device 10 generates a line drawing based on an input image. The image is used to explain a diagnosis, and is, for example, an image of a region that is the basis of the diagnosis (hereinafter also referred to as a "diagnostic region"). The image of the diagnostic region is extracted from an endoscopic image or a pathology slide image by AI (artificial intelligence) or a doctor.

[0012] 2 is a block diagram showing the hardware configuration of the diagnosis explanation support device 10. As shown in the figure, the diagnosis explanation support device 10 includes a processor 11, an interface (IF) 12, a ROM (Read Only Memory) 13, a RAM (Random Access Memory) 14, a database (DB) 15, a recording medium 16, an input unit 17, and a display unit 18. The components are connected to each other via a bus 19, for example.

[0013] The processor 11 is a computer such as a CPU (Central Processing Unit), and executes a prepared program to control the entire diagnosis explanation support device 10. Specifically, the processor 11 may be a CPU, a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating Point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof.

[0014] The processor 11 also loads programs stored in the ROM 13, the recording medium 16, etc., and executes each process coded in the program. The processor 11 also functions as a part or the whole of the diagnosis explanation support device 10. The processor 11 then executes a line drawing creation process, which will be described later.

[0015] The IF 12 inputs and outputs data to and from an external device. Specifically, an image of a diagnostic region is input to the diagnosis explanation support device 10 through the IF 12.

[0016] The ROM 13 stores various programs executed by the processor 11. The RAM 14 is used as a working memory while the processor 11 is executing various processes.

[0017] DB15 stores, as needed, images of diagnostic regions input from an external device and the line drawing creation AI described below. DB15 also stores line drawings published in medical textbooks (hereinafter also referred to as "example line drawings"). For example, DB15 stores cases and example line drawings of those cases in association with each other.

[0018] The recording medium 16 is a non-volatile, non-temporary storage device such as a disk-shaped recording medium or a semiconductor memory. The recording medium 16 may be configured to be detachable from the diagnosis explanation support device 10. The recording medium 16 records various programs executed by the processor 11.

[0019] The input unit 17 is, for example, a mouse, a keyboard, etc., and is used by the user to input data. The display unit 18 is, for example, a liquid crystal display device, etc., and displays the created line drawing.

[0020] 3 is a block diagram showing the functional configuration of the diagnosis explanation support device 10 according to the first embodiment. Functionally, the diagnosis explanation support device 10 includes a diagnosis region input unit 101 and a line drawing creation unit 102 in addition to the display unit 18 described above.

[0021] An image of a diagnostic region (hereinafter also referred to as a "target image") is input to the diagnosis explanation support device 10 via the IF 12. The target image is input to the diagnostic region input unit 101. The diagnostic region input unit 101 outputs the target image to the line drawing creation unit 102.

[0022] The line drawing creation unit 102 inputs a target image into the line drawing creation AI and obtains a line drawing of the target image. Note that in this embodiment, a line drawing creation AI is prepared for each case. The line drawing creation unit 102 determines the line drawing creation AI to use based on the case of the diagnostic region. This allows the line drawing creation unit 102 to create a line drawing that reflects the characteristics of the case.

[0023] In this embodiment, tumor cases are used as cases. Tumor cases include, for example, the malignancy of cells (grade of cancer cells), the type of cancer, and the stage of cancer. In addition, the line drawing creation AIs prepared for each case include line drawing creation AIs corresponding to the type of cancer and the grade of cancer cells, such as "line drawing creation AI for XX cancer grade 3."

[0024] The line drawing creation AI is generated by deep learning using pairs of case images and example line drawings of the cases as training data. For example, an autoencoder can be used as an algorithm used in deep learning. A diffusion-based image generation algorithm may also be used as an algorithm used in deep learning. Hereinafter, a line drawing creation AI using a diffusion-based algorithm is also referred to as a "diffusion-based line drawing creation AI." It is assumed that the diffusion-based line drawing creation AI has been trained on a plurality of cases in advance.

[0025] When a diffusion-based line drawing creation AI is used, the line drawing creation unit 102 generates a prompt to be input to the line drawing creation AI based on the case of the diagnostic region. For example, the line drawing creation unit 102 generates a prompt such as "XX cancer, grade 3" based on the case of the diagnostic region. Then, the line drawing creation unit 102 inputs the prompt and the target image into the line drawing creation AI to obtain a line drawing of the target image. The line drawing creation unit 102 may include reference information such as the staining method and the conditions during the examination in the prompt. For example, the line drawing creation unit 102 may generate a prompt such as "XX cancer, grade 3, YY staining, ZZ was performed during the examination." This allows the line drawing creation unit 102 to create a line drawing that better reflects the characteristics of the case.

[0026] The line drawing creation unit 102 outputs the line drawing of the target image to the display unit 18 .

[0027] The display unit 18 displays the line drawing of the target image input from the line drawing creation unit 102. The display unit 18 may also display intermediate case images together with the line drawing of the target image. The intermediate case images are images of the process of generating the line drawing by the line drawing creation AI. In this case, the line drawing creation unit 102 saves the images of the generation process when creating the line drawing and outputs the images of the generation process to the display unit 18.

[0028] In the above configuration, the diagnostic region input unit 101 is an example of an acquisition unit, and the line drawing creation unit 102 is an example of a line drawing creation unit and an output unit.

[0029] [Line Drawing Creation Process] Next, the line drawing creation process will be described. Fig. 4 is a flowchart of the pathological diagnosis support process performed by the diagnosis explanation support device 10. This process is realized by the processor 11 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 3.

[0030] First, a target image is input to the diagnosis explanation support device 10 via the IF 12. The target image is input to the diagnostic region input unit 101 (step S101). The diagnostic region input unit 101 outputs the target image to the line drawing creation unit 102. Next, the line drawing creation unit 102 inputs the target image to the line drawing creation AI and acquires a line drawing of the target image (step S102). The line drawing creation unit 102 outputs the line drawing of the target image to the display unit 18. Next, the display unit 18 displays the line drawing input from the line drawing creation unit 102 (step S103). Then, the processing ends.

[0031] The diagnosis explanation support device 10 according to the first embodiment can create a line drawing that reflects the characteristics of a case from a target image in which the case is defined, thereby enabling users such as doctors to explain the characteristic parts of the case using the line drawing.

[0032] Second Embodiment Next, a second embodiment will be described. A diagnosis explanation support device according to the second embodiment creates a line drawing for a target image for which a case has not been determined, and displays the created line drawing together with an example line drawing. Note that the system configuration and hardware configuration of the diagnosis explanation support device according to the second embodiment are similar to those of the diagnosis explanation support device 10 according to the first embodiment, and therefore description thereof will be omitted.

[0033] 5 is a block diagram showing the functional configuration of a diagnosis explanation support device 10a according to the second embodiment. Functionally, the diagnosis explanation support device 10a includes a diagnostic region input unit 101a, a line drawing creation unit 102a, and a line drawing comparison unit 103a in addition to the display unit 18. Note that the diagnostic region input unit 101a has the same configuration and operates in the same manner as the diagnostic region input unit 101 of the diagnosis explanation support device 10 according to the first embodiment, and therefore a description thereof will be omitted.

[0034] The line drawing creating unit 102a and the line drawing comparing unit 103a create a line drawing of the target image by the following method 1 or 2, and generate display data for comparing the line drawing of the target image with the example line drawing.

[0035] (Method 1) A target image for which the case of the diagnostic region has not been determined is input to the diagnosis explanation support device 10a according to the second embodiment. Therefore, in Method 1, the line drawing creation unit 102a creates a line drawing using a line drawing creation AI corresponding to a case that is considered to be a case of the diagnostic region (hereinafter also referred to as a "case candidate"). Note that there are multiple case candidates, and in Method 1, a line drawing creation AI is prepared in advance for each case candidate. The line drawing creation AI prepared for each case candidate is hereinafter also referred to as a "case-specific line drawing creation AI."

[0036] FIG. 6 is a diagram illustrating Method 1. First, the line drawing creation unit 102a inputs a target image into each line drawing creation AI for each case, and acquires a line drawing of the target image from each line drawing creation AI. In FIG. 6, case 1 to case n are provided as candidate cases, and n line drawing creation AIs for cases 1 to n are prepared. The line drawing creation unit 102a inputs a target image into each line drawing creation AI for cases 1 to n, and acquires the line drawings of cases 1 to n from each line drawing creation AI. Note that in the case of a diffusion-based line drawing creation AI, a prompt is prepared for each candidate case. For example, the line drawing creation unit 102a inputs prompts for cases 1 to n and the target image into the line drawing creation AI, and acquires the line drawings of cases 1 to n.

[0037] Next, the line drawing comparison unit 103a extracts example line drawings from DB 15 based on the candidate cases. Then, the line drawing comparison unit 103a associates the line drawings of the target image with the example line drawings for each case. In FIG. 6 , the line drawing comparison unit 103a extracts example line drawings for cases 1 to n from DB 15, associates the line drawing of case 1 with the example line drawing of case 1, associates the line drawing of case 2 with the example line drawing of case 2, and associates the line drawing of case n with the example line drawing of case n. The line drawing comparison unit 103a generates display data such as that shown in FIG. 7 based on information associating the line drawings of the target image with the example line drawings for each case.

[0038] 7 shows an example of display data by the line drawing comparison unit, which includes a target image 71, AI line drawings 72a and 72b, example line drawings 73a and 73b, and input fields 74a and 74b in a display area 70.

[0039] Target image 71 is a target image input to the diagnosis explanation support device 10a. AI line drawing 72a is a line drawing of the target image created by the line drawing creation AI for case 1. Example line drawing 73a is an example line drawing of case 1. AI line drawing 72b is a line drawing of the target image created by the line drawing creation AI for case 2. Example line drawing 73b is an example line drawing of case 2. In Figure 7, the line drawing of the target image and the example line drawing are displayed side by side for each case, case 1, case 2, ...

[0040] Input fields 74a and 74b are fields where the user inputs an evaluation. The user inputs the comparison result between the AI ​​line drawing 72a and the example line drawing 73a in input field 74a, and the comparison result between the AI ​​line drawing 72b and the example line drawing 73b in input field 74b. In FIG. 7, input fields 74a and 74b include three items: "unnatural," "appropriate," and "optimal." If the AI ​​line drawing and the example line drawing are most similar, the user selects "optimal." If the user cannot decide on "optimal," the user can select multiple "appropriate" options.

[0041] (Method 2) In Method 2, the line drawing creation unit 102a generates line drawings using a line drawing creation AI that corresponds to all case candidates. The line drawing creation AI is assumed to have been trained in advance on all case candidates.

[0042] FIG. 8 is a diagram for explaining Method 2. First, the line drawing creation unit 102a inputs a target image to the line drawing creation AI and acquires a line drawing of the target image from the line drawing creation AI. In FIG. 8, there are case 1 to case n as candidate cases, and one line drawing creation AI corresponding to each of cases 1 to n is prepared. The line drawing creation unit 102a inputs a target image to the line drawing creation AI and acquires a line drawing of one target image.

[0043] Next, the line drawing comparison unit 103a extracts example line drawings from DB 15 based on the candidate cases. In Fig. 8, the line drawing comparison unit 103a extracts example line drawings of cases 1 to n from DB 15 and associates the example line drawings of cases 1 to n with the line drawing of one target image. The line drawing comparison unit 103a generates display data as shown in Fig. 9 based on the line drawing of the target image and the example line drawings.

[0044] 9 shows another example of display data by the line drawing comparison unit. In FIG. 9, a display area 90 includes a target image 91, an AI line drawing 92, example line drawings 93a and 93b, and input fields 94a and 94b.

[0045] A target image 91 is a target image input to the diagnosis explanation support device 10a. An AI line drawing 92 is a line drawing of the target image created by the line drawing creation AI. An example line drawing 93a is an example line drawing of case 1, and an example line drawing 93b is an example line drawing of case 2. In FIG. 9, a line drawing of one target image and multiple example line drawings are displayed.

[0046] The input fields 94a and 94b are fields where the user inputs an evaluation. The user inputs the comparison result between the AI ​​line drawing 92 and the example line drawing 93a in the input field 94a, and the comparison result between the AI ​​line drawing 92 and the example line drawing 93b in the input field 94b. In FIG. 9 , the input fields 94a and 94b include three items: "unnatural," "appropriate," and "optimal." If the AI ​​line drawing and the example line drawing are most similar, the user selects "optimal." If the user cannot decide on "optimal," the user can select multiple "appropriate" options.

[0047] The diagnosis explanation support device according to the second embodiment can display the line drawing of the target image and the example line drawing in a comparable manner, allowing the user to visually compare the line drawing of the target image with the example line drawing and identify the most explainable case.

[0048] In the above configuration, the line drawing comparison unit 103a is an example of an output means.

[0049] <Third Embodiment> Next, a third embodiment will be described. The diagnosis explanation support device according to the third embodiment differs from the diagnosis explanation support device according to the second embodiment in that it calculates the similarity between a line drawing of a target image and an example image and displays the result. Note that the system configuration and hardware configuration of the diagnosis explanation support device according to the third embodiment are similar to those of the diagnosis explanation support device 10 according to the first embodiment, and therefore description thereof will be omitted.

[0050] 10 is a block diagram showing the functional configuration of a diagnosis explanation support device 10b according to the third embodiment. Functionally, the diagnosis explanation support device 10b includes a diagnostic region input unit 101b, a line drawing creation unit 102b, a line drawing comparison unit 103b, and a line drawing similarity calculation unit 104b in addition to the display unit 18. The diagnostic region input unit 101b and the line drawing creation unit 102b have the same configuration and operate in the same manner as the diagnostic region input unit 101a and the line drawing creation unit 102a of the diagnosis explanation support device 10a according to the second embodiment, and therefore, description thereof will be omitted.

[0051] The line drawing comparison unit 103b outputs the line drawing of the target image and the example line drawing to the line drawing similarity calculation unit 104b. At this time, the line drawing comparison unit 103b associates the line drawing of the target image with the example line drawing for each case, as in method 1 of the second embodiment, and outputs the result to the line drawing similarity calculation unit 104b. Alternatively, the line drawing comparison unit 103b associates the line drawing of one target image with the example line drawings of multiple cases, as in method 2 of the second embodiment, and outputs the result to the line drawing similarity calculation unit 104b.

[0052] The line drawing similarity calculation unit 104b calculates the similarity between the line drawing of the target image and the example line drawing. For example, the line drawing similarity calculation unit 104b calculates the similarity between the line drawing of the target image and the example line drawing using an image similarity index such as SSIM (Structural Similarity). In SSIM, two images are quantified using three scales: "brightness," "contrast," and "structure," and an evaluation value is calculated by multiplying these scales. The evaluation value is expressed as a number between 0 and 1, and the closer the value is to 1, the more similar the two images are. Note that in this embodiment, the line drawing similarity calculation unit 104b calculates the evaluation value by referring only to "structure," without taking "brightness" and "contrast" into consideration.

[0053] The line drawing similarity calculation unit 104b may also use a machine learning model that determines the similarity between image data to determine the similarity between the line drawing of the target image and the example line drawing. The machine learning model may be configured using a convolutional neural network (CNN) or the like. The machine learning model may be trained using the following method, for example. The following literature is incorporated herein by reference: Mingxiang Chen, Zhanguo Chang, Haonan Lu, Bitao Yang, Zhuang Li, Liufang Guo, Zhecheng Wang, "AugNet: End-to-End Unsupervised Visual Representation Learning with Image Augmentation," arXiv:2106.06250.

[0054] Furthermore, if the line drawing or example line drawing of the target image contains multiple cell patterns, the line drawing similarity calculation unit 104b may divide the line drawing into cell patterns and use the divided line drawings for similarity calculation. FIG. 11 shows an example of line drawing division. Cancers have grades that indicate the malignancy of cells. FIG. 11 uses an example line drawing of a grade 3 cancer cell (hereinafter also referred to as a "grade 3 example line drawing"). In FIG. 11, the grade 3 example line drawing contains cells of three sizes: large, medium, and small. The line drawing similarity calculation unit 104b divides the grade 3 example line drawing into grades 3a, 3b, and 3c based on the cell sizes. The line drawing similarity calculation unit 104b then calculates the similarity between the line drawing of the target image and each of the example line drawings of grades 3a to 3c. The line drawing similarity calculation unit 104b, for example, adopts the highest similarity among the calculated similarities as the similarity between the line drawing of the target image and the grade 3 example line drawing.

[0055] The line drawing similarity calculation unit 104b generates display data including the line drawing of the target image, the example line drawing, and the calculated similarity.

[0056] 12 shows an example of display data generated by the line drawing similarity calculation unit. In FIG. 12, a display area 75 includes AI line drawings 76 a and 76 b, example line drawings 77 a and 77 b, and similarity scores 78 a and 78 b. The AI ​​line drawings 76 a and 76 b and example line drawings 77 a and 77 b are created by method 1 of the second embodiment.

[0057] The AI ​​line drawing 76a is a line drawing of the target image created by the line drawing creation AI for case 1. The example line drawing 77a is an example line drawing of case 1. The similarity score 78a is the similarity score between the AI ​​line drawing 76a and the example line drawing 77a. The AI ​​line drawing 76b is a line drawing of the target image created by the line drawing creation AI for case 2. The example line drawing 77b is an example line drawing of case 2. The similarity score 78b is the similarity score between the AI ​​line drawing 76b and the example line drawing 77b. In FIG. 12 , the line drawing of the target image, the example line drawing, and the similarity score are displayed side by side for each of the cases, Case 1, Case 2, ...

[0058] 13 shows another example of display data generated by the line drawing similarity calculation unit. In FIG. 13, a display area 95 includes an AI line drawing 96, example line drawings 97a and 97b, and similarity scores 98a and 98b. The AI ​​line drawing 96 and example line drawings 97a and 97b are generated by method 2 of the second embodiment.

[0059] The AI ​​line drawing 92 is a line drawing of the target image created by the line drawing creation AI. The example line drawing 97a is an example line drawing of case 1, and the example line drawing 97b is an example line drawing of case 2. The similarity score 98a is the similarity score between the AI ​​line drawing 96 and the example line drawing 97a. The similarity score 98b is the similarity score between the AI ​​line drawing 96 and the example line drawing 97b. In FIG. 13 , the similarity scores between the line drawing of one target image and multiple example line drawings are respectively displayed.

[0060] 12 and 13, the line image similarity calculation unit 104b may display only the combination of the line image of the target image with the highest similarity score and the example line image side by side.

[0061] The diagnosis explanation support device according to the third embodiment can display the similarity between the line drawing of the target image and the example line drawing, allowing the user to identify the most explainable case based on the similarity between the line drawing of the target image and the example line drawing.

[0062] In the above configuration, the line drawing similarity calculation unit 104b is an example of a first similarity calculation means and a second similarity calculation means.

[0063] 14 is a block diagram showing the functional configuration of a diagnosis explanation support device according to Embodiment 4. The diagnosis explanation support device 400 includes an acquisition unit 401, a line drawing creation unit 402, and an output unit 403.

[0064] 15 is a flowchart of processing by the diagnosis explanation support device of the fourth embodiment. The acquisition unit 401 acquires a target image that is the target of explanation support (step S401). The line drawing creation unit 402 creates a line drawing from the target image using a first machine learning model trained with training data including a case image and ground truth data that is a line drawing of the case image (step S402). The output unit 403 outputs the target image and the line drawing of the target image (step S403).

[0065] According to the diagnosis explanation support device 400 of the fourth embodiment, it is possible to provide a diagnosis explanation support device that generates line drawings that reflect cases. As a result, the diagnosis explanation support device 400 can support user decision-making in the medical field.

[0066] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0067] (Supplementary Note 1) A diagnostic explanation support device comprising: an acquisition means for acquiring a target image that is the subject of explanation support; a line drawing creation means for creating a line drawing from the target image using a first machine learning model trained with learning data including case images and ground truth data that is a line drawing of the case images; and an output means for outputting the target image and the line drawing of the target image.

[0068] (Supplementary Note 2) The diagnosis explanation support device according to Supplementary Note 1, wherein the line drawing creation means acquires an intermediate case image that is an image in the process of generating the line drawing of the target image, and the output means outputs the intermediate case image.

[0069] (Supplementary Note 3) The diagnostic explanation support device described in Supplementary Note 1, wherein the learning data is learning data collected for each case, the first machine learning model is a plurality of machine learning models trained for each of the cases, the line drawing creation means uses the first machine learning model to create a line drawing of the target image for each of the cases, and the output means combines and outputs the line drawing of the target image and the line drawing of the case image for each of the cases.

[0070] (Supplementary Note 4) The diagnosis explanation support device according to Supplementary Note 3, further comprising a first similarity calculation means for calculating the similarity of each of the combinations.

[0071] (Supplementary Note 5) The diagnostic explanation support device according to Supplementary Note 1, wherein the training data is training data for a plurality of cases, the first machine learning model is a machine learning model trained on the plurality of cases collectively, the line drawing creation means creates a line drawing from the target image using the first machine learning model, and the output means outputs a line drawing of the target image and case line drawings of the plurality of cases.

[0072] (Supplementary Note 6) The diagnosis explanation support device according to Supplementary Note 5, further comprising a second similarity calculation means for calculating a similarity between the line drawing of the target image and each of the case line drawings of the plurality of cases.

[0073] (Supplementary Note 7) The diagnostic explanation support device according to Supplementary Note 4, wherein the first similarity calculation means calculates the similarity based on an SSIM evaluation index or by using a second machine learning model that determines the similarity between images.

[0074] (Supplementary Note 8) The diagnostic explanation support device according to Supplementary Note 6, wherein the second similarity calculation means calculates the similarity based on an SSIM evaluation index or by using a second machine learning model that determines the similarity between images.

[0075] (Supplementary Note 9) A diagnostic explanation support method comprising: acquiring a target image that is the subject of explanation support; creating a line drawing from the target image using a first machine learning model trained with training data including case images and ground truth data that is a line drawing of the case images; and outputting the target image and the line drawing of the target image.

[0076] (Supplementary Note 10) A recording medium having recorded thereon a program that causes a computer to execute the following processes: acquire a target image that is the subject of explanation assistance; create a line drawing from the target image using a first machine learning model trained with learning data including case images and correct answer data that is a line drawing of the case image; and output the target image and the line drawing of the target image.

[0077] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-described embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0078] This application claims priority based on Japanese Patent Application No. 2024-034500, filed March 7, 2024, the disclosure of which is incorporated herein in its entirety by reference.

[0079] 10 diagnosis explanation support device 15 database (DB) 18 display unit 101, 101a, 101b diagnosis region input unit 102, 102a, 102b line drawing creation unit 103a, 103b line drawing comparison unit 104b line drawing similarity calculation unit

Claims

1. A diagnostic explanation support device comprising: an acquisition means for acquiring a target image that is the subject of explanation support; a line drawing creation means for creating a line drawing from the target image using a first machine learning model trained with learning data including case images and correct answer data that is a line drawing of the case images; and an output means for outputting the target image and the line drawing of the target image.

2. The diagnostic explanation support device according to claim 1, wherein the line drawing creation means acquires an intermediate case image, which is an image in the process of generating the line drawing of the target image, and the output means outputs the intermediate case image.

3. The diagnostic explanation support device of claim 1, wherein the training data is training data collected for each case, the first machine learning model is a plurality of machine learning models trained for each of the cases, the line drawing creation means uses the first machine learning model to create a line drawing of the target image for each of the cases, and the output means combines and outputs the line drawing of the target image and the line drawing of the case image for each of the cases.

4. A diagnosis explanation support device according to claim 3, further comprising first similarity calculation means for calculating the similarity of each of said combinations.

5. The diagnostic explanation support device of claim 1, wherein the training data is training data that compiles a plurality of cases, the first machine learning model is a machine learning model that has been trained on the plurality of cases collectively, the line drawing creation means creates a line drawing from the target image using the first machine learning model, and the output means outputs a line drawing of the target image and case line drawings of the plurality of cases.

6. A diagnosis explanation support device according to claim 5, further comprising second similarity calculation means for calculating the similarity between the line drawing of the target image and each of the case line drawings of the plurality of cases.

7. A diagnostic explanation support device as described in claim 4, wherein the first similarity calculation means calculates the similarity based on an SSIM evaluation index or by using a second machine learning model that determines the similarity between images.

8. The diagnostic explanation support device according to claim 6, wherein the second similarity calculation means calculates the similarity based on an SSIM evaluation index or by using a second machine learning model that determines the similarity between images.

9. A diagnostic explanation support method that acquires a target image that is the subject of explanation support, creates a line drawing from the target image using a first machine learning model trained with training data including case images and correct answer data that is a line drawing of the case image, and outputs the target image and the line drawing of the target image.

10. A recording medium having recorded thereon a program that causes a computer to execute the following process: acquire a target image that is the subject of explanation assistance; create a line drawing from the target image using a first machine learning model trained with training data including case images and correct answer data that is a line drawing of the case image; and output the target image and the line drawing of the target image.

Images