Endoscopy assistance device, endoscopy assistance method, and recording medium

Abstract

In this endoscopy assistance device, an acquisition means acquires an endoscopic image including a lesion. An attribute prediction means predicts attribute information of the lesion and the endoscopic image on the basis of the endoscopic image and an attribute identification prompt prepared in advance. A prompt generation means generates, on the basis of the result of predicting the attribute information, a diagnostic prompt for performing a predetermined diagnosis. A diagnosis means diagnoses the lesion on the basis of the endoscopic image and the diagnostic prompt. A display control means displays the result of diagnosing the lesion and the attribute information. The endoscopy assistance device is able to assist decision-making by users in the medical field.

Description

Endoscopic examination support device, endoscopic examination support method, and recording medium

[0001] The present disclosure relates to techniques for assisting in the diagnosis of lesions.

[0002] There are known endoscopic diagnosis support systems that use artificial intelligence (AI) models to support the diagnosis of lesions. For example, Patent Literature 1 proposes a support system that determines the size and classification of a lesion based on an endoscopic image and displays treatment guidelines.

[0003] JP 2023-119573 A

[0004] However, even with the technique of Patent Document 1, it is not always possible to accurately diagnose a lesion.

[0005] One object of the present disclosure is to accurately diagnose lesions in endoscopic examinations.

[0006] In one aspect of the present disclosure, an endoscopic examination assistance device includes: an acquisition means for acquiring an endoscopic image including a lesion; an attribute prediction means for predicting attribute information of the lesion and the endoscopic image based on the endoscopic image and a prepared attribute identification prompt; a prompt generation means for generating a diagnostic prompt for making a predetermined diagnosis based on the predicted result of the attribute information; a diagnosis means for diagnosing the lesion based on the endoscopic image and the diagnostic prompt; and a display control means for displaying the diagnosis result of the lesion and the attribute information.

[0007] In another aspect of the present disclosure, a method for assisting endoscopic examination includes acquiring an endoscopic image including a lesion; predicting attribute information of the lesion and the endoscopic image based on the endoscopic image and a prepared attribute identification prompt; generating a diagnostic prompt for performing a predetermined diagnosis based on the predicted attribute information; diagnosing the lesion based on the endoscopic image and the diagnostic prompt; and displaying the diagnosis result of the lesion and the attribute information.

[0008] In yet another aspect of the present disclosure, a recording medium records a program that causes a computer to execute the following processes: acquire an endoscopic image including a lesion; predict attribute information of the lesion and the endoscopic image based on the endoscopic image and a prepared attribute identification prompt; generate a diagnostic prompt for performing a predetermined diagnosis based on the predicted attribute information; diagnose the lesion based on the endoscopic image and the diagnostic prompt; and display the diagnosis result of the lesion and the attribute information.

[0009] According to the present disclosure, it is possible to accurately diagnose lesions in endoscopic examinations.

[0010] 1 is a block diagram showing a schematic configuration of an endoscopic examination system; FIG. 2 is a block diagram showing a hardware configuration of an endoscopic examination support device; FIG. 3 is a block diagram showing a functional configuration of the endoscopic examination support device; FIG. 4 is an explanatory diagram of processing by a lesion attribute classification unit; FIG. 5 is an example of data used for learning an AI model; FIG. 6 is an example of creating a classification prompt; FIG. 7 is an explanatory diagram of processing by a lesion classification unit; FIG. 8 is an example of display by a display device; FIG. 9 is a flowchart of processing by an endoscopic examination support device; FIG. 10 is a diagram for explaining a modified example; FIG. 11 is a block diagram showing a functional configuration of another endoscopic examination support device according to the present disclosure; FIG. 12 is a flowchart of processing by another endoscopic examination support device according to the present disclosure.

[0011] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings.

[0012] First Embodiment The AI ​​model described above is trained to diagnose lesions using images as input. At this time, the accuracy of the AI ​​model is improved by taking into account attribute information such as the type of case and the model of endoscope. However, it is difficult to collect all attribute information and have the AI ​​model comprehensively learn the attribute information with limited resources. Therefore, this embodiment provides a method for improving diagnostic accuracy without having the AI ​​model comprehensively learn the attribute information. As will be described in detail later, the endoscopic examination support device of this embodiment predicts attribute information by using an AI model trained on pairs of images and text, or by using prepared prompts, and can diagnose lesions by combining the predicted attribute information.

[0013] [System Configuration] Fig. 1 shows a schematic configuration of an endoscopic examination system 100. The endoscopic examination system 100 diagnoses a lesion detected during an examination (including treatment) using an endoscope. Lesion diagnoses include whether the lesion is neoplastic or non-neoplastic, its depth of invasion, and its size, but the purpose of this embodiment is to diagnose whether a detected lesion is neoplastic or non-neoplastic (i.e., classify the lesion as neoplastic or non-neoplastic).

[0014] As shown in FIG. 1 , the endoscopic examination system 100 mainly includes an endoscopic examination support device 1 , a display device 2 , and an endoscope 3 connected to the endoscopic examination support device 1 .

[0015] The endoscopic examination support device 1 acquires, from the endoscope 3, images captured by the endoscope 3 during endoscopic examination (hereinafter also referred to as "endoscopic images Ic") and displays display data on the display device 2 for confirmation by the examiner (doctor) performing the endoscopic examination. Specifically, the endoscopic examination support device 1 acquires, as endoscopic images Ic, moving images of the inside of organs captured by the endoscope 3 during endoscopic examination. Furthermore, when the doctor finds a lesion during endoscopic examination, he or she operates the endoscope 3 to input an instruction to capture the lesion. The endoscopic examination support device 1 generates lesion images that capture the lesion based on the doctor's imaging instructions. Specifically, the endoscopic examination support device 1 generates lesion images, which are still images, from the endoscopic images Ic, which are moving images, based on the doctor's imaging instructions.

[0016] The display device 2 is a display or the like that displays a predetermined image based on a display signal supplied from the endoscopic examination support device 1 .

[0017] The endoscope 3 mainly comprises an operating unit 36 ​​for the doctor to input instructions such as air supply, water supply, angle adjustment, and photography instructions, a flexible shaft 37 that is inserted into the subject's organ to be examined, a tip 38 that incorporates an endoscopic camera such as an ultra-small imaging element, and a connection unit 39 for connecting to the endoscopic examination support device 1.

[0018] The following explanation will be based mainly on the processing involved in an endoscopic examination of the large intestine, but the subject of examination is not limited to the large intestine, and may be any part of the digestive tract (digestive organs) such as the stomach, esophagus, small intestine, or duodenum.

[0019] 2 shows the hardware configuration of the endoscopic examination support device 1. The endoscopic examination support device 1 mainly includes a processor 11, a memory 12, an interface 13, an input unit 14, a light source unit 15, a sound output unit 16, and a database (hereinafter referred to as "DB") 17. These elements are connected via a data bus 19.

[0020] The processor 11 performs predetermined processing by executing programs stored in the memory 12. The processor 11 is a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0021] The memory 12 is composed of various volatile memories used as working memories, such as RAM (Random Access Memory) and ROM (Read Only Memory), and nonvolatile memories that store information necessary for processing by the endoscopic examination support device 1. The memory 12 may include an external storage device such as a hard disk connected to or built into the endoscopic examination support device 1, or may include a storage medium such as a removable flash memory or disk medium. The memory 12 stores programs for the endoscopic examination support device 1 to execute each process in this embodiment.

[0022] Furthermore, under the control of the processor 11, the memory 12 temporarily stores a series of endoscopic images Ic captured by the endoscope 3 during an endoscopic examination.

[0023] The interface 13 performs interface operations between the endoscopic examination assistance device 1 and an external device. For example, the interface 13 supplies the display data Id generated by the processor 11 to the display device 2. The interface 13 also supplies illumination light generated by the light source unit 15 to the endoscope 3. The interface 13 also supplies an electrical signal indicating an endoscopic image Ic supplied from the endoscope 3 to the processor 11. The interface 13 may be a communication interface such as a network adapter for wired or wireless communication with an external device, or may be a hardware interface compliant with USB (Universal Serial Bus), SATA (Serial AT Attachment), or the like.

[0024] The input unit 14 generates an input signal based on an operation by a doctor. The input unit 14 is, for example, a button, a touch panel, a remote controller, or a voice input device. The light source unit 15 generates light to be supplied to the tip 38 of the endoscope 3. The light source unit 15 may also incorporate a pump or the like for sending water or air to be supplied to the endoscope 3. The sound output unit 16 outputs sound based on the control of the processor 11.

[0025] The DB 17 stores prompts to be input to the AI ​​model, etc. The DB 17 may include an external storage device such as a hard disk connected to or built into the endoscopic examination support device 1, or may include a storage medium such as a removable flash memory. Note that instead of providing the DB 17 within the endoscopic examination system 100, the DB 17 may be provided on an external server or the like, and related information may be obtained from the server via communication.

[0026] 3 is a block diagram showing the functional configuration of the endoscopic examination support device 1. In terms of functionality, the endoscopic examination support device 1 includes, in addition to the DB 17 described above, an image encoding unit 101, a lesion attribute classification unit 102, a lesion classification prompt creation unit 103, and a lesion classification unit 104.

[0027] The image encoding unit 101, the lesion attribute classification unit 102, the lesion classification prompt creation unit 103, and the lesion classification unit 104 are configured by the processor 11 shown in FIG.

[0028] An endoscopic image Ic is input to the endoscopic examination support device 1 from the endoscope 3. When the endoscopic examination support device 1 receives an imaging instruction from a doctor or detects a lesion in the endoscopic image Ic, it generates a lesion image, which is a still image, from the endoscopic image Ic, which is a moving image. The lesion image is input to the image encoding unit 101.

[0029] The image encoding unit 101 generates a feature vector from the lesion image using an encoder, and outputs the feature vector of the lesion image to the lesion attribute classification unit 102 and the lesion classification unit 104.

[0030] The lesion attribute classification unit 102 predicts attribute information of the lesion and image and outputs the prediction result to the lesion classification prompt creation unit 103. The attribute information includes attributes and their attribute values. Attributes include, for example, lesion size, macroscopic type of lesion (protruding or flat), differential diagnosis, detected site, patient attributes, presence or absence of image enhancement (special light, dye), presence or absence of image magnification, endoscope manufacturer, and scope model. Each attribute may have one or more attribute values. DB 17 stores all conceivable combinations of attributes and attribute values ​​as prompts for identifying attributes. The prompt for identifying attributes is an example of an attribute-identifying prompt.

[0031] 4 is an explanatory diagram of processing by the lesion attribute classification unit 102. As shown in the figure, the lesion attribute classification unit 102 first inputs the feature vector of the lesion image and a prompt 41 for identifying attributes acquired from DB 17 into the AI ​​model. Then, the lesion attribute classification unit 102 predicts attribute information of the lesion and image based on a response 42 from the AI ​​model. Note that the lesion attribute classification unit 102 uses, as the AI ​​model, a model that inputs the feature vector of the image and a sentence and outputs the similarity between the image and the sentence.

[0032] The attribute-specific prompt 41 includes prompts related to various attributes. In FIG. 4, prompts related to lesion size (P1, P2) and image enhancement (P3) are shown as examples. P1: The size of the lesion is diminutive. P2: The size of the lesion is non-diminutive. P3: The lesion is stained by indigo carmine.

[0033] The answer 42 includes the similarity between the feature vector of the lesion image and each prompt included in the attribute-identifying prompt 41. This similarity is hereinafter referred to as the "first similarity." The lesion attribute classification unit 102 selects one or more prompts from the multiple prompts included in the attribute-identifying prompt 41 based on the answer 42. For example, the lesion attribute classification unit 102 may select a prompt whose first similarity is equal to or greater than a predetermined threshold, or may select a predetermined number of prompts in descending order of first similarity. The lesion attribute classification unit 102 predicts the attribute information indicated by the selected prompt as the attribute information of the lesion and image.

[0034] The lesion attribute classification unit 102 uses, for example, a vision language model (VLM) as an AI model. The VLM is a model that handles images and text in a combined manner and is trained using a dataset including pairs of lesion images and text corresponding to the lesion. FIG. 5 shows an example of data used for training. The data in FIG. 5 includes a lesion image 51 and related information 52 related to the lesion. The related information 52 includes attribute information of the lesion and the image. The related information 52 is created by extracting descriptions from the examination report and adding image information (e.g., whether a light source was used, whether image enlargement was performed, whether treatment was performed, etc.). During training of the VLM, the related information 52 is converted into text for prompts. The VLM is then trained to increase the similarity between the lesion image and the prompt corresponding to the lesion.

[0035] 3 , the lesion classification prompt creation unit 103 creates a prompt for making a target diagnosis and outputs the created prompt to the lesion classification unit 104. In this embodiment, since the objective is to classify lesions into neoplastic and non-neoplastic lesions, the prompt created by the lesion classification prompt creation unit 103 will hereinafter also be referred to as a "classification prompt." The classification prompt is an example of a diagnostic prompt.

[0036] 6 shows an example of the creation of a classification prompt. The lesion classification prompt creation unit 103 creates a classification prompt 62 based on the prediction result 61 from the lesion attribute classification unit 102.

[0037] The classification prompt 62 includes a normal classification prompt 62a and a classification prompt 62b including attribute information. The normal classification prompt 62a is a prompt for classifying a lesion into neoplastic or non-neoplastic. In the example of FIG. 6 , the lesion classification prompt creation unit 103 creates a prompt P4 indicating neoplastic and a prompt P5 indicating non-neoplastic. The classification prompt 62b including attribute information is a prompt that includes predicted attribute information in the normal classification prompt. In the example of FIG. 6 , the lesion attribute classification unit 102 predicts that the attribute information indicated by prompts P1 and P3 is attribute information of the lesion and the image. The lesion classification prompt creation unit 103 creates a prompt P6 combining prompts P1 and P4, a prompt P7 combining prompts P3 and P4, and a prompt P8 combining prompts P3 and P5.

[0038] Returning to Figure 3, the lesion classification unit 104 classifies the lesions included in the lesion image and outputs the results to the display device 2. Figure 7 is an explanatory diagram of the processing by the lesion classification unit 104. As shown in the figure, the lesion classification unit 104 first inputs the feature vector of the lesion image and a classification prompt 71 to the AI ​​model. Then, the lesion classification unit 104 classifies the lesion based on a response 72 from the AI ​​model. Note that the lesion classification unit 104 can use a VLM as the AI ​​model, similar to the lesion attribute classification unit 102.

[0039] The answer 72 includes a similarity between the feature vector of the lesion image and each prompt included in the classification prompt 71. This similarity is hereinafter referred to as a "second similarity." The lesion classification unit 104 classifies the lesion based on the second similarity.

[0040] For example, if the prompt with the highest second similarity includes the description "neoplastic," the lesion classification unit 104 outputs "neoplastic" as the classification result. On the other hand, if the prompt with the highest second similarity includes the description "non-neoplastic," the lesion classification unit 104 outputs "non-neoplastic" as the classification result. In FIG. 7 , since prompt P7, which includes "neoplastic," has the highest second similarity, the lesion classification unit 104 outputs "neoplastic" as the classification result. The lesion classification unit 104 may also classify lesions using a method such as majority voting. For example, the lesion classification unit 104 classifies lesions by taking a majority vote for prompts whose second similarity is equal to or greater than a predetermined threshold.

[0041] Furthermore, the lesion classification unit 104 may multiply the second similarity by a predetermined weight and classify the lesion based on the calculated value. The predetermined weight is assumed to be predetermined for each attribute information. For example, the lesion classification unit 104 may determine the weight based on the first similarity obtained when predicting the attribute information. Specifically, if the first similarity of the attribute-identifying prompt P3 is "0.6," the lesion classification unit 104 may multiply the second similarities of the classification prompts P7 and P8, which include the attribute information indicated by the prompt P3, by "0.6," and classify the lesion based on the calculated value.

[0042] Furthermore, the lesion classification unit 104 may determine the weights by calculating the importance of the attribute information based on medical information or the like, or may calculate the weights from the data used for VLM training. The lesion classification unit 104 sets the weights so that attribute information that influences the classification of the lesion is weighted heavily.

[0043] In the above configuration, the image encoding unit 101 is an example of an acquisition means, the lesion attribute classification unit 102 is an example of an attribute prediction means, the lesion classification prompt creation unit 103 is an example of a prompt generation means, the lesion classification unit 104 is an example of a diagnosis means, and the display device 2 is an example of a display control means.

[0044] [Display Example] Next, a display example on the display device 2 will be described.

[0045] 8A shows an example of a display on the display device 2. In this example, the display device 2 displays an endoscopic video 81, a lesion image 82, a diagnosis result 83, and an attribute information display area 84.

[0046] The endoscopic video 81 is an endoscopic image Ic during the examination, and is updated as the endoscopic camera moves. The lesion image 82 is an endoscopic image that includes a lesion detected during the endoscopic examination. The diagnosis result 83 indicates the diagnosis result of the lesion included in the lesion image 82. The attribute information display area 84 is an area that displays attribute information of the lesion included in the lesion image 82. The attribute information display area 84 displays attribute information of the lesion predicted by the lesion attribute classification unit 102. By looking at the attribute information display area 84, a doctor can confirm whether the diagnosis result is reliable.

[0047] 8(B) is another example of display by the display device 2. This example shows the case where the reliability of attribute information is displayed. In FIG. 8(B), the attribute information display area 84a displays the attribute information of the lesion predicted by the lesion attribute classification unit 102 and its reliability (first similarity) side by side. A display such as that in the attribute information display area 84a allows the doctor to confirm the basis for selecting the attribute information.

[0048] In the attribute information display area 84 and the attribute information display area 84a, if there are multiple pieces of attribute information for the lesion, the attribute information may be displayed in descending order of reliability.

[0049] [Processing Flow] Next, a description will be given of the diagnostic processing for diagnosing the lesion as described above. Fig. 9 is a flowchart of the processing by the endoscopic examination support device 1. This processing is realized by the processor 11 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 3.

[0050] First, the image encoding unit 101 acquires a lesion image. Then, the image encoding unit 101 generates a feature vector from the lesion image (step S11). The image encoding unit 101 outputs the feature vector of the lesion image to the lesion attribute classification unit 102 and the lesion classification unit 104.

[0051] Next, the lesion attribute classification unit 102 inputs the feature vector of the lesion image and the multiple attribute identification prompts acquired from DB 17 into the AI ​​model (step S12).The lesion attribute classification unit 102 then predicts attribute information of the lesion and the image based on the response from the AI ​​model (step S13).The lesion attribute classification unit 102 outputs the predicted attribute information to the lesion classification prompt creation unit 103.

[0052] Next, the lesion classification prompt creation unit 103 creates a classification prompt based on the prediction result input from the lesion attribute classification unit 102 (step S14). The lesion classification prompt creation unit 103 outputs the classification prompt to the lesion classification unit 104.

[0053] Next, the lesion classification unit 104 inputs the feature vector of the lesion image and the classification prompt to the AI ​​model (step S15). Then, the lesion classification unit 104 classifies the lesion based on the response from the AI ​​model (step S16). The lesion classification unit 104 outputs the classification result to the display device 2 (step S17). Then, the process ends.

[0054] [Modifications] Next, a description will be given of modifications of the first embodiment. The following modifications can be applied to the first embodiment in appropriate combinations.

[0055] (Variation 1) The lesion classification prompt creation unit 103 may create a classification prompt by combining multiple pieces of attribute information. Fig. 10 is a diagram for explaining this variation. In the example of Fig. 10, the lesion attribute classification unit 102 predicts attribute information using an attribute identification prompt 91 and outputs a prediction result 92. Furthermore, the lesion classification prompt creation unit 103 creates a classification prompt 93 based on the prediction result 92.

[0056] The attribute identification prompt 91 includes prompts related to lesion size (P11, P12), a prompt related to image enhancement (P13), and prompts related to the macroscopic type of the lesion (P14, P15). As shown in the prediction result 92, the lesion attribute classification unit 102 predicts that the attribute information indicated by prompts P11, and P13 to P15 is attribute information of the lesion and image.

[0057] As described above, the attribute identification prompt 91 includes prompts related to multiple attributes. Therefore, the lesion classification prompt creation unit 103 can combine prompts with different attributes to create the classification prompt 93. In the example of Figure 10, the lesion classification prompt creation unit 103 creates the classification prompt P19 by combining the attribute information indicated by prompt P13 with the attribute information indicated by prompt P14. The lesion classification prompt creation unit 103 also creates the classification prompt P20 by combining the attribute information indicated by prompt P13 with the attribute information indicated by prompt P15.

[0058] (Variation 2) When there are multiple prompts related to a certain attribute and the first similarity is approximately the same, the lesion classification prompt creation unit 103 may create a classification prompt for each of them. In the prediction result 92 of Figure 10, the first similarity is approximately the same for prompts P14 and P15 related to the macroscopic type of the lesion. Therefore, the lesion classification prompt creation unit 103 creates a classification prompt P19 that includes the attribute information indicated by prompt P14, and a classification prompt P20 that includes the attribute information indicated by prompt P15.

[0059] 11 is a block diagram showing the functional configuration of an endoscopic examination support device according to Embodiment 2. The endoscopic examination support device 200 includes an acquisition unit 201, an attribute prediction unit 202, a prompt generation unit 203, a diagnosis unit 204, and a display control unit 205.

[0060] 12 is a flowchart of processing by the endoscopic examination support device of the second embodiment. The acquisition unit 201 acquires an endoscopic image including a lesion (step S201). The attribute prediction unit 202 predicts attribute information of the lesion and the endoscopic image based on the endoscopic image and a prepared attribute identification prompt (step S202). The prompt generation unit 203 generates a diagnostic prompt for performing a predetermined diagnosis based on the predicted attribute information (step S203). The diagnosis unit 204 diagnoses the lesion based on the endoscopic image and the diagnostic prompt (step S204). The display control unit 205 displays the lesion diagnosis result and the attribute information (step S205).

[0061] The endoscopic examination support device 200 of the second embodiment enables accurate diagnosis of lesions during endoscopic examination. Furthermore, the endoscopic examination support device 200 can support the decision-making of users (doctors) in the medical field.

[0062] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0063] (Supplementary Note 1) An endoscopic examination support device comprising: an acquisition means for acquiring an endoscopic image including a lesion; an attribute prediction means for predicting attribute information of the lesion and the endoscopic image based on the endoscopic image and a prepared attribute identification prompt; a prompt generation means for generating a diagnostic prompt for making a predetermined diagnosis based on the predicted result of the attribute information; a diagnosis means for diagnosing the lesion based on the endoscopic image and the diagnostic prompt; and a display control means for displaying the diagnosis result of the lesion and the attribute information.

[0064] (Appendix 2) The endoscopic examination support device described in Appendix 1, wherein the attribute prediction means calculates a first similarity between the endoscopic image and the attribute identification prompt, and predicts attribute information of the lesion and the endoscopic image based on the first similarity, and the diagnosis means calculates a second similarity between the endoscopic image and the diagnostic prompt, and diagnoses the lesion based on the second similarity.

[0065] (Supplementary Note 3) The endoscopic examination support device according to Supplementary Note 2, wherein the attribute prediction means and the diagnosis means calculate the first similarity and the second similarity, respectively, using an AI model trained using paired data of a lesion image and an examination report of the lesion.

[0066] (Supplementary Note 4) The endoscopic examination support device according to Supplementary Note 2, wherein a plurality of the attribute specifying prompts are prepared for each attribute information, and the attribute prediction means calculates the first similarity for each of the attribute specifying prompts.

[0067] (Appendix 5) The endoscopic examination support device according to Appendix 4, wherein the attribute prediction means predicts attribute information indicated by an attribute identification prompt in which the first similarity is equal to or greater than a first threshold as attribute information of the lesion and the endoscopic image.

[0068] (Supplementary Note 6) The endoscopic examination support device according to Supplementary Note 4, wherein, when there are a plurality of attribute identification prompts for which the first similarity is equal to or greater than a second threshold, the attribute prediction means selects a predetermined number of attribute identification prompts in descending order of the first similarity, and predicts attribute information indicated by the selected attribute identification prompts as attribute information of the lesion and the endoscopic image.

[0069] (Appendix 7) The endoscopic examination support device according to Appendix 4, wherein the prompt generation means, when there are multiple pieces of attribute information, generates multiple diagnostic prompts by combining the multiple pieces of attribute information, and the diagnostic means calculates the second similarity for each diagnostic prompt.

[0070] (Supplementary Note 8) The endoscopic examination support device according to Supplementary Note 7, wherein the diagnostic means diagnoses the lesion based on the diagnostic prompt with the highest second similarity.

[0071] (Appendix 9) An endoscopic examination support device as described in Appendix 7, further comprising a weight setting means for setting a weight for each attribute information, wherein the diagnostic means diagnoses the lesion based on the diagnostic prompt, the second similarity, and the weight.

[0072] (Supplementary Note 10) The endoscopic examination support device according to Supplementary Note 9, wherein the weight setting means determines a weight for each attribute information item based on the first similarity.

[0073] (Supplementary Note 11) The endoscopic examination support device according to Supplementary Note 9, wherein the weight setting means determines a weight for each attribute information item based on medical information.

[0074] (Supplementary Note 12) The endoscopic examination support device according to Supplementary Note 7, wherein the diagnosing means diagnoses the lesion by a voting method based on the diagnostic prompt and the second similarity.

[0075] (Supplementary Note 13) The endoscopic examination support device according to Supplementary Note 2, wherein the display control means displays the diagnosis result of the lesion, the attribute information, and the reliability of the attribute information, and the reliability of the attribute information is determined based on the first similarity.

[0076] (Supplementary Note 14) The endoscopic examination support device according to Supplementary Note 13, wherein the display control means displays the attribute information in descending order of the reliability.

[0077] (Appendix 15) The attribute information of the lesion includes at least one of information regarding the size of the lesion, the macroscopic type of the lesion, a differential diagnosis, and the part of the organ where the lesion was detected, and the attribute information of the endoscopic image includes at least one of information regarding whether or not the image is enhanced, whether or not the image is enlarged, the manufacturer of the endoscope, and the model of the scope.

[0078] (Appendix 16) An endoscopic examination support method comprising: acquiring an endoscopic image including a lesion; predicting attribute information of the lesion and the endoscopic image based on the endoscopic image and a prepared attribute identification prompt; generating a diagnostic prompt for making a predetermined diagnosis based on the predicted attribute information; diagnosing the lesion based on the endoscopic image and the diagnostic prompt; and displaying the diagnosis result of the lesion and the attribute information.

[0079] (Appendix 17) A recording medium having recorded thereon a program that causes a computer to execute the following processes: acquiring an endoscopic image including a lesion; predicting attribute information of the lesion and the endoscopic image based on the endoscopic image and a prepared attribute identification prompt; generating a diagnostic prompt for performing a predetermined diagnosis based on the predicted attribute information; diagnosing the lesion based on the endoscopic image and the diagnostic prompt; and displaying the diagnosis result of the lesion and the attribute information.

[0080] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-described embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0081] REFERENCE SIGNS LIST 1 Endoscopy support device 2 Display device 3 Endoscope 11 Processor 12 Memory 13 Interface 100 Endoscopy system 101 Image encoding unit 102 Lesion attribute classification unit 103 Lesion classification prompt creation unit 104 Lesion classification unit

Claims

1. An endoscopic examination support device comprising: an acquisition means for acquiring an endoscopic image including a lesion; an attribute prediction means for predicting attribute information of the lesion and the endoscopic image based on the endoscopic image and a prepared attribute identification prompt; a prompt generation means for generating a diagnostic prompt for making a predetermined diagnosis based on the predicted result of the attribute information; a diagnosis means for diagnosing the lesion based on the endoscopic image and the diagnostic prompt; and a display control means for displaying the diagnosis result of the lesion and the attribute information.

2. The endoscopic examination support device of claim 1, wherein the attribute prediction means calculates a first similarity between the endoscopic image and the attribute identification prompt, and predicts attribute information of the lesion and the endoscopic image based on the first similarity, and the diagnosis means calculates a second similarity between the endoscopic image and the diagnostic prompt, and diagnoses the lesion based on the second similarity.

3. The endoscopic examination support device according to claim 2, wherein the attribute prediction means and the diagnosis means calculate the first similarity and the second similarity, respectively, using an AI model trained using paired data of lesion images and lesion examination reports.

4. An endoscopic examination support device as described in claim 2, wherein a plurality of the attribute specifying prompts are prepared for each attribute information, and the attribute prediction means calculates the first similarity for each of the attribute specifying prompts.

5. An endoscopic examination support device as described in claim 4, wherein the attribute prediction means predicts attribute information indicated by an attribute identification prompt in which the first similarity is equal to or greater than a first threshold as attribute information of the lesion and the endoscopic image.

6. The endoscopic examination support device according to claim 4, wherein, when there are a plurality of attribute-identifying prompts for which the first similarity is equal to or greater than a second threshold, the attribute prediction means selects a predetermined number of attribute-identifying prompts in descending order of the first similarity, and predicts the attribute information indicated by the selected attribute-identifying prompts as the attribute information of the lesion and the endoscopic image.

7. The endoscopic examination support device of claim 4, wherein the prompt generation means, when there are multiple pieces of attribute information, generates multiple diagnostic prompts by combining the multiple pieces of attribute information, and the diagnostic means calculates the second similarity for each diagnostic prompt.

8. The endoscopic examination support device according to claim 7, wherein the diagnostic means diagnoses the lesion based on the diagnostic prompt with the highest second similarity.

9. An endoscopic examination support device as described in claim 7, further comprising a weight setting means for setting a weight for each attribute information, wherein the diagnostic means diagnoses the lesion based on the diagnostic prompt, the second similarity, and the weight.

10. The endoscopic examination support device according to claim 9, wherein the weight setting means determines a weight for each attribute information item based on the first similarity.

11. The endoscopic examination support device according to claim 9, wherein the weight setting means determines a weight for each attribute information item based on medical information.

12. An endoscopic examination support device according to claim 7, wherein the diagnostic means diagnoses the lesion by a voting method based on the diagnostic prompt and the second similarity.

13. The endoscopic examination support device according to claim 2, wherein the display control means displays the diagnosis result of the lesion, the attribute information, and the reliability of the attribute information, and the reliability of the attribute information is determined based on the first similarity.

14. The endoscopic examination support device according to claim 13, wherein the display control means displays the attribute information in descending order of reliability.

15. The endoscopic examination support device of claim 1, wherein the attribute information of the lesion includes at least one of the following information: lesion size, macroscopic type of the lesion, differential diagnosis, and information on the part of the organ in which the lesion was detected; and the attribute information of the endoscopic image includes at least one of the following information: whether or not image enhancement is performed, whether or not image enlargement is performed, the manufacturer of the endoscope, and the model of the scope.

16. A method for supporting endoscopic examinations, comprising: acquiring an endoscopic image including a lesion; predicting attribute information of the lesion and the endoscopic image based on the endoscopic image and a prepared attribute identification prompt; generating a diagnostic prompt for making a predetermined diagnosis based on the predicted attribute information; diagnosing the lesion based on the endoscopic image and the diagnostic prompt; and displaying the diagnosis result of the lesion and the attribute information.

17. A recording medium having recorded thereon a program that causes a computer to execute the following processes: acquire an endoscopic image including a lesion; predict attribute information of the lesion and the endoscopic image based on the endoscopic image and a prepared attribute identification prompt; generate a diagnostic prompt for making a predetermined diagnosis based on the predicted attribute information; diagnose the lesion based on the endoscopic image and the diagnostic prompt; and display the diagnosis result of the lesion and the attribute information.

Images