Diagnostic apparatus, diagnostic system, and diagnostic method

Abstract

To provide a diagnostic apparatus, diagnostic system, and diagnostic method capable of achieving a correct answer ratio of diagnosis for various diseases utilizing AI equal to or higher than a correct answer ratio of diagnosis by ophthalmic experts.SOLUTION: A diagnostic apparatus according to one embodiment of the invention comprises a learned model and a processing unit. The learned model is the model constructed by machine learning using a dataset in which learning images obtained by imaging diseased portions and learning questions corresponding to characteristic images of the diseases in the learning images are used as explanatory variables, and learning answers to the learning questions are used as objective variables. The processing unit inputs a diagnostic image and a diagnostic question into the learned model, acquires a diagnostic answer to the diagnostic question from the learned model, and executes input of the diagnostic image and the diagnostic question to the learned model for the number of diagnostic questions.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a diagnostic device, a diagnostic system, and a diagnostic method. [Background technology]

[0002] Fundus photographs obtained by fundus examination can detect diseases such as diabetic retinopathy, macular degeneration, glaucoma, retinal detachment, and retinal vein occlusion. In recent years, attempts have begun to diagnose these various diseases using artificial intelligence (AI) (see Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] https: / / www.tsukazaki-hp.jp / data / media / tsukazaki_hp / page / deseases / report / pdf01.pdf Summary of the Invention [Problem to be solved by the invention]

[0004] However, it is known that when diagnostic tests for the above-mentioned various diseases are conducted, the accuracy rate of diagnoses for the above-mentioned various diseases using AI is significantly lower than the accuracy rate of diagnoses by ophthalmological experts. Therefore, it is desirable to provide a diagnostic device, diagnostic system, and diagnostic method that can make the accuracy rate of diagnoses for the various diseases using AI equal to or higher than the accuracy rate of diagnoses by ophthalmological experts. [Means for solving the problem]

[0005] A diagnostic device according to one embodiment of the present invention includes a reception unit, a learning model, a processing unit, and a display unit. The reception unit receives a diagnostic image and a plurality of diagnostic questions corresponding to images characteristic of a disease included in the diagnostic image. The learning model is a learning model that has undergone machine learning using, as explanatory variables, a dataset of pairs of learning images obtained by capturing an image of a diseased part and learning questions corresponding to images characteristic of the disease included in the learning image, and learning answers to the learning questions as objective variables. The processing unit inputs the diagnostic image and the diagnostic questions to the learning model, thereby obtaining diagnostic answers to the diagnostic questions from the learning model, and inputs the diagnostic image and the diagnostic questions to the learning model as many times as the number of diagnostic questions received by the reception unit. The display unit displays the diagnostic image, the plurality of diagnostic questions, and the plurality of diagnostic answers.

[0006] A diagnostic system according to one embodiment of the present invention is a system in which a diagnostic device and a server device are connected via a network. The diagnostic device includes a reception unit, a first communication unit, and a display unit. The reception unit receives diagnostic images and multiple diagnostic questions for images characteristic of a disease included in the diagnostic images. The first communication unit transmits the diagnostic images and multiple diagnostic questions received by the reception unit to the server device via the network and receives multiple learning answers for the multiple diagnostic questions in response. The display unit displays the diagnostic images, the multiple diagnostic questions, and the multiple diagnostic answers. The server device includes a learning model, a processing unit, and a second communication unit. The learning model is a model that has undergone machine learning using a data set that pairs learning images obtained by capturing images of a diseased area with learning questions for images characteristic of the disease included in the learning images as explanatory variables, and the learning answers to the learning questions as objective variables. The processing unit inputs diagnostic images and diagnostic questions to the learning model, thereby obtaining diagnostic answers from the learning model, and inputs diagnostic images and diagnostic questions to the learning model as many times as the number of diagnostic questions. The second communication unit receives a diagnostic image and a plurality of diagnostic questions from the diagnostic device via the network and outputs them to the processing unit, and also obtains a plurality of diagnostic answers from the processing unit in response to the output of the diagnostic image and the plurality of diagnostic questions to the processing unit and transmits them to the diagnostic device via the network.

[0007] A diagnostic method according to one embodiment of the present invention includes the following three steps. (1) Accepting diagnostic images and multiple diagnostic questions for images characteristic of diseases contained in the diagnostic images. (2) A data set of pairs of learning images obtained by imaging a diseased area and learning questions for images characteristic of the disease contained in the learning images is used as an explanatory variable, and diagnostic images and diagnostic questions are input to a learning model that has undergone machine learning using learning answers to the learning questions as a target variable, thereby obtaining diagnostic answers to the diagnostic questions from the learning model, and inputting diagnostic images and diagnostic questions to the learning model as many times as the number of diagnostic questions. (3) Displaying diagnostic images, multiple diagnostic questions, and multiple diagnostic answers [Effects of the Invention]

[0008] In a diagnostic device, diagnostic system, and diagnostic method according to one embodiment of the present invention, a dataset is used in which training images and training questions corresponding to characteristic images of diseases included in the training images are paired as explanatory variables. Then, a diagnostic image and multiple diagnostic questions corresponding to characteristic images of diseases included in the diagnostic images are input to a trained learning model, and diagnostic answers to the diagnostic questions are obtained from the learning model. This makes it possible, for example, to input a diagnostic image, multiple diagnostic questions, and multiple diagnostic answers into an external device equipped with open AI, and obtain the name of the disease included in the diagnostic image from the external device in response. As a result, diagnostic results with a higher accuracy rate can be obtained compared to when using a conventional learning model that uses only images as explanatory variables. Furthermore, the accuracy rate of diagnoses of various diseases using AI can be equal to or higher than that of diagnoses by ophthalmological experts. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a diagnostic system according to a first embodiment of the present invention. [Figure 2] This is a diagram conceptually showing the learning of the learning model in Figure 1 and how output data is obtained from the trained learning model. [Figure 3]FIG. 10 is a diagram illustrating an example of a learning interface. [Figure 4] 4 is a diagram showing an example of a learning procedure using the learning interface of FIG. 3 in the diagnostic system of FIG. 1. [Figure 5] FIG. 10 is a diagram illustrating an example of a diagnostic interface. [Figure 6] 6 is a diagram showing an example of a diagnostic procedure using the diagnostic interface of FIG. 5 in the diagnostic system of FIG. [Figure 7] FIG. 2 is a diagram illustrating a modified example of the schematic configuration of the diagnostic system of FIG. [Figure 8] FIG. 2 is a diagram illustrating a modified example of the schematic configuration of the diagnostic system of FIG. [Figure 9] FIG. 2 is a diagram illustrating a modified example of the schematic configuration of the diagnostic system of FIG. [Figure 10] FIG. 10 is a diagram illustrating an example of a schematic configuration of a diagnostic system according to a second embodiment of the present invention. [Figure 11] FIG. 11 is a diagram illustrating a modified example of the schematic configuration of the diagnostic system of FIG. [Figure 12] FIG. 11 is a diagram illustrating a modified example of the schematic configuration of the diagnostic system of FIG. [Figure 13] FIG. 11 is a diagram illustrating a modified example of the schematic configuration of the diagnostic system of FIG. [Figure 14] FIG. 10 is a diagram illustrating an example of a schematic configuration of a diagnostic system according to a third embodiment of the present invention. [Figure 15] FIG. 15 is a diagram illustrating a modified example of the schematic configuration of the diagnostic system of FIG. [Figure 16] FIG. 15 is a diagram illustrating a modified example of the schematic configuration of the diagnostic system of FIG. [Figure 17] FIG. 15 is a diagram illustrating a modified example of the schematic configuration of the diagnostic system of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following description is a specific example of the present invention, and the present invention is not limited to the following aspects. The description will be given in the following order. 1. First embodiment (FIGS. 1 to 6) Example of setting up a learning model on a server device 2. Modification of the First Embodiment Modification 2-1: Example in which a learning model and disease name dictionary data are provided in a server device (Fig. 7) Modification 2-2: Example in which the learning model and medical interview data are stored in the server device (Fig. 8) Modification 2-3: The server device stores the learning model, the disease name dictionary data, and Example of questionnaire data (Figure 9) 3. Second embodiment (FIG. 10) Example of a learning model installed in a diagnostic device 4. Modification of the Second Embodiment Modification 4-1: Example of providing a learning model and disease name dictionary data in a diagnostic device (Fig. 11) Modification 4-2: Example of providing a learning model and interview data to a diagnostic device (Fig. 12) Modification 4-3: Learning model in diagnostic device, Example of disease name dictionary data and medical interview data (Figure 13) 5. Third embodiment (FIG. 14) Example of a diagnostic device equipped with a learning model and patient information database 6. Modification of the Third Embodiment Modification 6-1: A learning model, a patient information database, and Example of disease name dictionary data (Figure 15) Modification 6-2: A learning model, a patient information database, and Example of providing medical interview data (Figure 16) Variation 6-3: Learning model, patient information database, Example of disease name dictionary data and medical interview data (Figure 17)

[0011] <1. First embodiment> [composition] FIG. 1 shows an example of a schematic configuration of a diagnostic system 1 according to a first embodiment of the present invention. The diagnostic system 1 includes, for example, a diagnostic device 10, a patient information database 20, and a server device 30. The diagnostic device 10, the patient information database 20, and the server device 30 are connected via a network 40. The diagnostic device 10 is configured to be able to communicate with the patient information database 20 and the server device 30 via the network 40. The patient information database 20 is configured to be able to communicate with the diagnostic device 10 and the server device 30 via the network 40. The server device 30 is configured to be able to communicate with the diagnostic device 10 and the patient information database 20 via the network 40.

[0012] The network 40 is, for example, a network that performs communication using a communication protocol (TCP / IP) that is commonly used on the Internet. The network 40 may be, for example, a secure network that performs communication using a communication protocol unique to that network. The network 40 is, for example, the Internet, an intranet, or a local area network. The network 40 may be connected to the diagnostic device 10, the patient information database 20, and the server device 30 via, for example, a wired LAN (Local Area Network) such as Ethernet, a wireless LAN such as Wi-Fi, or a mobile phone line.

[0013] (Diagnostic device 10) The diagnostic device 10 includes, for example, a control unit 11, a memory 12, a network IF 13, an input unit 14, and a display unit 15, as shown in FIG.

[0014] The control unit 11 is configured to include a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and executes, for example, a web browser program, an operating system, etc. (not shown) stored in the memory 12. The control unit 11 further executes, for example, a program 12A stored in the memory 12. The program 12A is a program including a series of procedures (see FIGS. 4 and 6) included in the "diagnostic method" of the present invention. The program 12A causes the control unit 11 to execute the series of procedures (see FIGS. 4 and 6) included in the "diagnostic method" of the present invention. The series of procedures that the program 12A causes the control unit 11 to execute will be described in detail later.

[0015] The memory 12 stores programs (for example, a web browser program and an operating system) executed by the control unit 11. The memory 12 is configured by, for example, a random access memory (RAM), a read only memory (ROM), an auxiliary storage device (such as a hard disk), etc. The memory 12 further stores a program 12A.

[0016] The display unit 15 is formed of a display device such as a liquid crystal panel or an organic EL (Electro Luminescence) panel. The display unit 15 displays an image based on a video signal from the control unit 11 on a display screen. The display screen of the display unit 15 displays, for example, a learning interface IFa as shown in Fig. 3 and a diagnosis interface IFb as shown in Fig. 5.

[0017] The input unit 14 receives instructions from the outside (for example, a user) and outputs the received instructions to the control unit 11. The input unit 14 may be, for example, a mechanical input interface including buttons, dials, etc., or a voice input interface including a microphone, etc. The input unit 14 may be, for example, a touch panel provided on the display screen of the display unit 15. The network IF 13 is a communication interface for communicating with the patient information database 20 and the server device 30 via the network 40.

[0018] (Server device 30) The server device 30 includes, for example, a control unit 31, a learning model 32, and a network IF 33, as shown in FIG.

[0019] The control unit 31 is configured to include a CPU, a GPU, and the like, and inputs data input from the diagnostic device 10 via the network IF 33 to the learning model 32. The control unit 31 further outputs data output from the learning model 32 to the diagnostic device 10 via the network IF 33. The network IF 33 is a communication interface for communicating with the diagnostic device 10 and the patient information database 20 via the network 40.

[0020] The learning model 32 is a model that has undergone machine learning using a data set Dset_test, which pairs a training image Itest and a training question Qtest, as an explanatory variable, and a training answer Atest as a target variable, as shown in the upper part of Figure 2, for example. When a data set Dset_diag, which pairs a diagnostic image Idiag and a diagnostic question Qdiag, is input, the learning model 32 outputs a diagnostic answer Adiag, as shown in the lower part of Figure 2, for example. The training image Itest and the diagnostic image Idiag are images obtained by capturing an image of a diseased part.

[0021] The training question Qtest is a question about a characteristic image of a disease contained in the training image Itest. The training answer Atest is an answer to the training question Qtest. The diagnostic question Qdiag is a question about a characteristic image of a disease contained in the diagnostic image Idiag. The diagnostic answer Adiag is an answer to the diagnostic question Qdiag.

[0022] The training image Itest and the diagnostic image Idiag are fundus photographs depicting diseases such as diabetic retinopathy, macular degeneration, glaucoma, retinal detachment, or retinal vein occlusion. Examples of the training question Qtest and the diagnostic question Qdiag include the questions shown below. The training question Qtest and the diagnostic question Qdiag may be, for example, a question that directly asks for the name of a disease ("What disease is in this image?"). Examples of the training answer Atest and the diagnostic answer Adiag include, for example, the answers shown below. When the training question Qtest and the diagnostic question Qdiag are questions that directly ask for the name of a disease ("What disease is in this image?"), the training answer Atest and the diagnostic answer Adiag may be the name of a disease (for example, "You have severe diabetic retinopathy").

[0023] [Question] [Answer] "What kind of image is this?" "It's an ultra-wide-angle fundus image." "What do you see?" "A broom hemorrhage." "Where is the bleeding?" "Around the upper arcade vessels." "What is the extent of the bleeding?" "The bleeding is from the first to second branches of the retinal vein."

[0024] (Patient Information Database 20) The patient information database 20 stores, for example, multiple patient information data. Each patient information data includes, for example, one or multiple captured images obtained by an examination, a patient name, a patient ID (Identification), and the like. One or multiple captured images included in specific multiple patient information data in the patient information database 20 correspond to, for example, learning images Itest. Also, one or multiple captured images included in specific multiple patient information data in the patient information database 20 correspond to, for example, diagnostic images Idiag.

[0025] [Operation] Next, we will explain an example of the operation of the diagnostic system 1. First, we will explain an example of the machine learning procedure of the learning model 32, and then we will explain an example of the procedure for diagnosing a disease included in the diagnostic image Idiag using the trained learning model 32.

[0026] (Machine Learning) Fig. 3 shows an example of the learning interface IFa displayed on the display screen of the display unit 15. Fig. 4 shows an example of a learning procedure using the learning interface IFa. The learning interface IFa includes, for example, an image input window 110, a disease name selection tab 120, a findings selection button 130, a text display window 140, a create button 150, a Q&A display window 160, a send button 170, a next button 180, and an end button 190.

[0027] First, the user instructs diagnostic device 10 to display learning interface IFa, for example, by operating input unit 14. Then, diagnostic device 10 (control unit 11) instructs display unit 15 to display learning interface IFa, and display unit 15 displays learning interface IFa on the display screen in accordance with the instruction (step S101).

[0028] Next, the user selects the image input window 110 in the learning interface IFa, for example, by operating the input unit 14. Then, the control unit 11 instructs the display unit 15 to display a display prompting the input of the learning image Itest into the selected image input window 110, and the display unit 15 displays a display prompting the input of the learning image Itest on the display screen in accordance with the instruction. Subsequently, when the user inputs the learning image Itest, for example, by operating the input unit 14, the control unit 11 acquires the input learning image Itest (step S102).

[0029] Next, the user selects one disease name tab included in the disease name selection tabs 120 in the learning interface IFa, for example, by operating the input unit 14. The control unit 11 then instructs the display unit 15 to display a message indicating that one disease name tab has been selected and to display a finding selection button 130 including a list of findings corresponding to the selected disease name tab. In response to the instruction, the display unit 15 displays a message indicating that one disease name tab has been selected and the finding selection button 130 including a list of findings corresponding to the selected disease name tab. At this time, if "DR" is selected as the disease name tab, the finding selection button 130 includes, for example, buttons for "UWF," "PPF," "Blot," "Dot," "HE," "SE," "ME," "Macular," "Intermediate," "Peripheral," "Mild," "Moderate," and "Severe."

[0030] Next, when the user selects at least one finding included in the finding selection buttons 130, for example, by operating the input unit 14, the control unit 11 acquires the disease name that has already been selected and the currently selected finding (step S103). Subsequently, the control unit 11 generates a sentence including the acquired disease name and finding, and instructs the display unit 15 to display a sentence display window 140 including the generated sentence, and the display unit 15 displays the sentence display window 140 including the generated sentence on the display screen in accordance with the instruction (step S104).

[0031] Next, when the user selects the generate button 150, for example, by operating the input unit 14, the control unit 11 sets the input learning image Itest and the sentence (explanatory sentence) generated in step S104 as a dataset Dset_out1. Subsequently, the control unit 11 transmits the dataset Dset_out1 to an external device equipped with an open AI such as ChatGPT via the network IF13. In response to the transmission of the dataset Dset_out1, the diagnostic device 10 (control unit 11) receives the explanatory sentence Stest from the external device via the network IF13. The explanatory sentence Stest is composed of multiple Q&A, and each Q&A is composed of a learning question Qtest and a learning answer Atest.

[0032] That is, when the generate button 150 is selected, the control unit 11 requests the external device to generate explanatory text S Test (step S106). Then, the control unit 11 acquires explanatory text S Test from the external device in response to the request (step S106). The control unit 11 instructs the display unit 15 to display the Q&A display window 160 including the acquired explanatory text S Test, and the display unit 15, in accordance with the instruction, displays the Q&A display window 160 including the acquired explanatory text S Test on the display screen (step S106).

[0033] Next, when the user selects the send button 170, for example, by operating the input unit 14, the control unit 11 requests the server device 30 to perform machine learning on the learning model 32. Specifically, the control unit 11 transmits transmission data including the learning image Itest and the explanatory text Stest to the server device 30 via the network IF 13. The server device 30 (control unit 31) receives the transmission data via the network IF 33. The control unit 31 performs machine learning on the learning model 32 using a data set Dset_test, which pairs the learning image Itest and the learning question Qtest, as an explanatory variable and using the learning answer Atest as a target variable. The control unit 31 performs this machine learning as many times as the number of learning questions Qtest.

[0034] Subsequently, when the user selects the next button 180, the control unit 11 continues the learning (step S107; Y) and executes step S102. On the other hand, when the user selects the end button 190, for example, by operating the input unit 14, the control unit 11 ends the learning (step S107; N). In this way, machine learning for the learning model 32 is executed.

[0035] (diagnosis) Fig. 5 shows an example of the diagnostic interface IFb displayed on the display screen of the display unit 15. Fig. 6 shows an example of a diagnostic procedure using the diagnostic interface IFb. The diagnostic interface IFb displays, for example, an image input window 210, a question input window 220, a send button 230, a Q&A display window 240, a send button 250, a disease name display window 260, and an end button 270.

[0036] First, the user instructs the diagnostic device 10 to display the diagnostic interface IFb, for example, by operating the input unit 14. Then, the diagnostic device 10 (control unit 11) instructs the display unit 15 to display the diagnostic interface IFb, and the display unit 15 displays the diagnostic interface IFb on the display screen in accordance with the instruction (step S201).

[0037] Next, the user selects the image input window 210 in the diagnostic interface IFb, for example, by operating the input unit 14. Then, the control unit 11 instructs the display unit 15 to display a display prompting the user to input a diagnostic image Idiag into the selected image input window 210, and the display unit 15 displays a display prompting the user to input the diagnostic image Idiag on the display screen in accordance with the instruction. Next, when the user inputs the diagnostic image Idiag, for example, by operating the input unit 14, the control unit 11 acquires the input diagnostic image Idiag (step S202).

[0038] Next, the user inputs a question (diagnostic question Qdiag) about the characteristic image of the disease included in the diagnostic image Idiag into the question input window 220 of the diagnostic interface IFb, for example, by operating the input unit 14. Then, the control unit 11 instructs the display unit 15 to display the question input window 220 including the input diagnostic question Qdiag, and the display unit 15 displays the question input window 220 including the input diagnostic question Qdiag on the display unit 15 in accordance with the instruction. Figure 5 illustrates an example in which four diagnostic questions Qdiag are displayed in the question input window 220.

[0039] Next, the user selects the send button 230, for example, by operating the input unit 14. Then, the control unit 11 transmits a data set Dset_diag, which is a pair of the diagnostic image Idiag and the diagnostic question Qdiag, to the server device 30 via the network IF 13. The control unit 11 transmits the same number of data sets Dset_diag as the number of diagnostic questions Qdiag to the server device 30 via the network IF 13.

[0040] The server device 30 (control unit 31) receives the data set Dset_diag via the network IF 33. The control unit 31 inputs the data set Dset_diag to the learning model 32. Every time the data set Dset_diag is input, the control unit 31 inputs the input data set Dset_diag to the learning model 32.

[0041] When the data set Dset_diag is input, the learning model 32 outputs a diagnostic answer Adiag corresponding to the diagnostic question Qdiag. The server device 30 (control unit 31) transmits the diagnostic answer Adiag output from the learning model 32 to the diagnostic device 10 via the network IF 33. The diagnostic device 10 (control unit 11) receives (acquires) the diagnostic answer Adiag from the server device 30 via the network IF 13 (step S206). When the control unit 11 receives multiple diagnostic answers Adiag for the same diagnostic image Idiag, it generates an explanatory sentence Sdiag consisting of multiple Q&As paired with the diagnostic question Qdiag and the received diagnostic answer Adiag. The control unit 11 instructs the display unit 15 to display a Q&A display window 240 including the explanatory sentence Sdiag, and the display unit 15, in accordance with the instruction, displays the Q&A display window 240 including the acquired explanatory sentence Sdiag on the display screen (step S204). FIG. 5 shows an example of four sets of diagnostic questions Qdiag and diagnostic answers Adiag displayed in the Q&A display window 240.

[0042] Next, the user selects the send button 250, for example, by operating the input unit 14. The control unit 11 then transmits a data set Dset_out2, which includes a pair of diagnostic image Idiag and explanatory text Sdiag, to an external device equipped with an open AI such as ChatGPT via the network IF 13 (step S205). In response to the transmission of the data set Dset_out2, the diagnostic device 10 (control unit 11) receives (acquires) a disease name Di_diag from the external device via the network IF 13 (step S206). The disease name Di_diag corresponds to the name of the disease included in the diagnostic image Idiag. The control unit 11 instructs the display unit 15 to display a disease name display window 260 including the received (acquired) disease name Di_diag, and the display unit 15, in accordance with the instruction, displays the disease name display window 260 including the acquired disease name Di_diag on the display screen (step S206).

[0043] Next, when the user selects the end button 270, for example, by operating the input unit 14, the control unit 11 ends the diagnosis (step S207; Y). In this way, machine learning of the learning model 32 and diagnosis using the learning model 32 are executed.

[0044] [effect] Next, the effects of the diagnostic system 1 will be described.

[0045] Fundus photographs obtained through fundus examinations can reveal diseases such as diabetic retinopathy, macular degeneration, glaucoma, retinal detachment, and retinal vein occlusion. In recent years, attempts have begun to use AI to diagnose these diseases (see Non-Patent Document 1).

[0046] However, it is known that when conducting diagnostic tests for the above-mentioned various diseases, the accuracy rate of diagnoses using AI for the above-mentioned various diseases is significantly lower than the accuracy rate of diagnoses made by ophthalmological experts. Therefore, it is desirable to increase the accuracy rate of diagnoses for the above-mentioned various diseases using AI to be higher than the accuracy rate of diagnoses made by ophthalmological experts.

[0047] On the other hand, in this embodiment, a plurality of sets of data sets Dset_test, each pairing a training image Itest with a description Stest of a characteristic image of a disease included in the training image Itest, are used as explanatory variables. Then, the diagnostic image Idiag and the description Sdiag of the characteristic image of the disease included in the diagnostic image Idiag are input to a trained learning model 32, whereby the disease name Di_diag of the disease included in the diagnostic image Idiag is obtained from the learning model 32. This makes it possible to obtain a diagnostic result with a higher accuracy rate than when a conventional learning model using only images as explanatory variables is used.

[0048] Furthermore, in this embodiment, the explanatory text Stest is composed of a plurality of Q&As, each of which includes a learning question for a characteristic image of a disease included in the learning image Itest and a learning answer to the learning question. The explanatory text Sdiag is also composed of a plurality of Q&As, each of which includes a diagnostic question for a characteristic image of a disease included in the diagnostic image Idiag and a diagnostic answer to the diagnostic question. This makes it possible to obtain diagnostic results with a higher accuracy rate than when using a conventional learning model in which only images are used as explanatory variables during learning and only images are input during diagnosis.

[0049] In this embodiment, when the diagnostic interface IFb is displayed on the display unit 15, the diagnostic image Idiag is input to the input unit 14, and the explanatory text Sdiag is input to the network IF 13. By inputting the diagnostic image Idiag and the explanatory text Sdiag as the data set Dset_diag to the learning model 32, the disease name Di_diag of the disease included in the diagnostic image Idiag can be obtained. As a result, a diagnostic result with a higher accuracy rate can be obtained compared to when a conventional learning model is used in which only an image is input during diagnosis.

[0050] Furthermore, in this embodiment, when the learning interface IFa is displayed on the display unit 15, the learning model 32 is trained by inputting the learning image Itest and the explanatory text Stest into the learning model 32 as the data set Dset_test. As a result, the disease name Di_diag of the disease included in the diagnostic image Idiag can be obtained by inputting the diagnostic image Idiag and the explanatory text Sdiag into the learning model 32 as the data set Dset_diag. As a result, a diagnostic result with a higher accuracy rate can be obtained compared to when a conventional learning model in which only an image is input during diagnosis is used.

[0051] <2. Modification of the First Embodiment> Next, a modification of the diagnostic system 1 according to the first embodiment will be described.

[0052] [Variation 2-1] Fig. 7 shows a modified example of the diagnostic system 1 according to the first embodiment. In the diagnostic system 1, the server device 30 may further include disease name dictionary data 34, for example, as shown in Fig. 7. The disease name dictionary data 34 serves to complement the explanatory text Sdiag. The disease name dictionary data 34 includes, for example, a plurality of disease names and, for each disease name, its definition, symptoms, and test findings.

[0053] Test findings refer to characteristic findings that appear in images obtained by testing. For example, suppose the disease is glaucoma. In this case, the definition of the disease is, for example, "a disease characterized by functional and structural abnormalities of the eye, with characteristic changes in the optic nerve and visual field, and optic nerve damage can usually be improved or suppressed by sufficiently lowering intraocular pressure." Symptoms include, for example, "the appearance of a blind spot (scotoma), or a narrowing of the range of vision (visual field), eye pain, redness, blurred vision, as well as headache and nausea." Test findings for images obtained by visual field testing include, for example, "a characteristic arcuate scotoma and temporal visual field defect are observed." Test findings for images obtained by fundus examination include, for example, "enlarged cup and nerve fiber loss are observed, and disc hemorrhage is observed."

[0054] In this modification, when the send button 250 is selected after executing the diagnosis step S204, the control unit 11 transmits a data set Dset_out2 including the diagnostic image Idiag, the explanatory text Sdiag, and the disease name dictionary data 34 to an external device equipped with an open AI such as ChatGPT via the network IF 13 (step S205). In response to the transmission of the data set Dset_out2, the diagnostic device 10 (control unit 11) receives (acquires) the disease name Di_diag from the external device via the network IF 13 (step S206).

[0055] In this way, in this modification, when an open AI diagnosis is performed in an external device, not only the diagnostic image Idiag and explanatory text Sdiag but also the disease name dictionary data 34 is used. This makes it possible to obtain a more accurate diagnosis result (disease name Di_diag).

[0056] [Variation 2-2] Fig. 8 shows a modified example of the diagnostic system 1 according to the first embodiment. In the diagnostic system 1, the server device 30 may further include medical interview data 35, for example, as shown in Fig. 8. The medical interview data 35 is data on subjective symptoms and the like obtained by interviewing a patient. The medical interview data 35 serves to complement the explanatory text Sdiag.

[0057] In this modification, when the send button 250 is selected after performing the diagnosis step S204, the control unit 11 transmits a data set Dset_out2 including the diagnostic image Idiag, the explanatory text Sdiag, and the medical interview data 35 to an external device equipped with an open AI such as ChatGPT via the network IF 13 (step S205). In response to the transmission of the data set Dset_out2, the diagnostic device 10 (control unit 11) receives (acquires) the disease name Di_diag from the external device via the network IF 13 (step S206).

[0058] In this way, in this modification, when an open AI diagnosis is performed in an external device, not only the diagnostic image Idiag and explanatory text Sdiag but also the medical interview data 35 is used. This makes it possible to obtain a more accurate diagnosis result (disease name Di_diag).

[0059] [Variation 2-3] Fig. 9 shows a modified example of the diagnostic system 1 according to the first embodiment. In the diagnostic system 1, the server device 30 may further include, for example, disease name dictionary data 34 and medical interview data 35, as shown in Fig. 9.

[0060] In this modification, when the send button 250 is selected after executing the diagnosis step S204, the control unit 11 transmits a data set Dset_out2 including the diagnostic image Idiag, the explanatory text Sdiag, the disease name dictionary data 34, and the medical interview data 35 to an external device equipped with an open AI such as ChatGPT via the network IF 13 (step S205). In response to the transmission of the data set Dset_out2, the diagnostic device 10 (control unit 11) receives (acquires) the disease name Di_diag from the external device via the network IF 13 (step S206).

[0061] In this way, in this modification, when an open AI diagnosis is performed in an external device, not only the diagnostic image Idiag and explanatory text Sdiag but also the disease name dictionary data 34 and the medical interview data 35 are used. This makes it possible to obtain a more accurate diagnosis result (disease name Di_diag).

[0062] In this modification, the control unit 11 may transmit the diagnostic image Idiag to an external device equipped with an open AI such as ChatGPT via the network IF 13, and in response, receive (acquire) findings (examination findings) observed in the diagnostic image Idiag from the external device via the network IF 13. In this case, the control unit 11 may compare the symptoms and examination findings (first data) included in the disease name dictionary data 34 with the interview data 35 and the examination findings (second data) obtained from the external device. The control unit 11 may, for example, search for exact or partial matches of terms (e.g., terms such as "nipple" and "bleeding") between the first data and the second data. For example, the control unit 11 may transmit the first data and the second data to an external device equipped with an open AI such as GPT, and in response, receive (acquire) from the external device the disease name Di_diag, which corresponds to the symptom and test finding with the highest similarity in the disease name dictionary data 34, obtained by performing a search for an exact or partial match of terms (e.g., terms such as "nipple" and "bleeding") in the first data and the second data. For example, the control unit 11 may transmit the first data and the second data to an external device equipped with an open AI such as GPT that can vectorize sentences using a language model, and in response, receive (acquire) from the external device the disease name Di_diag, which corresponds to the symptom and test finding with the highest similarity in the disease name dictionary data 34, obtained by performing a match search based on vector similarity between the first data and the second data. Even in this case, a more accurate diagnosis result (disease name Di_diag) can be obtained.

[0063] 3. Second Embodiment Next, a diagnostic system 2 according to a second embodiment of the present invention will be described. In this embodiment, the diagnostic system 2 includes a diagnostic device 10 and a patient information database 20, as shown in FIG. 10, for example. In this embodiment, the diagnostic device 10 includes a control unit 11, a memory 12, a learning model 32, a network IF 13, an input unit 14, and a display unit 15, as shown in FIG. 10, for example. In this embodiment, the learning model 32, which was provided in the server device 30, is provided in the diagnostic device 10. Even in this case, the same effects as those of the above embodiment can be obtained.

[0064] <4. Modification of the Second Embodiment> Next, a modified example of the diagnostic system 2 according to the second embodiment will be described.

[0065] [Variation 4-1] Fig. 11 shows a modified example of the diagnostic system 2 according to the second embodiment. In the diagnostic system 2, the diagnostic device 10 may further include disease name dictionary data 34, for example, as shown in Fig. 11. In this case, the diagnostic device 10 may perform the same processing as that described in the above modified example 2-1. In this case, it is possible to obtain a more accurate diagnostic result (disease name Di_diag).

[0066] [Variation 4-2] Fig. 12 shows a modified example of the diagnostic system 2 according to the second embodiment. In the diagnostic system 2, the diagnostic device 10 may further include, for example, interview data 35 as shown in Fig. 12. In this case, the diagnostic device 10 may perform the same processing as that described in the above modified example 2-2. In this case, it is possible to obtain a more accurate diagnostic result (disease name Di_diag).

[0067] [Variation 4-3] Fig. 13 shows a modified example of the diagnostic system 2 according to the second embodiment. In the diagnostic system 2, the diagnostic device 10 may further include, for example, disease name dictionary data 34 and medical interview data 35, as shown in Fig. 13. In this case, the diagnostic device 10 may perform the same processing as that described in the above modified example 2-3. In this case, it is possible to obtain a more accurate diagnostic result (disease name Di_diag).

[0068] 5. Third Embodiment Next, a diagnostic device 3 according to a third embodiment of the present invention will be described. In this embodiment, the diagnostic device 3 includes, for example, a control unit 11, a memory 12, a learning model 32, a patient information database 20, an input unit 14, and a display unit 15, as shown in FIG. 14. In this embodiment, the learning model 32, which was previously provided in the server device 30, and the patient information database 20, which was previously provided externally, are provided in the diagnostic device 10, making the diagnostic device 3 a standalone device. Even in this case, the same effects as those of the above embodiments can be obtained.

[0069] 6. Modification of the Third Embodiment Next, a modified example of the diagnostic system 3 according to the third embodiment will be described.

[0070] [Variation 6-1] Fig. 15 shows a modified example of the diagnostic system 3 according to the third embodiment. In the diagnostic system 3, the diagnostic device 3 may further include disease name dictionary data 34, for example, as shown in Fig. 15. In this case, the diagnostic device 3 may perform the same processing as that performed by the diagnostic device 10 described in the above modified example 2-1. In this case, it is possible to obtain a more accurate diagnostic result (disease name Di_diag).

[0071] [Variation 6-2] Fig. 16 shows a modified example of the diagnostic system 3 according to the third embodiment. In the diagnostic system 3, the diagnostic device 3 may further include, for example, interview data 35 as shown in Fig. 16. In this case, the diagnostic device 3 may perform the same processing as that performed by the diagnostic device 10 described in the above modified example 2-2. In this case, it is possible to obtain a more accurate diagnostic result (disease name Di_diag).

[0072] [Variation 6-3] Fig. 17 shows a modified example of the diagnostic system 3 according to the third embodiment. In the diagnostic system 3, the diagnostic device 3 may further include, for example, disease name dictionary data 34 and medical interview data 35, as shown in Fig. 17. In this case, the diagnostic device 3 may perform the same processing as that performed by the diagnostic device 10 described in Modification 2-3 above. In this case, it is possible to obtain a more accurate diagnostic result (disease name Di_diag).

[0073] Although the present invention has been described above with reference to several embodiments and their modifications, the present invention is not limited to the embodiments and various modifications are possible. For example, in the embodiments, the disease names Di_test and Di_diag may be identifiers corresponding to the disease names. In this case, the control unit 11 may convert the identifiers into disease names and use the disease names obtained by the conversion as the disease names Di_test and Di_diag. [Explanation of symbols]

[0074] 1,2...diagnostic system, 3,10...diagnostic device, 11...control unit, 12...memory, 12A...program, 13...network IF, 14...input unit, 15...display unit, 20...patient information database, 30...server device, 31...control unit, 32...learning model, 33...network IF, 34...disease name dictionary data, 35...interview data, 40...network, 110...image input window, 120...disease name selection tab, 130...findings selection button, 140...text display window, 150...generation button, 16 0...QandA display window, 170...Send button, 180...next button, 190...exit button, 210...image input window, 220...question input window, 230...send button, 240...QandA display window, 250...send button, 260...disease name display window 270...exit button, Di_test, Di_diag...disease name, IFa...learning interface, IFb...diagnostic interface, Idiag...diagnostic image, Itest...learning image, Sdiag, Stest...explanatory text.

Claims

1. a receiving unit that receives a diagnostic image and a plurality of diagnostic questions for an image characteristic of a disease included in the diagnostic image; a learning model in which machine learning is performed using a data set of pairs of learning images obtained by imaging a diseased part and learning questions for characteristic images of the disease included in the learning images as explanatory variables, and learning answers to the learning questions as objective variables; a processing unit that inputs the diagnostic images and the diagnostic questions to the learning model, thereby obtaining diagnostic answers to the diagnostic questions from the learning model, and executes input of the diagnostic images and the diagnostic questions to the learning model for the number of diagnostic questions received by the receiving unit; a display unit that displays the diagnostic image, the plurality of diagnostic questions, and the plurality of diagnostic answers; Equipped with Diagnostic equipment.

2. the processing unit transmits the diagnostic image, the plurality of diagnostic questions, and the plurality of diagnostic answers to an external device equipped with an open AI, and receives, in response, from the external device, the disease name of the disease included in the diagnostic image; The display unit displays the disease name. The diagnostic device of claim 1 .

3. the processing unit transmits the diagnostic image, the plurality of diagnostic questions, the plurality of diagnostic answers, and disease name dictionary data to an external device equipped with an open AI, and receives, in response, from the external device, the disease name of the disease included in the diagnostic image; The display unit displays the disease name. The diagnostic device of claim 1 .

4. the processing unit transmits the diagnostic image, the plurality of diagnostic questions, the plurality of diagnostic answers, and the interview data to an external device equipped with an open AI, and receives, in response, from the external device, the disease name of the disease included in the diagnostic image; The display unit displays the disease name. The diagnostic device of claim 1 .

5. The processing unit transmits disease name dictionary data in which symptoms and test findings are associated with each disease name, and the test findings and interview data obtained from the diagnostic image to an external device equipped with open AI, and receives, in response, from the external device, the disease name corresponding to the symptom and test finding with the highest similarity in the disease name dictionary data, obtained by performing an exact match search or partial match search of terms, or a match search using vector similarity, as the disease name of the disease included in the diagnostic image; The display unit displays the disease name. The diagnostic device of claim 1 .

6. the display unit displays a diagnostic interface; The receiving unit receives the diagnostic image and the plurality of diagnostic questions while the diagnostic interface is displayed on the display unit. The diagnostic device of claim 1 .

7. the display unit displays a learning interface; the receiving unit receives the data set while the learning interface is displayed on the display unit; The processing unit performs machine learning on the learning model by inputting the data set and the learning answer into the learning model. The diagnostic device of claim 6.

8. A diagnostic system in which a diagnostic device and a server device are connected via a network, The diagnostic device comprises: a receiving unit that receives a diagnostic image and a plurality of diagnostic questions for an image characteristic of a disease included in the diagnostic image; a first communication unit that transmits the diagnostic image and the plurality of diagnostic questions received by the reception unit to the server device via the network and receives a plurality of learning answers to the plurality of diagnostic questions in response thereto; a display unit that displays the diagnostic image, the plurality of diagnostic questions, and the plurality of diagnostic answers; Equipped with The server device a learning model in which machine learning is performed using a data set of pairs of learning images obtained by imaging a diseased part and learning questions for characteristic images of the disease included in the learning images as explanatory variables, and learning answers to the learning questions as objective variables; a processing unit that inputs the diagnostic images and the diagnostic questions to the learning model, thereby obtaining the diagnostic answers from the learning model, and executes input of the diagnostic images and the diagnostic questions to the learning model for the number of the diagnostic questions; a second communication unit that receives the diagnostic image and the plurality of diagnostic questions from the diagnostic device via the network and outputs them to the processing unit, and that obtains the plurality of diagnostic answers from the processing unit as responses to the output of the diagnostic image and the plurality of diagnostic questions to the processing unit and transmits them to the diagnostic device via the network; Equipped with Diagnostic system.

9. receiving a diagnostic image and a plurality of diagnostic questions for images characteristic of a disease included in the diagnostic image; inputting the diagnostic images and the diagnostic questions into a learning model that has undergone machine learning using a data set of pairs of learning images obtained by imaging a diseased part and learning questions for images characteristic of the disease included in the learning images as explanatory variables, and learning answers to the learning questions as objective variables, thereby obtaining diagnostic answers to the diagnostic questions from the learning model, and inputting the diagnostic images and the diagnostic questions into the learning model the same number of times as the number of diagnostic questions; and displaying the diagnostic image, the plurality of diagnostic questions, and the plurality of diagnostic answers. Diagnostic methods.

Images