Information processing system, information processing apparatus, information processing method, and program

Abstract

An information processing system has an image acquisition device that acquires an eye-to-be-examined image data of a patient, and a first information processing device that can communicate with the image acquisition device and stores the eye-to-be-examined image data, the image acquisition device transmitting first transmission data to the first information processing device, the first transmission data including the eye-to-be-examined image data and additional information for determining an image diagnostic device that performs an image diagnosis on the eye-to-be-examined image data, the first information processing device, if receiving the first transmission data from the image acquisition device, storing the eye-to-be-examined image data, and based on the additional information, determining at least one of a first image diagnostic device that performs a first image diagnosis on the eye-to-be-examined image data and a second image diagnostic device that performs a second image diagnosis different from the first image diagnosis on the eye-to-be-examined image data, and transmitting second transmission data including the eye-to-be-examined image data to the determined image diagnostic device.

Description

[0001] Reference-based introduction

[0002] This application claims priority to U.S. Provisional Application No. 62 / 880,980, filed July 31, 2019, and is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to an information processing system, an information processing apparatus, an information processing method, and a program. BACKGROUND

[0004] An ophthalmic information processing server capable of performing ophthalmic image analysis is known (see Patent Literature 1). However, in the past, there has been no consideration of selecting an appropriate ophthalmic information processing server according to a device that has captured an image or a party that has commissioned image analysis.

[0005] PRIOR ART DOCUMENTS

[0006] PATENT LITERATURE

[0007] Patent Literature 1: Japanese Patent No. 5951086 SUMMARY

[0008] An information processing system that is one technical solution of the invention disclosed in the present application has: an image acquisition apparatus that acquires a patient's examination eye image data; and a first information processing apparatus that is capable of communicating with the image acquisition apparatus and stores the examination eye image data, the image acquisition apparatus performing first transmission processing that transmits first transmission data including the examination eye image data and additional information for determining an image diagnostic apparatus that performs image diagnosis on the examination eye image data to the first information processing apparatus, the first information processing apparatus performing: storage processing that stores the examination eye image data if the first transmission data is received from the image acquisition apparatus; determination processing that determines at least one of a first image diagnostic apparatus that performs first image diagnosis on the examination eye image data and a second image diagnostic apparatus that performs second image diagnosis different from the first image diagnosis on the examination eye image data, based on the additional information; and second transmission processing that transmits second transmission data including the examination eye image data to the determined image diagnostic apparatus.

[0009] An information processing system according to one aspect of the invention disclosed in the present application includes an image acquisition device that acquires an eye to be examined image data of a patient, and a first information processing device that is capable of communicating with the image acquisition device and stores the eye to be examined image data. The image acquisition device performs a first transmission process of transmitting first transmission data including the eye to be examined image data and additional information for determining an artificial intelligence for image diagnosis of the eye to be examined image data to the first information processing device. The first information processing device performs a storage process of storing the eye to be examined image data if the first transmission data is received from the image acquisition device, a determination process of determining at least one of a first artificial intelligence for performing a first image diagnosis of the eye to be examined image data and a second artificial intelligence for performing a second image diagnosis of the eye to be examined image data different from the first image diagnosis based on the additional information, and a second transmission process of transmitting second transmission data including the eye to be examined image data and determination information of the determined artificial intelligence to an image diagnosis device including the determined artificial intelligence.

[0010] An information processing device according to one aspect of the invention disclosed in the present application includes a processor and a storage device. The storage device holds an eye to be examined image data of a patient, additional information of the eye to be examined image data, and correspondence information of the additional information and an image diagnosis device for image diagnosis of the eye to be examined image data. The processor determines at least one of a first image diagnosis device for performing a first image diagnosis of the eye to be examined image data and a second image diagnosis device for performing a second image diagnosis of the eye to be examined image data different from the first image diagnosis based on the correspondence information and the additional information of the eye to be examined image data, and transmits transmission data including the eye to be examined image data to the determined image diagnosis device.

[0011] An information processing device according to one aspect of the invention disclosed in the present application includes a processor and a storage device. The storage device holds an eye to be examined image data of a patient, additional information of the eye to be examined image data, and correspondence information of the additional information and an artificial intelligence for image diagnosis of the eye to be examined image data. The processor determines at least one of a first artificial intelligence for performing a first image diagnosis of the eye to be examined image data and a second artificial intelligence for performing a second image diagnosis of the eye to be examined image data different from the first image diagnosis based on the correspondence information and the additional information of the eye to be examined image data, and transmits second transmission data including the eye to be examined image data and determination information of the determined artificial intelligence to an image diagnosis device including the determined artificial intelligence.

[0012] An information processing method according to one aspect of the invention disclosed in this application is executed by an information processing apparatus including a processor and a storage device, the storage device holding examination eye image data of a patient, additional information of the examination eye image data, and correspondence information of the additional information and an image diagnostic apparatus that performs image diagnosis on the examination eye image data, in the information processing method, the processor determines at least one of a first image diagnostic apparatus and a second image diagnostic apparatus based on the correspondence information and the additional information of the examination eye image data, wherein the first image diagnostic apparatus performs first image diagnosis on the examination eye image data, and the second image diagnostic apparatus performs second image diagnosis on the examination eye image data different from the first image diagnosis, and the processor transmits transmission data including the examination eye image data to the determined image diagnostic apparatus.

[0013] An information processing method according to one aspect of the invention disclosed in this application is executed by an information processing apparatus including a processor and a storage device, the storage device holding examination eye image data of a patient, additional information of the examination eye image data, and correspondence information of the additional information and an image diagnostic apparatus that performs image diagnosis on the examination eye image data, in the information processing method, the processor determines at least one of a first image diagnostic apparatus and a second image diagnostic apparatus based on the correspondence information and the additional information of the examination eye image data, wherein the first image diagnostic apparatus performs first image diagnosis on the examination eye image data, and the second image diagnostic apparatus performs second image diagnosis on the examination eye image data different from the first image diagnosis, and the processor transmits transmission data including the examination eye image data to the determined image diagnostic apparatus.

[0014] A program according to one aspect of the invention disclosed in this application is for causing an information processing apparatus including a processor and a storage device to perform information processing, the storage device holding examination eye image data of a patient, additional information of the examination eye image data, and correspondence information of the additional information and an image diagnostic apparatus that performs image diagnosis on the examination eye image data, the program causing the processor to perform processing of determining at least one of a first image diagnostic apparatus and a second image diagnostic apparatus based on the correspondence information and the additional information of the examination eye image data, wherein the first image diagnostic apparatus performs first image diagnosis on the examination eye image data, and the second image diagnostic apparatus performs second image diagnosis on the examination eye image data different from the first image diagnosis, and processing of transmitting transmission data including the examination eye image data to the determined image diagnostic apparatus.

[0015] A program that is one of the technical solutions of the invention disclosed in this application is a program for causing an information processing apparatus to perform information processing, the information processing apparatus having a processor and a storage device, the storage device holding patient's eye image data, additional information of the eye image data, and additional information and corresponding information of an artificial intelligence that performs image diagnosis on the eye image data, the program causing the processor to perform processing of determining at least one of a first artificial intelligence and a second artificial intelligence based on the corresponding information and the additional information of the eye image data, wherein the first artificial intelligence is used to perform a first image diagnosis on the eye image data, and the second artificial intelligence performs a second image diagnosis on the eye image data different from the first image diagnosis; and processing of transmitting second transmission data including the eye image data and determination information of the determined artificial intelligence to an image diagnosis apparatus including the determined artificial intelligence. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is an explanatory diagram showing a structure example of an image diagnosis system in Embodiment 1.

[0017] Figure 2 is a block diagram showing a hardware structure example of a computer in Embodiment 1.

[0018] Figure 3 is a block diagram showing a functional structure example of a management server in Embodiment 1.

[0019] Figure 4 is a block diagram showing a functional structure example of a diagnosis server in Embodiment 1.

[0020] Figure 5 is a block diagram showing a functional structure example of an in-hospital server in Embodiment 1.

[0021] Figure 6 is a block diagram showing a functional structure example of a terminal in Embodiment 1.

[0022] Figure 7 is a timing chart showing an image diagnosis process of an image diagnosis system in Embodiment 1.

[0023] Figure 8 is a flowchart showing one example of diagnosis server decision processing in Embodiment 1.

[0024] Figure 9 is a flowchart showing one example of diagnosis server decision processing in Embodiment 1.

[0025] Figure 10 is one example of AI selection information in Embodiment 1.

[0026] Figure 11 is an example of a data structure of anonymized diagnosis target data in Example 1.

[0027] Figure 12 is an example of a display screen for displaying a diagnosis result in Example 1.

[0028] Figure 13 is an example of a display screen for displaying a diagnosis result in Example 1.

[0029] Figure 14 is an example of a display screen for displaying a diagnosis result in Example 1.

[0030] Figure 15 is an example of a display screen for displaying a diagnosis result in Example 1.

[0031] Figure 16 is an explanatory diagram showing a structure example of an image diagnosis system in Example 2.

[0032] Figure 17 is a timing chart showing an example of an image diagnosis process of the image diagnosis system in Example 2. DETAILED DESCRIPTION

[0033] Embodiments of the present application will be described below with reference to the accompanying drawings. The present embodiments are merely one example for implementing the present application, and it should be noted that the present embodiments do not limit the technical scope of the present application. The same reference numerals are assigned to common structures in the respective drawings.

[0034] Example 1

[0035] Figure 1 is an explanatory diagram showing a structure example of an image diagnosis system in Example 1. The image diagnosis system includes a management server 100, a diagnosis server 201, a diagnosis server 202, and a diagnosis server 203. In addition, the image diagnosis system includes an in-hospital server 300, a terminal 400, and a photographing apparatus 500 provided in, for example, a hospital, a clinic, or a medical center. The in-hospital server 300, the terminal 400, and the photographing apparatus 500 are connected via a network, respectively.

[0036] The photographing apparatus 500 is an ophthalmic apparatus that photographs a fundus, and examples thereof include a fundus camera, a scanning laser ophthalmoscope, an optical coherence tomography, and the like. The photographing apparatus 500 is connected to the terminal 400. The photographing apparatus 500 photographs an eye to be examined of a patient, and generates fundus image data of the right eye and the left eye of the eye to be examined. The generated fundus image data is transmitted to the terminal 400.

[0037] In addition, the fundus image data can be any one of fundus image data photographed by a fundus camera, fundus image data based on a fundus photographed by a scanning laser ophthalmoscope, and tomographic data based on a fundus photographed by an optical coherence tomography. Alternatively, it can be a combination of two or more of them, that is, a fundus image data set. The fundus image data is an example of the eye to be examined image data.

[0038] The terminal 400, which is an example of an image acquisition apparatus, is a PC (Personal Computer) or a tablet or the like used by a doctor or an operator of an ophthalmic apparatus, and the like. The terminal 400 is connected to the in-hospital server 300. The terminal 400 transmits data, which is an example of the first transmission data, including the fundus image data and the additional information, to the in-hospital server 300.

[0039] In addition, the additional information includes any one or a combination of the following information: apparatus information related to the performance and the specifications of the photographing apparatus 500; facility information including a medical department (ophthalmology, internal medicine, or diabetes internal medicine, and the like) of a hospital or a clinic using the terminal 400, a fee of a diagnosis item, and a name of a doctor, and the like; and diagnosis category information including a diagnosis mode and a disease name of a diagnosis target, and the like. The attribute information of the image including a field angle, a modality, and a resolution of a photographed image (an eye to be examined image) of the photographing apparatus 500, a model of the photographing apparatus 500, a terminal ID, and the like is an example of the apparatus information. The modality is information indicating a kind of the photographing apparatus 500 (for example, a fundus camera, a scanning laser ophthalmoscope, an optical coherence tomography, and the like) or a kind of an image as a medical image photographed by the photographing apparatus 500 (for example, a fundus image or an angiogram image photographed by red laser light or near-infrared laser light, and the like). In addition, a name of a doctor using the terminal 400, a name of a hospital, a place where the terminal is installed (information of a medical department of ophthalmology, internal medicine, or diabetes internal medicine, and the like, information related to a facility such as an optical shop or a health examination center, and the like) are examples of the facility information.

[0040] The in-hospital server 300, which is an example of an image acquisition apparatus, has a patient information DB (DataBase) 310 that holds patient information, and saves patient information received from the terminal 400 in the patient information DB 310. The in-hospital server 300 is connected to the management server 100 via a network. The in-hospital server 300 transmits, to the management server 100, diagnosis target data, which is an example of first transmission data, that contains patient information received from the terminal 400, fundus image data, and additional information. In addition, part or all of the patient information and part or all of the additional information in the diagnosis target data can also be generated by the in-hospital server 300.

[0041] The management server 100, which is an example of a first information processing apparatus, generates anonymized diagnosis target data, which is an example of second transmission data, that is diagnosis target data in which part of the information (e.g., patient information) received from the in-hospital server 300 has been anonymized. The management server 100 is connected to the diagnosis servers 201, 202, and 203 via a network. The management server 100 selects one of the diagnosis servers 201, 202, and 203 that performs image diagnosis of the fundus image data contained in the anonymized diagnosis target data, based on the additional information, and transmits the anonymized diagnosis target data to the selected diagnosis server.

[0042] The diagnosis servers 201 to 203, each of which is an example of an image diagnosis apparatus, each have an AI (Artificial Intelligence) that performs image diagnosis with respect to fundus image data. The functions (algorithms) of the AIs 1, 2, and 3 mounted on the diagnosis servers 201 to 203 are each different (details of which will be described later). The diagnosis server that receives the anonymized diagnosis target data performs image diagnosis with respect to the fundus image data contained in the anonymized diagnosis target data, using the AI mounted thereon. The diagnosis result is encrypted and transmitted to the in-hospital server 300 and the terminal 400 via the management server 100.

[0043] Hereinafter, one example of each of the diagnosis servers 201 to 203 will be described. Here, the fundus image that is the object of each AI and the disease name that is diagnosed are one example, and various combinations of fundus images and disease names that are diagnosed can be made.

[0044] The diagnostic server 201, which is an example of an image diagnostic apparatus, is a diagnostic server that takes as an object a fundus image captured by the imaging device 500 of a narrow field of view (30 to 100 (less than) degrees of field of view angle with the center of the eyeball as the origin) that is an example of a first field of view angle, and is equipped with an AI 221 that performs diagnosis of diabetic retinopathy. In a case where the device information in the additional information is information indicating a narrow field of view, the management server 100 transmits the anonymized diagnosis target data to the diagnostic server 201.

[0045] The diagnostic server 202 is a diagnostic server that takes as an object a fundus image captured by the imaging device 500 of a wide field of view (100 degrees or more to less than 200 degrees of field of view angle with the center of the eyeball as the origin), or a super wide field of view (200 degrees or more of field of view angle with the center of the eyeball as the origin) that are both examples of a second field of view angle, and is equipped with an AI 222 that performs diagnosis of diabetic retinopathy. In a case where the device information in the additional information is information indicating a wide field of view, and the diagnosis mode is information indicating diabetic retinopathy, the management server 100 transmits the anonymized diagnosis target data to the diagnostic server 202.

[0046] Also, the diagnostic server 203 is a diagnostic server that takes as an object a fundus image captured by the imaging device 500 of a super wide field of view (200 degrees or more of field of view angle with the center of the eyeball as the origin), and is equipped with an AI 223 that can diagnose not only diabetic retinopathy but also various fundus diseases. In a case where the device information of the additional information indicates a super wide field of view, and the facility information indicates an ophthalmologist, the management server 100 transmits the anonymized diagnosis target data to the diagnostic server 203.

[0047] Figure 2 is a block diagram indicating an example of a hardware structure of a computer that respectively constitutes the management server 100, the diagnostic server, the in-hospital server 300, and the terminal 400. The computer 600 has, for example, a processor (CPU) 601, a storage device 602, an input device 603, an output device 604, and a communication I / F (InterFace, interface) 605, which are connected to each other through an internal signal line 606.

[0048] The processor 601 executes a program stored in the storage device 602. The storage device 602 includes a memory. The memory includes a ROM that is a non-volatile storage element, and a RAM that is a volatile storage element. The ROM stores a program (for example, BIOS) that does not change, and the like. The RAM is a high-speed and volatile storage element like a DRAM (Dynamic Random Access Memory), and temporarily stores a program executed by the processor 601 and data used at the time of execution of the program.

[0049] In addition, the storage device 602 includes a secondary storage device. The secondary storage device is, for example, a large-capacity and nonvolatile storage device such as a magnetic storage device (HDD), a flash memory (SSD), and the like, which saves programs executed by the processor 601 and data used at the time of execution of the programs. That is, the programs are read out from the secondary storage device, loaded to the memory, and executed by the processor 601.

[0050] The input device 603 is an interface that accepts input from an operator, such as a keyboard, a mouse, and the like. The output device 604 is a device that outputs the result of execution of the programs in a form that can be visually confirmed by the operator, such as a display or a printer. Furthermore, the input device 603 and the output device 604 can be integrated, like a touch panel device. The communication I / F 605 is a network interface device that controls communication with other devices in compliance with a prescribed protocol.

[0051] The programs executed by the processor 601 are provided to the computer 600 via a removable medium (CD-ROM, flash memory, or the like) or a network, and saved in the nonvolatile secondary storage device as a non-transitory storage medium. Therefore, the computer 600 can have an interface that reads in data from the removable medium.

[0052] Furthermore, the management server 100, the diagnosis server, the in-hospital server 300, and the terminal 400 are each a computer system configured on one computer 600, or a plurality of computers 600 configured logically or physically, and can operate in different threads on the same computer 600, or on a virtual computer constructed on a plurality of physical computer resources.

[0053] Figure 3 is a block diagram that shows an example of a functional configuration of the management server 100. The management server 100 includes an anonymization processing section 101, an AI selection section 102, a display screen generation section 103, and a diagnosis result data generation section 104. The anonymization processing section 101 anonymizes patient information included in diagnosis target data transmitted from the in-hospital server 300. The AI selection section 102 selects an AI that performs image diagnosis on fundus image data included in the diagnosis target data, on the basis of additional information included in the diagnosis target data.

[0054] The display screen generation section 103 generates screen information displayed on the output device 604. The diagnosis result data generation section 104 decrypts the encrypted diagnosis result received from the diagnosis server, and generates a display screen (Fig. 13) that shows the diagnosis result, and transmits information of the display screen to the in-hospital server 300. Figure 12 Figure 13 Figure 14

[0055] ​​​Further, the functional units included in the management server 100 are realized by the processor 601 of the computer 600 that realizes the management server 100. Specifically, the processor 601 functions as the anonymization processing unit 101 by operating in accordance with the anonymization processing program loaded in the memory included in the storage device 602, and functions as the AI selection unit 102 by operating in accordance with the AI selection program loaded in the memory included in the storage device 602. The same applies to other functional units included in the management server 100 and functional units included in other apparatuses, which are realized by the processor 601 operating in accordance with the programs loaded in the memory.

[0056] The management server 100 holds the AI selection information 110. The AI selection information 110 holds correspondence information of the additional information and the diagnostic server 201, the diagnostic server 202, and the diagnostic server 203. In addition, as will be described later, since the AI 221, the AI 222, and the AI 223 each include different image diagnosis models, it can also be said that the AI selection information 110 includes correspondence information of the additional information and the image diagnosis model. In the AI selection information 110, the AI corresponding to the additional information is determined by describing a conditional branch based on the value of one or more kinds of additional information. In addition, in the AI selection information 110, the correspondence of the value (or the range of the value) of one or more kinds of additional information and the AI 220 can also be described in table form.

[0057] The AI selection information 110 is saved in the auxiliary storage device included in the storage device 602 of the computer 600 that realizes the management server 100. Further, the same applies to the information held by other apparatuses and the DB, which are saved in the auxiliary storage device included in the storage device 602 of the computer 600 that realizes the other apparatuses.

[0058] Further, in the present embodiment, the information used by each apparatus included in the image diagnosis system is not dependent on the data structure, and can be expressed in any data structure. For example, a data structure body appropriately selected from a table, a list, a database, or a queue can save the information. Various kinds of information are stored and held in a nonvolatile memory or the like.

[0059] Figure 4 is a block diagram that shows an example of the functional structure of the diagnostic server 201. The functional structures of the diagnostic server 201, the diagnostic server 202, and the diagnostic server 203 are the same except for the AI function, and the display screen generation unit, the management unit, and the like become the same function. Therefore, the functional structure of the diagnostic server 201 will be described.

[0060] The diagnosis server 201 includes, for example, an image diagnosis section 2011, a learning information management section 2021, a diagnosis image generation section 2031, and a management section 2041. The diagnosis server 201 holds a learning DB 2101 and an image diagnosis model 2111. The learning DB 2101 is a DB for constructing the image diagnosis model 2111. The image diagnosis model 2111 is a model that outputs a diagnosis result if image data is input, and is a model that takes, as an object, a fundus image captured by the imaging device 500 of a narrow field angle (30 to 100 (less than) degrees of field angle with the center of the eyeball as a starting point), and outputs, as a diagnosis result, a level of a symptom of diabetic retinopathy. In the present embodiment, the level of the symptom of diabetic retinopathy is set to five stages of the international classification of diseases.

[0061] The AI 221 is realized by the image diagnosis section 2011, the learning information management section 2021, the learning DB 2101, and the image diagnosis model 2111. The image diagnosis section 2011 performs image diagnosis using the image diagnosis model 2111 with respect to the fundus image data included in the anonymized diagnosis target data received from the management server 100.

[0062] The learning information management section 2021 saves the examination eye image data and the image diagnosis result included in the anonymized diagnosis target data in the learning DB 210 as learning data for the AI, and updates the learning DB 210. The learning information management section 2021 updates (for example, optimizes) the image diagnosis model 2111 by learning based on the updated learning DB 2101.

[0063] The diagnosis image generation section 2031 generates a diagnosis completed fundus image obtained by superimposing a marker showing a lesion position and / or a character or the like of a disease name on a fundus image after diagnosis. The management section 2041 manages the AI 221. The diagnosis completed fundus image is transmitted to the management server 100 together with the diagnosis result.

[0064] Further, the diagnosis server 201 can not have a learning function of the image diagnosis model 2111. That is, the diagnosis server 201 can continuously perform image diagnosis in a state where a predetermined image diagnosis model 2111 is fixed without updating the image diagnosis model 2111. In this case, the diagnosis server 201 can not have the learning information management section 2021 and the learning DB 2101.

[0065] The diagnosis server 202 and the diagnosis server 203 have an image diagnosis model different from that of the diagnosis server 201, and have the same structure as the diagnosis server 201 except for this aspect.

[0066] The image diagnosis model held by the diagnosis server 202 is a model that outputs a diagnosis result if image data is input, and is a model that takes as an object an ocular fundus image captured by the wide-angle camera 500 (field of view angle greater than or equal to 100 degrees to less than 200 degrees with the center of the eyeball as the starting point) or the super-wide-angle camera 500 (field of view angle greater than or equal to 200 degrees with the center of the eyeball as the starting point), and outputs a symptom level of diabetic retinopathy as a diagnosis result.

[0067] The image diagnosis model held by the diagnosis server 203 is a model that outputs a diagnosis result if image data is input, and is a model that takes as an object an ocular fundus image captured by the super-wide-angle camera 500 (field of view angle greater than or equal to 200 degrees with the center of the eyeball as the starting point), and outputs a diagnosis result not only of diabetic retinopathy but also of various ocular fundus diseases.

[0068] Further, the image diagnosis unit, the learning information management unit, and the learning DB of the diagnosis server 202 and the diagnosis server 203 are adapted to the image diagnosis model held by themselves.

[0069] Figure 5 is a block diagram showing an example of a functional structure of the in-hospital server 300. The in-hospital server 300 includes, for example, an anonymization processing unit 301, a patient information management unit 302, and a display screen generation unit 303. In addition, the in-hospital server 300 holds a patient information DB 310.

[0070] The anonymization processing unit 301 anonymizes the patient information included in the diagnosis target data. The patient information management unit 302 saves the patient information included in the diagnosis target data in the patient information DB 310, acquires the patient information from the patient information DB 310, and attaches it to the diagnosis target data. The display screen generation unit 303 generates screen information displayed on the output device 604. The patient information DB 310 holds information of patients.

[0071] Figure 6 is a block diagram showing an example of a functional structure of the terminal 400. The terminal 400 includes a diagnosis target data generation unit 401, an additional information acquisition unit 402, and a display screen generation unit 403. The diagnosis target data generation unit 401 generates diagnosis target data including patient information, additional information, and examination eye image data. The additional information acquisition unit 402 acquires additional information for selecting an AI (or a diagnosis server having an AI suitable for diagnosis). The display screen generation unit 403 generates screen information displayed on the output device 604.

[0072] Figure 7 is a timing chart showing the image diagnosis processing of the image diagnosis system of Embodiment 1. In Figure 7In the example, based on the device information of the photographing device 500 or the like, a diagnosis server that performs image diagnosis of the fundus image is selected.

[0073] First, the diagnosis target data generating section 401 of the terminal 400 receives input of patient information via the input device 603 (S701). The ID of the patient and the patient's age, sex, address, medical history, medication history, and the results of an interview are examples of the patient information. Further, with respect to a patient whose information has been entered in the patient information DB 310, the input of the other patient information can be omitted as long as the input of the ID of the patient is received.

[0074] The diagnosis target data generating section 401 acquires the fundus image data of both eyes of the patient transmitted from the photographing device 500 (S702). Further, in the present embodiment, the fundus image data of both eyes can be acquired, or the fundus image data of only the left eye or the right eye can be acquired. The diagnosis target data generating section 401 generates a left-right eye flag indicating whether the fundus image data is of both eyes, of only the right eye, or of only the left eye. The diagnosis target data generating section 401 can also acquire the fundus image data from a device other than the photographing device 500.

[0075] Next, the additional information acquiring section 402 acquires additional information (S703). Specifically, for example, the additional information acquiring section 402 acquires device information from the photographing device 500 as the additional information, or receives input of the medical department of the hospital or the doctor using the terminal 400 or the like.

[0076] Further, these pieces of additional information can be previously stored in the storage device 602 of the terminal 400. Alternatively, for example, the photographing device 500 can embed the device information as metadata in the fundus image data, and the additional information acquiring section 402 acquires the device information from the fundus image data.

[0077] Next, the diagnosis target data generating section 401 transmits the diagnosis target data including the patient information, the fundus image data, the left-right eye flag, and the additional information to the management server 100 via the in-hospital server 300 (S704). Further, the patient information managing section 302 of the in-hospital server 300 saves the patient information received from the terminal 400 in the patient information DB 310.

[0078] Further, in a case where the patient information received from the terminal 400 is incomplete, the patient information managing section 302 can acquire the patient information from the patient information DB 310 to supplement it. Specifically, for example, in a case where the patient information received from the terminal 400 is only the ID of the patient, the patient information managing section 302 acquires the patient information corresponding to the ID from the patient information DB 310, and transmits the diagnosis target data including the acquired patient information to the management server 100.

[0079] Next, the anonymization processing section 101 of the management server 100 performs anonymization processing based on a prescribed algorithm with respect to the patient information included in the received diagnosis target data (S705). As the anonymization processing, it is processing to anonymize the patient ID (replace it with an ID inherent to the fundus image data), or delete the patient's personal information such as the name, disease name, and the like. Further, the anonymization processing section 101 can also anonymize only a part of the patient information (for example, only the sensitive information related to privacy). In addition, the anonymization processing with respect to the patient information can also be performed in advance by the anonymization processing section 301 of the in-hospital server 300, for example, before the diagnosis target data is transmitted to the management server 100.

[0080] The AI selection section 102 of the management server 100 selects at least one from among the diagnosis server 201, the diagnosis server 202, and the diagnosis server 203 based on the AI selection information 110 and the additional information included in the received diagnosis target data, and transmits the anonymized diagnosis target data (refer to the anonymized diagnosis target data described later) including the anonymized patient information, the fundus image data, the left-right eye flag, and the additional information to the selected diagnosis server (in this case, it is assumed that the diagnosis server 201 is selected) (S706). Details of step S706 will be described later. Figure 11

[0081] Further, the AI selection section 102 can also transmit the anonymized diagnosis target data on the basis of being encrypted using an encryption key to the selected diagnosis server 201, in which case the diagnosis server 201 has a decryption key corresponding to the encryption key, and first decrypts the anonymized diagnosis target data using the decryption key in step S707 described later.

[0082] Next, the image diagnosis section 2011 of the diagnosis server 201 that received the anonymized diagnosis target data performs image diagnosis with respect to the fundus image data included in the received anonymized diagnosis target data using the image diagnosis model 2111 that performs diagnosis of diabetic retinopathy with the narrow-angle fundus image as the target (S707).

[0083] The learning information management section 2021 of the diagnosis server 201 updates the learning DB 2101 by saving the fundus image data as learning data in the learning DB 2101, and updates the image diagnosis model 2111 based on the updated learning DB 2101 (S708). Further, the learning information management section 2021 can also save the anonymized patient information and the additional information together as learning data in the learning DB 2101.

[0084] ​Next, the management unit 2041 generates image diagnostic result data that includes at least anonymized patient information and image diagnostic results. Using the encryption key held by the diagnostic server 201, the image diagnostic results are encrypted, and an encrypted image diagnostic result is generated and sent to the management server 100 (S709). The image diagnostic result data may also include a diagnosed fundus image obtained by overlaying markers indicating the location of the lesion and / or text indicating the disease name onto the diagnosed fundus image.

[0085] Next, the diagnostic result data generation unit 104 of the management server 100 decrypts the received encrypted image diagnostic result using the decryption key held by the management server 100 (S710). Furthermore, the diagnostic result data generation unit 104 restores the anonymized patient information. It then establishes an association between the decrypted image diagnostic result and the patient information of the patient before anonymization.

[0086] Next, the diagnostic result data generation unit 104 generates a display screen showing the diagnostic results, including the grade of diabetic retinopathy as a diagnostic result. Figure 12 ), and after assigning a patient ID, save it in a memory not shown.

[0087] When the management server 100 receives the encrypted image diagnostic result from the diagnostic server 202, the diagnostic result data generation unit 104 generates a display screen showing the diagnostic result. Figure 13 When the management server 100 receives the encrypted image diagnostic result from the diagnostic server 203, the diagnostic result data generation unit 104 generates a display screen showing the diagnostic result. Figure 14 ).

[0088] Next, the diagnosis result data generation unit 104 associates the display screen representing the diagnosis result with the patient information and sends it to the hospital server 300 (S711). The display screen generation unit 303 of the hospital server 300 displays the display screen representing the received diagnosis result and the patient information on the output device 604 of the hospital server 300 (S712). Alternatively, the terminal 400 may obtain the image diagnosis result and the patient information from the hospital server 300, and the display screen generation unit 403 of the terminal 400 may display the display screen representing the diagnosis result and the patient information on the output device 604 of the terminal 400.

[0089] Alternatively, the display screen generation unit 103 of the management server 100 may generate display screen information based on the image diagnosis results and patient information and send the information to the hospital server 300. The hospital server 300 and the terminal 400 shall display the display screen in accordance with the generated information.

[0090] Figure 8 is a flowchart showing one example of the diagnosis server decision processing of step S706. In Figure 8 In the example, the diagnosis server that transmits the anonymized diagnosis target data is decided on the basis of the field of view angle information included in the additional information. First, the AI selection section 102 of the management server 100 acquires the additional information including the field of view angle information from the diagnosis target data (S801).

[0091] The AI selection section 102 determines whether the field of view angle indicated by the field of view angle information is a wide field of view angle (S802). Specifically, in the case where the field of view angle information indicates a specific angle, if the angle is equal to or greater than a prescribed value (for example, 100 degrees), it is determined to be a wide field of view angle, and if it is less than the prescribed value, it is determined not to be a wide field of view angle (to be a narrow field of view angle).

[0092] In addition, for example, the management server 100 can also hold device information of each photographing device 500. Specifically, the device information is a lookup table defined by associating the model, terminal ID, and field of view angle, resolution, and the like of each photographing device 500. In this case, the AI selection section 102 can also acquire the model or terminal ID of the photographing device 500 from the additional information, and refer to the device information to determine whether the field of view angle of the photographing device 500 indicated by the model or terminal ID is a wide field of view angle.

[0093] The AI selection section 102 transmits the anonymized diagnosis target data to the diagnosis server 202 in the case where it is determined that the field of view angle indicated by the field of view angle information is a wide field of view angle (S802: Yes), and transmits the anonymized diagnosis target data to the diagnosis server 201 in the case where it is determined not to be a wide field of view angle (S802: No), and ends the processing of step S705.

[0094] In Figure 8 , the AI mounted on the diagnosis server 202 is able to perform image diagnosis on the fundus image photographed with a wide field of view angle with high accuracy, and the AI mounted on the diagnosis server 201 is able to perform image diagnosis on the examination eye image photographed with a narrow field of view angle with high accuracy. The AI selection section 102 is thereby able to select an appropriate diagnosis server in accordance with the photographing field of view angle of the examination eye image.

[0095] Further, in Figure 8 , the AI selection section 102 determines whether the field of view angle indicated by the field of view angle information is a wide field of view angle, and selects the diagnosis server that is the transmission destination from the two diagnosis servers 201 and 202. In addition, the AI selection section 102 can also determine which of the intervals of three or more field of view angles the field of view angle indicated by the field of view angle information corresponds to, and refer to the AI selection information 110 to select the diagnosis server corresponding to the interval.

[0096] In addition, in Figure 8 In this case, for example, if the medical treatment department information is ophthalmology, the AI selection unit 102 selects the diagnostic server 203, and if the medical treatment department information is internal medicine, the AI selection unit 102 selects the diagnostic server 201. In addition, for example, if the fee for the diagnostic project is equal to or higher than a predetermined value, the AI selection unit 102 selects the diagnostic server 202 or the diagnostic server 203, and if the fee for the diagnostic project is lower than the predetermined value, the AI selection unit 102 selects the diagnostic server 201.

[0097] In this case, for example, if the medical treatment department information is ophthalmology, the AI selection unit 102 selects the diagnostic server 203, and if the medical treatment department information is internal medicine, the AI selection unit 102 selects the diagnostic server 201. In addition, for example, if the fee for the diagnostic project is equal to or higher than a predetermined value, the AI selection unit 102 selects the diagnostic server 202 or the diagnostic server 203, and if the fee for the diagnostic project is lower than the predetermined value, the AI selection unit 102 selects the diagnostic server 201.

[0098] In addition, the diagnostic server 201 that mounts an AI that diagnoses a specific disease (for example, diabetic retinopathy) with high precision and the diagnostic server 203 that comprehensively diagnoses the onset symptoms of a plurality of diseases are provided. It is assumed that the additional information includes information indicating the disease of the diagnosis target. In this case, for example, if the disease of the diagnosis target indicated by the additional information is the specific disease, the AI selection unit 102 selects the diagnostic server 201, and if the disease of the diagnosis target indicated by the additional information is not the specific disease name but is a comprehensive diagnosis, the AI selection unit 102 selects the diagnostic server 203.

[0099] In addition, for example, the diagnostic server 201 that mounts an AI that performs image diagnosis with high precision with respect to the fundus camera-pictured eye image and the diagnostic server 202 that mounts an AI that performs image diagnosis with high precision with respect to the scanning laser ophthalmoscope-pictured eye image are provided. It is assumed that the additional information includes information indicating the form. In this case, for example, if the form indicated by the additional information is the fundus camera, the AI selection unit 102 selects the diagnostic server 201, and if the form indicated by the additional information is the scanning laser ophthalmoscope, the AI selection unit 102 selects the diagnostic server 202.

[0100] In this way, the AI selection unit 102 can select the diagnostic server that mounts the most suitable AI from a plurality of diagnostic servers on the basis of the additional information. Thus, the user does not hesitate and / or is not bothered by the selection of the diagnostic server, and can transmit the fundus image data to the appropriate diagnostic server.

[0101] Figure 9 is a flowchart indicating another example of the diagnostic server decision processing of step S706. In Figure 9In this context, the diagnostic server that sends anonymized diagnostic data is determined based on the various information contained in the supplementary information (field of view information and departmental information). In other words, the field of view information and departmental information are assumed to be included in the supplementary information. (Explanation and...) Figure 8 The differences.

[0102] If the AI ​​selection unit 102 determines that the field of view shown in the field of view information is a wide field of view (S802: Yes), it determines whether the department shown in the department information is ophthalmology or internal medicine (S901). If the AI ​​selection unit 102 determines that the department shown in the department information is ophthalmology (S901: ophthalmology), it sends anonymized diagnostic object data to the diagnostic server 203 (S803); if it determines that the department is internal medicine (S901: internal medicine), it sends anonymized diagnostic object data to the diagnostic server 202 (S902), and the processing in step S705 ends.

[0103] exist Figure 9 The AI ​​mounted on diagnostic server 202 is designed to perform detailed diagnosis of specific diseases (such as diabetic retinopathy) with high precision based on fundus image data captured with a wide field of view, including symptom severity. Additionally, the AI ​​mounted on diagnostic server 201 is designed to perform image diagnosis with high precision based on fundus image data captured with a narrow field of view. Furthermore, the AI ​​mounted on diagnostic server 203 is designed to diagnose the symptoms of various diseases with high precision based on fundus image data captured with a wide field of view.

[0104] Therefore, the AI ​​selection unit 102 can select an appropriate diagnostic server based on the field of view of the image of the examined eye and the hospital where the terminal 400 is installed or the clinic of the doctor using the terminal 400. In other words, it can send fundus image data to the appropriate diagnostic server based on various information (field of view information and clinic information) included in the supplementary information.

[0105] In addition, although Figure 9 In the process, all fundus image data captured with a narrow field of view is sent to the diagnostic server 201. However, for the image data of the examined eye captured with a narrow field of view, a conditional branch based on additional information can be added to select the diagnostic server as the destination from multiple different diagnostic servers.

[0106] In addition, although Figure 9 The method uses two types of additional information to select a specific diagnostic server from multiple diagnostic servers, but the mapping between additional information and diagnostic servers can be arbitrarily designed based on the characteristics of the AI ​​mounted on each diagnostic server. For example, more than three types of additional information can also be used, and the diagnostic server can be determined by using arbitrary conditional branches based on multiple types of additional information.

[0107] Figure 10 is an example of the AI selection information 110. In Figure 10 In the example, the additional information and the correspondence to the diagnosis server are described in a table form. The AI selection information 110 includes, for example, a record number column 1101, an additional information column 1102, and a diagnosis server ID column 1103. The record number column 1101 holds a number that identifies a record of the AI selection information 110. The additional information column 1102 holds elements of one or more kinds of additional information (field angle information, form information, medical department information, resolution information, and the like). The diagnosis server ID column 1103 holds an ID that identifies a diagnosis server that is a transmission destination of anonymized diagnosis target data corresponding to a combination of the additional information.

[0108] For example, the AI selection section 102 acquires additional information in step S706, and transmits the anonymized diagnosis target data to the diagnosis server having the diagnosis server ID corresponding to the acquired additional information in the table. Figure 10 In addition, the AI selection section 102 can acquire the additional information in step S706, and transmit the anonymized diagnosis target data to the diagnosis server having the diagnosis server ID corresponding to the acquired additional information in the table. Figure 8 Figure 9 Such a conditional branch can also be described in a table form as in Figure 10

[0109] Figure 11 is an example of a data structure of the anonymized diagnosis target data transmitted from the management server 100 to the diagnosis server. The anonymized diagnosis target data includes, for example, a header 701, DICOM (Digital Imaging and Communications in Medicine) data 702, user data 703, and fundus image data 704.

[0110] In the header 701, information such as a transmission source and a transmission destination of data, and a category of data (data categories such as a medical image, a file, and an email). In the DICOM data 702, information such as a format of a medical image captured by the imaging device 500, and information defining a communication protocol between medical devices including the imaging device 500.

[0111] ​​User data 703 includes, for example, diagnostic category markers, left / right eye markers, anonymized patient information, and additional information. The diagnostic category marker indicates the name of the disease diagnosed by the diagnostic server selected by the AI ​​selection unit 102 (a value identifying diabetic retinopathy, age-related macular degeneration, and all fundus diseases, etc.). The left / right eye marker indicates whether the fundus image data 704 is image data for the right eye, the left eye, or both eyes (e.g., a value among L, R, or LR). Furthermore, information such as the terminal ID and format of the imaging device 500 in the additional information can also be recorded in the DICOM data 702, or it can be recorded only in the DICOM data 702.

[0112] Figure 12 This is an example of a display screen showing the diagnostic results based on the diagnostic server 201. Figure 12 This is the display screen (screen layout) used to diagnose the symptom level of diabetic retinopathy based on fundus images (images of the examined eye) captured with a narrow field of view.

[0113] The display screen includes a patient information display area 1201, an AI information display area 1202, a supplementary information display area 1203, and a diagnosis result display area 1204. The patient information display area 1201 displays, for example, patient information contained in the image diagnostic data. The AI ​​information display area 1202 displays, for example, the ID of the diagnostic server that performed the image diagnosis and the ID (or version number) of the AI ​​that performed the image diagnosis. The supplementary information display area 1203 displays, for example, part or all of the supplementary information contained in the diagnostic object data (field of view, resolution, diagnostic category, etc.).

[0114] The diagnostic results display area 1204 shows information about the diagnostic results based on the fundus image data diagnosed by the diagnostic server. Figure 12 In the example, fundus images of both eyes, rectangular bars showing the symptom levels of the five stages of diabetic retinopathy in both eyes, and observations of both eyes are displayed in the diagnostic results display area 1204.

[0115] exist Figure 12 In the example, the right eye image and a right-pointing arrow (indicator) showing the symptom level of diabetic retinopathy in the right eye are displayed on the right side of the rectangular bar, while the left eye image and a left-pointing arrow showing the symptom level of diabetic retinopathy in the left eye are displayed on the left side of the rectangular bar. Thus, a user can grasp the symptom levels of diabetic retinopathy in both eyes and the differences in symptom levels with just a glance at the diagnostic results display area 1204. Furthermore, the observations in the diagnostic results display area 1204 are generated by the AI ​​of the diagnostic server. Alternatively, the observations can be input and edited by the user of the terminal 400.

[0116] Figure 13 is an example of a display screen that displays a diagnosis result based on the diagnosis by the diagnosis server 202. Figure 13 is a display screen in a case where a symptom grade of diabetic retinopathy is diagnosed for wide-angle fundus image data taken with a wide field angle. If diabetic retinopathy occurs, an abnormality occurs in the peripheral portion of the fundus as an initial symptom, and then the abnormality spreads toward the central portion. By taking the fundus with a wide field angle, wide-angle fundus image data including not only the central portion but also the peripheral portion of the fundus can be obtained. Therefore, the AI that performs image diagnosis of the wide-angle fundus image taken with a wide field angle can not only perform estimation of the current symptom grade of diabetic retinopathy but also perform prediction of the future symptom grade taking into account the state of the peripheral portion of the fundus.

[0117] Therefore, in the diagnosis result display area 1204 in the example of Figure 13 , in addition to the rectangular bars showing the current symptom grades of the five stages of diabetic retinopathy of both eyes, rectangular bars showing the future symptom grades of diabetic retinopathy of both eyes are displayed. In addition, in the observation result in the diagnosis result display area 1204, the predicted state after one year (in Figure 13 , but it can be after three months, after six months, or the like) is described.

[0118] Further, although in the example of Figure 13 , the current symptom grade and the future symptom grade are displayed in the diagnosis result display area 1204, the current symptom grade at the central portion of the fundus and the current symptom grade at the peripheral portion can be displayed in the diagnosis result display area 1204.

[0119] Figure 14 is an example of a display screen that displays a diagnosis result based on the diagnosis by the diagnosis server 203. Figure 14 is a display screen in a case where diagnoses of a plurality of diseases are performed for wide-angle fundus image data taken with a wide field angle. As the plurality of diseases, one of them is diabetic retinopathy. This is the same as the diagnosis result of diabetic retinopathy in Figure 13 . In Figure 14 , rectangular bars and arrows (pointers) showing the category and the degree of aggravation of the symptoms of disease 2 and disease 3 are further displayed. Disease 2 and disease 3 are fundus diseases such as age-related macular degeneration and retinal detachment that can be discriminated by the AI 223 of the diagnosis server 203.

[0120] A modification example of the display screen that displays a diagnosis result will be described below.

[0121] In a case where the image diagnosis is performed although it is possible to diagnose the presence or absence of the onset symptom of the disease, but it is not possible to determine the symptom level of the disease, the information indicating the disease from which the onset symptom is presumed can be displayed in the diagnosis result display region 1204.

[0122] In this case, the message recommending the image diagnosis by the diagnosis server based on the AI capable of diagnosing the disease from which the onset symptom is presumed with the specific symptom level can be further displayed in the diagnosis result display region 1204.

[0123] In addition, the management server 100 can also hold the correspondence information of the disease and the diagnosis server equipped with the AI capable of diagnosing the symptom level of the disease. In this case, if the management server 100 receives the command to perform the image diagnosis in compliance with the message from the in-hospital server 300, the management server 100 refers to the correspondence information, determines one diagnosis server equipped with the AI capable of diagnosing the symptom level of the disease from among the diagnosis servers capable of transmission. Then, the management server 100 can transmit the information indicating the diagnosis server to the in-hospital server 300 and cause it to be displayed, or can re-transmit the anonymized diagnosis target data to the diagnosis server and entrust the image diagnosis.

[0124] In addition, it is assumed that the image diagnosis system includes a diagnosis server equipped with an AI (assumed to be AI A) for performing image diagnosis on an image with low resolution (for example, a first resolution less than a predetermined value), and a diagnosis server equipped with an AI (assumed to be AI B) for performing image diagnosis on an image with high resolution (for example, a second resolution equal to or greater than the predetermined value) (both AIs). At this time, in a case where the diagnosis result based on the AI A is diagnosed to have an abnormality in the fundus, a message recommending the image diagnosis based on the AI B using fundus image data with higher resolution can be displayed in the diagnosis result display region 1204. Alternatively, a message recommending the photographing with fundus image data with higher resolution can be displayed.

[0125] Further, the anonymized diagnosis target data can be transmitted to a plurality of diagnosis servers, that is, the image diagnosis of the fundus image data can be performed by AIs different in image diagnosis model. Figure 15 is one example of a display screen displaying the diagnosis result in a case where the image diagnosis is performed by a plurality of diagnosis servers. Figure 15 is one example of a display screen in a case where the diagnosis of the symptom level of diabetic retinopathy performed on wide-angle fundus image data photographed with a wide field of view is performed by two diagnosis servers. In addition, it is assumed that the classification and the number of classifications of the symptom level based on the diagnosis results diagnosed by the two diagnosis servers are different.

[0126] In this case, for example, as Figure 15The diagnosis result display area 1204 displays the first diagnosis result and the second diagnosis result, the first diagnosis result being diagnosed by the diagnosis server equipped with the AI that performs image diagnosis of diabetic retinopathy based on classification 1 (international disease classification classified into five stages of grades), and the second diagnosis result being diagnosed by the diagnosis server equipped with the AI that performs image diagnosis of diabetic retinopathy based on classification 2 (Davis classification (modified) classified into three stages of grades). That is, the grades of symptoms of diabetic retinopathy based on the two different classifications are displayed.

[0127] In addition, the display screen generation section 103 of the management server 100, for example, can generate the grade of symptoms taking into account the grades of symptoms based on different classifications, and display it as a comprehensive result.

[0128] Specifically, the scores corresponding to the grades of symptoms of each classification are determined in advance (for example, No DR is 1 point, Mild is 3 points, …, in classification 1, Al is 1 point, A2 is 7 points, …, in classification 2), and the display screen generation section 103 calculates the average of the scores corresponding to the grades of symptoms with respect to the left and right eyes.

[0129] In addition, in the comprehensive result, the grade of symptoms corresponding to each interval of the average of the scores is determined in advance (if greater than or equal to 1 point and less than 2.5 points, the grade of symptoms is 1, if greater than or equal to 2.5 points and less than 5 points, the grade of symptoms is 2, …). The display screen generation section 103 determines the grade of symptoms corresponding to the calculated average with respect to the left and right eyes, and displays the determined grade of symptoms as a comprehensive result to the diagnosis result display area 1204. Thus, it is possible to present a new index by integrating the diagnosis results of the AI that diagnoses diabetic retinopathy according to a plurality of different classifications. In addition, classification 2 is not limited to Davis classification (modified), and can be Fukuda classification or the like.

[0130] Further, in the present embodiment, the diagnosis server that performs image diagnosis of the fundus image data included in the anonymized diagnosis target data is determined by the management server 100, but the diagnosis server can be determined by another device (for example, the in-hospital server 300, the terminal 400, or the photographing apparatus 500). In this case, the other device holds the AI selection information 110. In addition, the other device can include information (for example, a logo) indicating the determined diagnosis server in the user data 703.

[0131] Embodiment 2

[0132] The diagnosis server 900 in the image diagnosis system of Embodiment 2 is equipped with a plurality of AIs 901 to 902. Differences from Embodiment 1 will be described.

[0133] Figure 16is an explanatory diagram showing a configuration example of the image diagnosis system of Embodiment 2. In terms of the image diagnosis system having a plurality of AIs mounted on the diagnosis server 900, it is different from the image diagnosis system of Figure 1 . Further, in order to simplify the explanation, the image diagnosis system has one diagnosis server 900 in Figure 16 , but it can be an image diagnosis system having a plurality of diagnosis servers, and some or all of the plurality of diagnosis servers can have a plurality of AIs mounted thereon. The plurality of AIs (two AIs including the AI 901 and the AI 902, or three or more AIs) each have a different image diagnosis model.

[0134] The AI selection information 110 of Embodiment 2 includes information for determining an AI using the additional information transmitted from the in-hospital server 300 or the terminal 400. That is, for example, in Figure 10 , the AI selection information 110 can further include an ID column of the AI. The ID of the AI is an ID for identifying an AI that performs image diagnosis of the anonymized diagnosis target data corresponding to the value of the additional information.

[0135] Figure 17 is a time chart showing the image diagnosis processing of the image diagnosis system in Embodiment 2. Differences from Figure 7 will be described. Instead of the step S702, the step S1601 is executed. In the step S1601, the AI selection section 102 of the management server 100 selects at least one from the plurality of AIs (two AIs including the AI 901 and the AI 902, or three or more AIs) based on the AI selection information 110 and the additional information included in the received diagnosis target data.

[0136] Further, in the step S1601, the AI selection section 102 causes AI information (for example, an ID of the AI, or the like) indicating the selected AI 220 to be included in the user data 703, and transmits the anonymized diagnosis target data including the user data 703 to the diagnosis server on which the selected AI 220 is mounted.

[0137] Next, the management section of the diagnosis server that received the anonymized diagnosis target data selects the AI 220 indicated by the AI information included in the anonymized diagnosis target data, and inputs the fundus image data included in the anonymized diagnosis target data to the selected AI 220 (S1602). Next, the processing shifts to the step S707.

[0138] Further, although the AI that performs the image diagnosis of the fundus image data included in the anonymized diagnosis target data is decided by the management server 100 in Embodiment 2, the AI can be decided by another device (for example, the diagnosis server 900, the in-hospital server 300, the terminal 400, or the photographing apparatus 500). In this case, the other device holds the AI selection information 110. In addition, the other device includes information (for example, a flag) indicating the decided AI in the user data 703.

[0139] However, in a case where the diagnosis server 900 decides the AI, since the management server 100 cannot determine which diagnosis server 900 holds the appropriate AI, it is desirable to transmit the anonymized diagnosis target data to all the diagnosis servers. Further, the diagnosis server 900 that receives the anonymized diagnosis target data refers to the AI selection information 110, and determines whether or not the image diagnosis of the fundus image data can be performed by the AI mounted on itself. Further, the diagnosis server transmits the diagnosis result based on the AI diagnosis to the management server 100 in a case where the AI that can perform the image diagnosis is mounted.

[0140] Further, the present application is not limited to the above, and they can be combined arbitrarily. In addition, other modes that can be conceived within the scope of the technical idea of the present application are also included in the scope of the present application.

[0141] Explanation of Reference Numerals

[0142] 100 management server, 101 anonymization processing section, 102 AI selection section, 103 display screen generation section, 104 diagnosis result data generation section, 110 AI selection information, 201 diagnosis server, 2011 image diagnosis section, 300 in-hospital server, 301 anonymization processing section, 302 patient information management section, 303 display screen generation section, 310 patient information DB, 400 terminal, 401 diagnosis target data generation section, 402 additional information acquisition section, 403 display screen generation section, 600 computer, 601 processor, 602 storage device, 603 input device, 604 output device, 605 communication I / F, 900 diagnosis server, 2021 learning information management section, 2031 diagnosis image generation section, 2041 management section, 2101 learning DB, 2111 image diagnosis model.

Claims

1. An information processing system, comprising: Image acquisition device for acquiring image data of the patient's examined eye; and A first information processing device capable of communicating with the image acquisition device and storing the image data of the examined eye, wherein, The image acquisition device performs the first transmission process. The first transmission process sends first transmission data, including the image data of the examined eye and additional information for determining the image diagnostic apparatus for performing image diagnosis on the image data of the examined eye, to the first information processing apparatus. The first information processing device performs the following processing: Storage processing: If the first transmitted data is received from the image acquisition device, the image data of the examined eye is stored. The process involves determining at least one of a first image diagnostic device and a second image diagnostic device based on the additional information, wherein the first image diagnostic device performs a first image diagnosis on the examined eye image data, and the second image diagnostic device performs a second image diagnosis on the examined eye image data that differs from the first image diagnosis; and The second transmission process involves sending second transmission data, containing the image data of the examined eye, to the identified image diagnostic device. The additional information is determined based on the attribute information of the examined eye image data. The attribute information includes information representing the resolution of the examined eye image data. In the first image diagnosis, image diagnosis is performed on the image data of the examined eye captured at a first resolution. In the second image diagnosis, image diagnosis is performed on the examined eye image data captured at a second resolution, which is higher than the first resolution. If the first image diagnostic device detects an abnormality in the examined eye image data during the first image diagnosis, it will include information indicating a recommendation to acquire image data of the examined eye taken at the second resolution in the diagnostic result of the image diagnosis. The first image diagnostic device sends the diagnostic results to the first information processing device.

2. The information processing system as described in claim 1, wherein, The number of image diagnostic devices, including the first image diagnostic device and the second image diagnostic device, is three or more. Three or more of the aforementioned image diagnostic devices can perform different image diagnostics on the image data of the examined eye. The first information processing device determines at least one of the three or more image diagnostic devices in the determination process.

3. The information processing system as described in claim 1, wherein, The image acquisition device performs the following processing: The process involves acquiring the attribute information; and The decision is made to process the information based on the acquired attribute information, determining the flag of at least one of the first image diagnostic device and the second image diagnostic device as the additional information. In the first transmission process, the flag is sent to the first information processing device.

4. The information processing system as described in claim 1, wherein, The image acquisition device performs an acquisition process that uses the attribute information of the patient's examined eye image data as additional information. The first information processing device performs the determination process based on the received attribute information.

5. The information processing system as described in claim 1, wherein, The attribute information includes field-of-view information, which indicates the field of view of the image captured when the image data of the examined eye was taken. In the first image diagnosis, image diagnosis is performed on the image data of the examined eye captured at the first field of view. In the second image diagnosis, image diagnosis is performed on the image data of the examined eye taken at a second field of view that is wider than the first field of view.

6. The information processing system as described in claim 5, wherein, If the first image diagnostic device detects an abnormality in the examined eye image data during the first image diagnosis, it will include information indicating a recommendation to acquire image data of the examined eye taken at the second field of view in the diagnostic result of the image diagnosis. The first image diagnostic device sends the diagnostic results to the first information processing device.

7. The information processing system as described in claim 1, wherein, The examined eye image data includes at least one of fundus image data obtained from a fundus camera, fundus image data obtained from a scanning laser ophthalmoscope, and tomographic data obtained from an optical coherence tomography (OCT) system.

8. The information processing system as described in claim 7, wherein, The examined eye image data includes fundus image data. In the first image diagnosis and the second image diagnosis, the diagnosis of fundus lesions is performed.

9. The information processing system as described in claim 8, wherein, In the first image diagnosis and the second image diagnosis, a diagnosis of diabetic retinopathy was performed using fundus images.

10. The information processing system as described in claim 8, wherein, In the first image diagnosis, the diagnostic results are presented according to the first classification indicating the state of fundus lesions. In the second image diagnosis, the diagnostic results are displayed based on the second category, which has a different number of categories than the first category. The first image diagnostic device sends the first diagnostic result based on the first classification from the first image diagnosis to the first information processing device. The second image diagnostic device sends the second diagnostic result based on the second classification from the second image diagnostic to the first information processing device. The first information processing device sends the first diagnostic result and the second diagnostic result to the image acquisition device. The image acquisition device displays the combined diagnostic result after integrating the first diagnostic result and the second diagnostic result.

11. The information processing system as described in claim 1, wherein, The additional information identifies both the first image diagnostic device and the second image diagnostic device. The additional information includes information to determine the image diagnosis. The first information processing device sends the examined eye image data and the additional information to the first image diagnostic device and the second image diagnostic device. When the first image diagnostic device and the second image diagnostic device are capable of performing the image diagnostics indicated by the additional information, they perform the image diagnostics indicated by the additional information on the received image data of the examined eye.

12. An information processing system, comprising: An image acquisition device for acquiring image data of the patient's examined eye; and A first information processing device capable of communicating with the image acquisition device and storing the image data of the examined eye, wherein, The image acquisition device performs the first transmission process. The first transmission process sends first transmission data, including the examined eye image data and additional information for determining artificial intelligence for image diagnosis of the examined eye image data, to the first information processing device. The first information processing device performs the following processing: Storage processing: If the first transmitted data is received from the image acquisition device, the image data of the examined eye is stored. The process is determined based on the additional information, at least one of a first artificial intelligence and a second artificial intelligence, wherein the first artificial intelligence is used to perform a first image diagnosis of the examined eye image data, and the second artificial intelligence performs a second image diagnosis of the examined eye image data that is different from the first image diagnosis; as well as The second transmission process involves sending second transmission data, containing the image data of the examined eye and the determination information for the determined artificial intelligence, to an image diagnostic device containing the determined artificial intelligence. The additional information is determined based on the attribute information of the examined eye image data. The attribute information includes information representing the resolution of the examined eye image data. In the first image diagnosis, image diagnosis is performed on the image data of the examined eye captured at a first resolution. In the second image diagnosis, image diagnosis is performed on the examined eye image data captured at a second resolution, which is higher than the first resolution. If the first image diagnostic device detects an abnormality in the examined eye image data during the first image diagnosis, it will include information indicating a recommendation to acquire image data of the examined eye taken at the second resolution in the diagnostic result of the image diagnosis. The first image diagnostic device sends the diagnostic results to the first information processing device.

13. The information processing system as described in claim 12, wherein, The image diagnostic device incorporates more than three types of artificial intelligence. The three or more types of artificial intelligence described above can perform different image diagnoses on the image data of the examined eye. The first information processing device determines at least one of three or more types of artificial intelligence in the determination process.

14. The information processing system as described in claim 13, wherein, The image acquisition device performs the following processing: The process involves acquiring the attribute information; and The decision to process the data, based on the acquired attribute information, determines the flag identifying at least one of the first AI and the second AI as the additional information. In the first transmission process, the flag is sent to the first information processing device.

15. The information processing system as described in claim 12, wherein, The image acquisition device performs an acquisition process that uses the attribute information of the patient's examined eye image data as additional information. The first information processing device performs the determination process based on the received attribute information.

16. The information processing system as described in claim 12, wherein, The attribute information includes field-of-view information, which indicates the field of view of the image captured when the image data of the examined eye was taken. In the first image diagnosis, image diagnosis is performed on the image data of the examined eye captured at the first field of view. In the second image diagnosis, image diagnosis is performed on the image data of the examined eye taken at a second field of view that is wider than the first field of view.

17. The information processing system as described in claim 16, wherein, If the image diagnostic device detects an abnormality in the patient's examined eye image data during the first image diagnosis, it will include information indicating a recommendation to acquire image data of the patient's examined eye taken at the second field of view in the diagnostic result of the image diagnosis. The image diagnostic device sends the diagnostic results to the first information processing device.

18. The information processing system as described in claim 12, wherein, The examined eye image data includes at least one of fundus image data obtained from a fundus camera, fundus image data obtained from a scanning laser ophthalmoscope, and tomographic data obtained from an optical coherence tomography (OCT) system.

19. The information processing system as described in claim 18, wherein, The examined eye image data includes fundus image data. In the first image diagnosis and the second image diagnosis, the diagnosis of fundus lesions is performed.

20. The information processing system as described in claim 19, wherein, In the first image diagnosis and the second image diagnosis, a diagnosis of diabetic retinopathy was performed using fundus images.

21. The information processing system as described in claim 19, wherein, In the first image diagnosis, the diagnostic results are presented according to the first classification indicating the state of fundus lesions. In the second image diagnosis, the diagnostic results are displayed based on the second category, which has a different number of categories than the first category. The image diagnostic device sends the first diagnostic result based on the first classification from the first image diagnosis to the first information processing device. The image diagnostic device sends the second diagnostic result based on the second classification from the second image diagnosis to the first information processing device. The first information processing device sends the first diagnostic result and the second diagnostic result to the image acquisition device. The image acquisition device displays the combined diagnostic result after integrating the first diagnostic result and the second diagnostic result.

22. The information processing system as described in claim 12, wherein, The additional information identifies both the first artificial intelligence and the second artificial intelligence. The additional information includes information to determine the image diagnosis. The first information processing device sends the examined eye image data and the additional information to the image diagnostic device. When the image diagnostic device is capable of performing the image diagnostics indicated by the additional information, it performs the image diagnostics indicated by the additional information on the received image data of the examined eye.

Images