Information processing system

Abstract

To improve the convenience of consent information and privacy protection.SOLUTION: An image acquisition device generates first consent information indicating permission / non-permission of transmission of subject eye image data and second consent information indicating permission / non-permission of reuse of subject eye image data, and when the first consent information indicates transmission permission, transmits first transmission data including patient information, the subject eye image data, and the second consent information. A first information processing device stores the subject eye image data in first transmission data reception, generates identification information specific to the subject eye image data in first transmission data reception, and transmits second transmission data including the identification information, the subject eye image data, and the second consent information. A second information processing device performs image diagnosis on the subject eye image data in second transmission data reception, when the second consent information indicates non-permission, erases the subject eye image data, and when the second consent information indicates permission, stores the subject eye image data and diagnostic results of the image diagnosis, and transmits third transmission data including the identification information and the diagnosis results.SELECTED DRAWING: Figure 1

Description

Incorporation by Reference

[0001] This application claims the priority benefit of U.S. Provisional Application No. 62 / 880,975, filed on July 31, 2019 (Reiwa 1), the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present invention relates to an information processing system, an information processing apparatus, an information processing method, and an information processing program. Background Art

[0003] An ophthalmic information processing server capable of performing a wide variety of ophthalmic image analyses is known in the art. However, no consideration has been given to the handling of patient consent information. Prior Art Documents Patent Documents

[0004] Patent Document 1 Japanese Unexamined Patent Application Publication No. 2015-221276 Summary of the Invention

[0005] The first disclosed information processing system is an information processing system comprising: an image acquisition device for acquiring image data of a patient's eye; a first information processing device that can communicate with the image acquisition device and stores the image data of the eye; and a second information processing device that can communicate with the first information processing device and performs image diagnosis using the image data of the eye, wherein the image acquisition device performs a first generation process for generating first consent information indicating permission or denial for transmission of the image data of the eye to the first information processing device, and second consent information indicating permission or denial for reuse of the image data of the eye in the second information processing device after image diagnosis in the second information processing device; and, if the first consent information indicates permission for transmission, a first transmission process for transmitting first transmission data including the patient's patient information, the image data of the eye, and the second consent information to the first information processing device, and the first information processing device receives the first transmission data from the image acquisition device. Upon receiving the first transmission data, the second information processing device performs a storage process to store the image data of the eye being examined, a second generation process to generate identification information specific to the image data of the eye being examined, and a second transmission process to transmit the second transmission data, which includes the identification information, the image data of the eye being examined, and the second consent information, to the second information processing device. The second information processing device, upon receiving the second transmission data from the first information processing device, performs an image diagnosis process to perform an image diagnosis based on the image data of the eye being examined, a data processing process to delete the image data of the eye being examined after the image diagnosis process if the second consent information indicates that the reuse of the image data of the eye being examined is not permitted, and to save the image data of the eye being examined and the image diagnosis results from the image diagnosis process to a database if the second consent information indicates that the reuse of the image data of the eye being examined is permitted, and a third transmission process to transmit the third transmission data, which includes the identification information and the image diagnosis results, to the first information processing device.

[0006] The second disclosed information processing system is an information processing system that can communicate with a first information processing device that stores a patient's eye image data and a second information processing device that can communicate with the first information processing device, wherein the first information processing device performs a first transmission process to transmit the eye image data to the second information processing device based on consent information regarding consent for the use of the patient's eye image data in the second information processing device, and the second information processing device performs an image diagnosis process to perform an image diagnosis based on the eye image data transmitted by the first transmission process and a second transmission process to transmit the image diagnosis result from the image diagnosis process to the first information processing device.

[0007] The third disclosed technology is an information processing device that can communicate with other information processing devices, and comprises a processor that executes a program and a storage device that stores the program, wherein the processor performs an acquisition process to acquire consent information regarding consent for the use of a patient's eye image data in the other information processing device, and a transmission process to transmit the eye image data to the other information processing device based on the consent information acquired by the acquisition process.

[0008] The fourth disclosed information processing device is an information processing device capable of communicating with another information processing device that stores a patient's eye image data, and comprises a processor that executes a program, a storage device that stores the program, and is capable of accessing a database that stores learning data, wherein the processor performs: a receiving process that receives the eye image data and consent information indicating permission or denial of reuse of the eye image data in the information processing device from the other information processing device; an image diagnosis process that performs an image diagnosis based on the eye image data received by the receiving process; a transmission process that transmits the image diagnosis results from the image diagnosis process to the other information processing device; and a data processing related to the deletion of the eye image data or saving it to the database based on the consent information received by the receiving process.

[0009] The information processing device of the fifth disclosed technology is an information processing device that can communicate with another first information processing device that stores a patient's eye image data and another second information processing device that has a database for storing learning data, and comprises a processor that executes a program and a storage device that stores the program, wherein the processor performs a receiving process that receives the eye image data and consent information indicating permission or denial of reuse of the eye image data in the information processing device from the other first information processing device, an image diagnosis process that performs an image diagnosis based on the eye image data received by the receiving process, a transmission process that transmits the image diagnosis results from the image diagnosis process to the other first information processing device, and a data processing that performs deletion of the eye image data or transmission to the other second information processing device based on the consent information received by the receiving process.

[0010] The information processing program of the sixth disclosed technology causes the processor of an information processing device that can communicate with other information processing devices to execute an acquisition process to acquire consent information regarding the use of a patient's eye image data in the other information processing device, and a transmission process to transmit the eye image data to the other information processing device based on the consent information acquired by the acquisition process.

[0011] The information processing program of the seventh disclosed technology is capable of communicating with another information processing device that stores a patient's eye image data, and causes the processor of the information processing device, which has access to a database that stores learning data, to perform a receiving process to receive the eye image data and consent information indicating whether or not the reuse of the eye image data in the information processing device is permitted or not from the other information processing device; an image diagnosis process to perform an image diagnosis based on the eye image data received by the receiving process; a transmission process to transmit the image diagnosis results from the image diagnosis process to the other information processing device; and data processing related to the deletion of the eye image data or saving it to the database based on the consent information received by the receiving process.

[0012] The information processing program of the eighth disclosed technology causes the processor of an information processing device that can communicate with another first information processing device that stores a patient's eye image data and another second information processing device that has a database for storing learning data to perform a receiving process that receives the eye image data and consent information indicating permission or denial for reuse of the eye image data in the information processing device from the other first information processing device; an image diagnosis process that performs an image diagnosis based on the eye image data received by the receiving process; a transmission process that transmits the image diagnosis results from the image diagnosis process to the other information processing device; and data processing related to the deletion of the eye image data or transmission to the other second information processing device based on the consent information received by the receiving process.

[0013] The information processing method of the 9th disclosed technology is an information processing method performed by an information processing device that can communicate with another information processing device, and includes: an acquisition process for acquiring consent information regarding consent for the use of a patient's eye image data in the other information processing device; and a transmission process for transmitting the eye image data to the other information processing device based on the consent information acquired by the acquisition process.

[0014] The information processing method of the 10th disclosed technology is an information processing method performed by an information processing device that can communicate with another information processing device that stores a patient's eye image data and can access a database that stores learning data, and includes: a receiving process that receives the eye image data and consent information indicating permission or denial of reuse of the eye image data in the information processing device from the other information processing device; an image diagnosis process that performs an image diagnosis based on the eye image data received by the receiving process; a transmitting process that transmits the image diagnosis result from the image diagnosis process to the other information processing device; and a data control process that controls the deletion of the eye image data or saving it to the database based on the consent information received by the receiving process.

[0015] The information processing method of the 11th disclosed technology is an information processing method performed by an information processing device that can communicate with another first information processing device that stores a patient's eye image data and another second information processing device that has a database for storing learning data, and includes: a receiving process that receives the eye image data and consent information indicating permission or denial of reuse of the eye image data in the information processing device from the other first information processing device; an image diagnosis process that performs an image diagnosis based on the eye image data received by the receiving process; a transmission process that transmits the image diagnosis result from the image diagnosis process to the other information processing device; and a data control process that controls the deletion of the eye image data or transmission to the other second information processing device based on the consent information received by the receiving process. [Brief explanation of the drawing]

[0016] [Figure 1] Figure 1 is an explanatory diagram showing an example of image diagnosis using consent information. [Figure 2] Figure 2 is a block diagram showing an example of a computer hardware configuration. [Figure 3] Figure 3 is an explanatory diagram showing an example of the contents stored in the patient information database. [Figure 4] Figure 4 is an explanatory diagram showing an example of the operation sequence of the information processing system according to Example 1. [Figure 5] Figure 5 is a flowchart showing a detailed example of the processing procedure for the diagnostic data transmission decision process (step S414) shown in Figure 4. [Figure 6] Figure 6 is a flowchart showing a detailed example of the processing procedure (step S433) for handling fundus image data shown in Figure 4. [Figure 7] Figure 7 is a flowchart showing a detailed example of the processing procedure (step S433) for handling fundus image data shown in Figure 4. [Figure 8] Figure 8 is an explanatory diagram showing Example 1 of the correction process for fundus image data (step S702). [Figure 9]FIG. 9 is an explanatory diagram showing Example 2 of the correction process for fundus image data (step S702). [Figure 10] FIG. 10 is a flowchart showing Detailed Processing Procedure Example 3 of the handling process for the fundus image data shown in FIG. 4 (step S433). [Figure 11] FIG. 11 is a flowchart showing another detailed example of the processing procedure of the fundus image data correction process (step S702) shown in FIGS. 7 and 9. [Figure 12] FIG. 12 is an explanatory diagram showing examples of the severity of diabetic retinopathy. [Figure 13] FIG. 13 is a block diagram showing an example of a functional configuration of the information processing system according to Embodiment 1. [Figure 14] FIG. 14 is an explanatory diagram showing an example of an operation sequence of the information processing system according to Embodiment 2. [Figure 15] FIG. 15 is a flowchart showing a detailed example of the processing procedure of the handling process for the fundus image data shown in FIG. 14 (step S432). [Figure 16] FIG. 16 is a block diagram showing an example of a functional configuration of the information processing system according to Embodiment 2.

Mode for Carrying Out the Invention

Examples

[0017] <Example of Image Diagnosis Using Consent Information> FIG. 1 is an explanatory diagram showing an example of fundus image diagnosis using consent information. The fundus image diagnosis using consent information is executed by an information processing system 100. The information processing system 100 includes an in-hospital system 101, a management server 120, and an AI (Artificial Intelligence) server 130. The in-hospital system 101 and the management server 120 are communicatively connected to each other. The management server 120 and the AI server 130 are communicatively connected to each other.

[0018] The in-hospital system 101 is installed, for example, in a hospital or clinic (such as an ophthalmologist's office, an internist's office, or a diabetologist's office). The in-hospital system 101 includes an ophthalmic device 111, a terminal 112, and an in-hospital server 113. The ophthalmic device 111 is connected to the terminal 112 for communication. The ophthalmic device 111 includes a fundus camera, a scanning laser ophthalmoscope, and an optical coherence tomography (OCT) that scan the eye under examination with laser light and generate an image based on the reflected light from the fundus, and generates fundus image data 102R and 102L of the eye under examination. If the fundus image data 102R of the right eye and the fundus image data 102L of the left eye are not distinguished, they are referred to as fundus image data 102. The same applies to other codes that have "R" or "L" at the end. The ophthalmic device 111 transmits the generated fundus image data 102 to the terminal 112. The fundus image data 102 includes the date of acquisition. The fundus image data may be fundus image data acquired by a fundus camera, fundus image data acquired by a scanning laser ophthalmoscope, or fundus tomographic data acquired by an optical coherence tomography (OCT). Alternatively, it may be a fundus image dataset consisting of two or more of these.

[0019] The ophthalmic device 111 is comprised of, for example, at least one of a fundus camera, a scanning laser ophthalmoscope, and an optical coherence tomography (OCT) device. Therefore, the fundus image data 102 consists of one or more combinations of fundus image data from the fundus camera, fundus image data from the scanning laser ophthalmoscope, or tomographic data from the OCT device.

[0020] Terminal 112 is a computer capable of communicating with the ophthalmic device 111 and the hospital server 113. Terminal 112 may also be capable of direct communication with the management server 120. Terminal 112 is used, for example, by a physician. Terminal 112 is, for example, a personal computer or a tablet. Terminal 112 transfers fundus image data 102 from the ophthalmic device 111 to the hospital server 113.

[0021] Furthermore, terminal 112 transmits consent information 103 to the hospital server 113. Consent information 103 is information indicating whether patient i consents to the use of fundus image data 102 of their eye outside the hospital system 101. Specifically, for example, consent information 103 includes first consent information 103a and second consent information 103b.

[0022] The first consent information 103a is, for example, an external storage flag that indicates permission or denial for sending fundus image data 102 to the management server 120 outside the hospital system 101. If the external storage flag is "1", it indicates permission (OK) for sending fundus image data 102 to the management server 120 outside the hospital system 101, and if it is "0", it indicates denial (NG).

[0023] The second consent information 103b is, for example, a secondary use flag indicating permission or denial for the reuse of fundus image data 102 on the AI ​​server 130 after image diagnosis on the AI ​​server 130 outside the hospital system 101. If the secondary use flag is "1", it indicates permission (OK) for secondary use of fundus image data 102 on the AI ​​server 130, and if it is "0", it indicates denial (NG). Consent information 103 exists for each patient i. Consent information 103 may also have an expiration date. Furthermore, consent information 103 may exist for each patient i for each imaging session. The data format of consent information 103 can be any data format as long as it is recognizable by the hospital server 113, the management server 120, and the AI ​​server 130.

[0024] Furthermore, terminal 112 receives patient information and diagnostic result data 105 from the in-hospital server 113 and displays them on the display screen. The fundus image data 102 from the ophthalmic device 111, the patient information from the in-hospital server 113, the diagnostic results included in the diagnostic result data 105, and the consent information 103 are associated with the same ID (for example, patient ID) by terminal 112 or the in-hospital server 113.

[0025] The hospital server 113 is a computer capable of communicating with terminals 112 and management server 120. The hospital server 113 has a patient information database 114. The patient information database 114 is a database that stores patient information. The hospital server 113 receives patient IDs and fundus image data 102 from terminals 112. The hospital server 113 stores the fundus image data 102, the diagnostic results included in the diagnostic result data, and consent information 103 in the patient information database 114 in association with the patient information identified by the patient ID.

[0026] If the first consent information 103a indicates permission to send fundus image data 102 to the management server 120 outside the hospital system 101, the hospital server 113 sends the diagnostic data 104 to the management server 120; if it indicates permission is denied, it does not send the data. This allows the hospital server 113 to comply with patient i's first consent information 103a. The diagnostic data 104 includes patient i's patient information, fundus image data 102, and second consent information 103b. The hospital server 113 receives diagnostic result data 105 from the management server 120. The hospital server 113 stores the diagnostic results contained in the received diagnostic result data 105 in the patient information DB 114.

[0027] The management server 120 is a computer capable of communicating with the hospital server 113 and the AI ​​server 130. The management server 120 receives diagnostic data 104 from the hospital server 113. The management server 120 anonymizes the received diagnostic data 104. Specifically, for example, the management server 120 issues a new ID (hereinafter referred to as an anonymized ID) and associates it with the patient information in the received diagnostic data 104. The management server 120 then uses the combination of the anonymized ID and the fundus image data 102 as anonymized diagnostic data 106.

[0028] Furthermore, the patient ID within the patient information may be used as an anonymized ID. In this case, the anonymized diagnostic data 106 consists of the patient ID (which is the anonymized ID), fundus image data 102, and second consent information 103b. This anonymizes the patient information. The management server 120 may also include in the anonymized diagnostic data 106 any information from the patient information that does not uniquely identify patient i. Examples of information that does not uniquely identify patient i include patient i's visual acuity, gender, age, and nationality.

[0029] Furthermore, the management server 120 may encrypt the patient information. In this case, the management server 120 transmits the combination of the encrypted patient information and the fundus image data 102 to the AI ​​server 130 as anonymized diagnostic data.

[0030] The management server 120 receives anonymized diagnostic result data 107 from the AI ​​server 130. The anonymized diagnostic result data 107 includes the anonymized ID contained in the anonymized diagnostic data 106 and the diagnostic result, which is the severity (also called the progression) of the symptoms of the eye being examined. The management server 120 converts the received anonymized diagnostic result data 107 into diagnostic result data 105.

[0031] Specifically, for example, the management server 120 retrieves patient information associated with an anonymized ID contained in the received anonymized diagnostic result data 107, and generates diagnostic result data 105 containing patient information and diagnostic results by replacing the anonymized ID with the retrieved patient information.

[0032] When the management server 120 sends anonymized diagnostic data 106, which is a combination of encrypted patient information, fundus image data 102, and second consent information 103b, to the AI ​​server 130, the management server 120 will receive anonymized diagnostic result data 107, which includes the encrypted patient information and the diagnostic result, from the AI ​​server 130.

[0033] In this case, the management server 120 decrypts the encrypted patient information, converting the anonymized diagnostic result data 107 into diagnostic result data 105. In this way, the management server 120 can conceal patient information through anonymization and encryption, thereby protecting personal information. After this, the management server 120 sends the generated diagnostic result data 105 to the in-hospital server 113.

[0034] The AI ​​server 130 is a computer that performs fundus image diagnosis using AI, employing learning parameters obtained through machine learning and deep learning. The AI ​​server 130 learns from past fundus image data 103 and its progression as training data, and generates learning parameters. The training data set and learning parameters are stored in the learning database 1131. The learning database 1131 also stores findings 105 corresponding to the progression. Using these learning parameters, features of the fundus image are extracted using a convolutional neural network (CNN). Then, the symptoms of the input fundus image are estimated based on these features.

[0035] The AI ​​server 130 receives anonymized diagnostic data 106. The AI ​​server 130 inputs the fundus image data 102 contained in the anonymized diagnostic data 106 into a learning model to which learning parameters have been applied to a CNN, and outputs the severity level.

[0036] The AI ​​server 130 generates anonymized diagnostic result data 107 which includes the anonymized ID contained in the anonymized diagnostic data 106 and the severity level output from the learning model. The AI ​​server 130 sends the generated anonymized diagnostic result data 107 to the management server 120.

[0037] If the second consent information 103 indicates permission for the reuse of fundus image data at the AI ​​server 130 after image diagnosis at the AI ​​server 130 outside the hospital system 101, the AI ​​server 130 adds the combination of severity and the fundus image data 102 from which the severity was output as training data to the training data set in the training database. If it indicates denial, the AI ​​server 130 does not add it. In this way, the AI ​​server 130 can comply with patient i's second consent information 103. In this manner, the information processing system 100 can protect patient i's personal information.

[0038] <Example of computer hardware configuration> Figure 2 is a block diagram showing an example of the hardware configuration of a computer (terminal 112, in-hospital server 113, management server 120, AI server 130, DB server 1400). Computer 200 includes a processor 201, a storage device 202, an input device 203, an output device 204, and a communication interface (communication IF) 205. The processor 201, storage device 202, input device 203, output device 204, and communication IF 205 are connected by a bus 206. The processor 201 controls computer 200.

[0039] The memory device 202 serves as the work area for the processor 201. The memory device 202 is also a non-temporary or temporary recording medium for storing various programs and data. Examples of memory devices 202 include ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), and flash memory.

[0040] Input device 203 inputs data. Examples of input devices 203 include keyboards, mice, input pens, touch panels, numeric keypads, and scanners. Output device 204 outputs data. Examples of output devices 204 include displays and printers. Communication IF 205 connects to the network and sends and receives data.

[0041] <Patient information DB114> Figure 3 is an explanatory diagram showing an example of the contents stored in the patient information DB 114. The patient information DB 114 has a patient information field 301 and a diagnostic information field 302. The patient information field 301 has subfields: a patient ID field 311, a name field 312, a gender field 313, a date of birth field 314, and a contact information field 315. Subfields 311 to 315 in the same row represent the patient information of patient i (where i is, for example, an integer of 1 or more).

[0042] The patient ID field 311 is a memory area that stores the patient ID. The patient ID Pi is identification information that uniquely identifies patient i. The name field 312 is a memory area that stores the name FNi of patient i. The gender field 313 is a memory area that stores the gender Si of patient i. The date of birth field 314 is a memory area that stores the date of birth DOBi of patient i. The contact field 315 is a memory area that stores the contact ADi of patient i.

[0043] The diagnostic information field 302 is a memory area that stores the diagnostic information Di1 to Dij up to the jth time (where j is an integer greater than or equal to 1) for patient i. The diagnostic information Dij includes the fundus image data 102, the diagnostic result Rij, the date of imaging Tij, and the consent information 103. The diagnostic result Rij includes the severity level from the AI ​​server 130. The date of imaging Tij is the date on which the fundus image data 102 was generated by imaging the eye of the subject with the ophthalmic device 111. The consent information 103 includes the first consent information 103a (external storage flag) and the second consent information 103b (secondary use flag) described above.

[0044] <Example of operation sequence of information processing system 100> Figure 4 is an explanatory diagram showing an example of the operation sequence of the information processing system 100 according to Embodiment 1. When terminal 112 and in-hospital server 113 are not distinguished, they are referred to as the image acquisition device 400. The image acquisition device 400 acquires patient information of patient i, who will be the subject of the examination, from patient information DB 114 (step S411). The image acquisition device 400 acquires fundus image data 102 of patient i via terminal 112 (step S412). In addition, the in-hospital server 113 acquires consent information 103 of patient i via terminal 112 (step S413).

[0045] Then, the image acquisition device 400 performs a transmission decision process (step S414) for the diagnostic data 104, which includes the patient information acquired in step S411, the fundus image data 102 acquired in step S412, and the second consent information 103b acquired in step S413. This transmission decision process for the diagnostic data 104 (step S414) is a process that determines whether or not to transmit the diagnostic data 104 according to the contents of the first consent information 103a.

[0046] The image acquisition device 400 decides to transmit the diagnostic data 104 if the first consent information 103a indicates permission to transmit, and decides not to transmit the diagnostic data 104 if the first consent information 103a indicates permission not to transmit. Details of the process for deciding whether to transmit the diagnostic data 104 (step S414) will be described later. As a result, the image acquisition device 400 transmits the diagnostic data 104 to the management server 120 (step S415) only if it decided to transmit the diagnostic data 104 in step S414.

[0047] When diagnostic data 104 is transmitted from the image acquisition device 400, the management server 120 receives the diagnostic data 104 (step S421). Next, the management server 120 anonymizes the patient information contained in the diagnostic data 104 (step S422). Then, the management server 120 transmits the anonymized diagnostic data 106, which includes an anonymized ID associated with the patient information, fundus image data 102, and second consent information 103b (secondary use flag), to the AI ​​server 130 (step S423).

[0048] When the AI ​​server 130 receives the anonymized diagnostic data 106 from the management server 120, it receives the anonymized diagnostic data 106 (step S431). Next, the AI ​​server 130 performs fundus image diagnosis by inputting the fundus image data 102 contained in the anonymized diagnostic data 106 into the learning model (step S432).

[0049] After performing fundus image diagnosis (step S432), the AI ​​server 130 performs processing on the fundus image data 102 (step S433). The processing on the fundus image data 102 (step S433) determines how to handle the fundus image data 102 obtained from fundus image diagnosis (step S432), such as whether or not to use it as training data, based on the content of the second consent information 103b. Details of the processing on the fundus image data 102 (step S433) will be described later.

[0050] Then, the AI ​​server 130 sends anonymized diagnostic result data 107, which includes an anonymized ID and a diagnostic result Rij, which is the severity level output by the fundus image diagnosis, to the management server 120 (step S434).

[0051] Next, the AI ​​server 130 performs the learning process (step S435). Specifically, for example, in the processing of handling fundus image data 102 (step S433), the AI ​​server 130 adds the combination of fundus image data 102 included in the anonymized diagnostic data received in step S431 and the severity level Rij output in step S432 to the learning data set of the learning DB 131 only if it is permitted to use this combination as learning data (when the second consent information 103b (secondary use flag) indicates that secondary use is OK), and updates the learning model with the added learning data set.

[0052] Furthermore, when the management server 120 receives the anonymized diagnostic result data 107 from the AI ​​server 130, it receives the anonymized diagnostic result data 107 (step S424). At this time, the management server 120 may send completion information to the image acquisition device 400 indicating that it has finished receiving the anonymized diagnostic result data 107. Next, the management server 120 restores the patient information based on the anonymized ID contained in the anonymized diagnostic result data 107 (step S425).

[0053] For example, the management server 120 reads patient information, including the patient ID associated with the anonymized ID, which was held in step S422. Then, the management server 120 generates diagnostic result data 105, which includes the acquired patient information and the diagnostic result Rij (step S426), and transmits the diagnostic result data 105 to the image acquisition device 400 (step S427).

[0054] When the image acquisition device 400 receives completion information from the management server 120, it may send a request to the management server 120 to acquire the diagnostic result data 105. Upon receiving this request, the management server 120 sends the diagnostic result data 105 to the image acquisition device 400. After this, the management server 120 saves the diagnostic result data 105 (step S428). Then, the management server 120 terminates its processing.

[0055] When the image acquisition device 400 receives the diagnostic result data 105 from the management server 120, it receives the diagnostic result data 105 (step S416). As a result, the terminal 112 can use the diagnostic result data 105 and fundus image data 102 from the in-hospital server 113 to display the fundus image data 102, severity, and patient information on the display screen. The display process is then terminated when the user operates the end button (not shown) as instructed.

[0056] Figure 5 is a flowchart showing a detailed example of the processing procedure for the decision to transmit the diagnostic data 104 shown in Figure 4 (step S414). If the external storage flag, which is the first consent information 103a, is "1" indicating permission for external storage (OK) (step S501: Yes), the image acquisition device 400 decides to transmit the diagnostic data 104 and terminates the decision to transmit the diagnostic data 104 (step S414). On the other hand, if the external storage flag, which is the first consent information 103a, is "0" indicating permission for external storage (NG) (step S501: No), the in-hospital server 113 decides not to transmit the diagnostic data 104 and terminates the decision to transmit the diagnostic data 104 (step S414). In other words, if the external storage flag, which is the first consent information 103a, indicates permission for external storage (NG), the captured fundus image is stored only on the in-hospital server, and no AI-based image diagnosis is performed.

[0057] Figure 6 is a flowchart of a detailed example of the processing procedure for handling the fundus image data 102 (step S433) shown in Figure 4. If the secondary use flag, which is the second consent information 103b, is "1" indicating permission for secondary use (OK) (step S601: Yes), the AI ​​server 130 saves the combination of fundus image data 102 and the diagnostic result Rij as training data in the training DB 131 and terminates the processing of fundus image data 102 (step S433). On the other hand, if the secondary use flag, which is the second consent information 103b, is "0" indicating denial of secondary use (NG) (step S601: No), the AI ​​server 130 deletes the fundus image data 102 and terminates the processing of fundus image data 102 (step S433).

[0058] Figure 7 is a flowchart of a detailed processing procedure example 2 for the handling of the fundus image data 102 shown in Figure 4 (step S433). In Figure 6, if the secondary use flag was OK, the fundus image data 102 and the diagnostic result Rij were saved directly as training data in the training DB 131. However, in Figure 7, even if the secondary use flag was OK, the AI ​​server 130 modifies the fundus image data 102 before saving it as training data in the training DB 131, taking into consideration the protection of personal information. The same step numbers are used for steps identical to those in Figure 6, and their explanations are omitted.

[0059] If the secondary use flag, which is the second consent information 103b, is "1" indicating permission for secondary use (OK) (step S601: Yes), the AI ​​server 130 performs a modification process on the fundus image data 102 (step S702). Here, an example of the modification process on the fundus image data 102 (step S702) is described. The modification process (step S702) is an image processing process to prevent the fundus image data from being misused if the fundus image data is leaked to an external party. For example, it is a process aimed at preventing the misuse of fundus image data in a retinal authentication system or the misregistration of fundus image databases.

[0060] Figure 8 is an explanatory diagram showing Example 1 of the correction process (step S702) for fundus image data 102. In Figure 8, the AI ​​server 130 flips the fundus image data 102R of the right eye horizontally to convert it into the fundus image data 802L of the left eye, and flips the fundus image data 102L of the left eye horizontally to convert it into the fundus image data 802R of the right eye. In this case, the AI ​​server 130 also changes the diagnostic result RijR to the diagnostic result RijL, and vice versa.

[0061] Figure 9 is an explanatory diagram showing example 2 of the correction process (step S702) for fundus image data 102. Figure 9 shows an example of correction of fundus image data 102 for the left eye, but similar corrections can be made for the right eye. The AI ​​server 130 can correct fundus image data 102L to fundus image data 102L1 by adding vascular shape data to fundus image data 102L.

[0062] Furthermore, the AI ​​server 130 can modify the fundus image data 102L to fundus image data 102L2 by changing the path of the vascular shape data in the fundus image data 102L. Also, the AI ​​server 130 can modify the fundus image data 102L to fundus image data 102L3 by deleting the vascular shape data from the fundus image data 102L. Unlike left-right reversal, the diagnostic result Rij does not change when vascular data is added, modified, or deleted.

[0063] In addition, when making additions, modifications, and deletions as shown in Figure 9, it is preferable for the AI ​​server 130 to modify the image data within healthy areas of the fundus image data 102L that are not related to abnormal areas, by referring to the diagnostic result Rij. Healthy areas are, for example, areas where microaneurysms do not exist if the diagnostic result Rij is mild non-proliferative retinopathy (Mild; hereinafter referred to as retinopathy 1), areas that are not avascular areas if the diagnostic result Rij is moderate or severe non-proliferative retinopathy (Moderate or Severe; hereinafter referred to as retinopathy 2), and areas where neovascularization has not occurred if the diagnostic result Rij is proliferative retinopathy (Proliferative; hereinafter referred to as retinopathy 3). Modifying healthy areas does not affect the diagnostic result Rij. This makes it possible to maintain the original diagnostic result even after modifying the fundus image data 102.

[0064] Furthermore, by modifying the fundus image data 102 as shown in Figures 9 and 10, even if the modified fundus image data 802 is leaked externally, it will no longer be authenticated even if misused in retinal authentication. In this way, Meng Huo authentication can be disabled by modification, thereby improving security. In addition to modifying the fundus image data 102, further security improvements may be made by encrypting the fundus image.

[0065] Returning to Figure 7, the AI ​​server 130 saves the combination of the corrected fundus image data 802 and the diagnostic result Rij as training data in the training DB 131 (step S704), and terminates the processing of the fundus image data 102 (step S433). This allows for the secondary use of the corrected fundus image data 802 while respecting the privacy of patient i.

[0066] Figure 10 is a flowchart of a detailed processing procedure example 3 for handling the fundus image data 102 shown in Figure 4 (step S433). Figures 7 to 9 show examples where the fundus image data 102 has been modified, but depending on the extent of the modification, the resulting diagnosis Rij may differ from the diagnosis Rij before the modification.

[0067] Therefore, in Figure 10, the AI ​​server 130 performs a fundus image diagnosis again on the corrected fundus image data 802. If the diagnosis result Rij is the same as the original fundus image data 102, it saves it in the training DB 131 for secondary use as training data, thereby achieving both the protection of personal information and the identity of the diagnosis results before and after correction.

[0068] Steps identical to those in Figures 6 and 7 are given the same step numbers, and their explanations are omitted. After the correction process of the fundus image data 102 shown in Figure 7 (step S702), the AI ​​server 130 re-executes the fundus image diagnosis using the corrected fundus image data 802 (step S1001).

[0069] Then, the AI ​​server 130 determines whether the re-diagnosis result Rij from step S1001 matches the diagnosis result Rij before the change (step S1002). Specifically, for example, if the fundus image data 102 is flipped horizontally in step S702 as shown in Figure 8, the AI ​​server 130 determines whether the re-diagnosis result RijR of the right eye matches the diagnosis result RijL of the left eye before the change, and whether the re-diagnosis result RijL of the left eye matches the diagnosis result RijR of the right eye before the change.

[0070] Furthermore, if step S702 involves adding, changing, or deleting vascular shape data as shown in Figure 9, rather than horizontally flipping the fundus image data 102, the AI ​​server 130 determines whether the re-diagnosis result RijR of the right eye matches the diagnosis result RijR of the right eye before correction, and whether the re-diagnosis result RijL of the left eye matches the diagnosis result RijL of the left eye before correction.

[0071] If neither eye matches (step S1002: No), the process returns to step S702, and the AI ​​server 130 re-executes the correction process for the fundus image data 102 (step S702). In this case, the AI ​​server 130 performs a different correction than those already performed, such as adding, changing, or deleting vascular shape data. However, if the vascular shape data to be added, changed, or deleted is different from that which has already been added, changed, or deleted, it will be considered a different correction.

[0072] On the other hand, if both eyes match (step S1002: Yes), the AI ​​server 130 saves the latest corrected fundus image data 802 and the diagnostic result Rij combination as training data in the training DB 131 (step S704), and terminates the processing of the fundus image data 102 (step S433).

[0073] This prevents the misuse of patient i's fundus image data and ensures the identity of the diagnostic result Rij before and after correction, while also considering the protection of patient i's privacy. This allows for the secondary use of the corrected fundus image data 802, which does not change the diagnostic result Rij. Furthermore, when the combination of the corrected fundus image data 802 and its diagnostic result Rij is used as training data, it is possible to suppress the decrease in prediction accuracy by the learning model used for fundus image diagnosis by the AI ​​server 130.

[0074] Figure 11 is a flowchart showing other detailed processing steps for the fundus image data correction process (step S702) shown in Figures 7 and 9. Figure 11 shows an example of a processing step in which the AI ​​server 130 corrects the shape data of blood vessels in the retina according to the severity level, which is the diagnostic result Rij. Figure 12 is an explanatory diagram showing examples of diabetic retinopathy severity levels.

[0075] In Figure 11, the AI ​​server 130 determines whether the diagnosis result Rij is normal (No DR) (step S1101). If the diagnosis result Rij is normal (No DR) (step S1101: Yes), the AI ​​server 130 corrects the vascular shape data of the fundus image data 102 as shown in Figure 8 or Figure 9 (step S1102). Then, it finishes the fundus image data correction process (step S702) and moves on to the next process.

[0076] On the other hand, if the diagnosis result Rij is not normal (No DR) (Step S1101: No), the AI ​​server 130 determines whether the diagnosis result Rij is mild or proliferative (Step S1103). If the diagnosis result Rij is mild or proliferative (Step S1103: Yes), the AI ​​server 130 corrects the vascular shape data of the fundus image data 102 as shown in Figure 8 or Figure 9 (Step S1104).

[0077] In mild retinopathy (stage 1), capillary accumulation occurs. In proliferative retinopathy (stage 3), neovascularization occurs. Therefore, to prevent the deletion of shape data of capillary accumulation and neovascularization, deletion of shape data is not permitted in the modification in step S1104, which includes addition, modification, and deletion. Then, the fundus image data modification process (step S702) is completed and the process moves on to the next stage.

[0078] On the other hand, if the diagnosis Rij is neither mild nor proliferative (step S1103: No), then the diagnosis Rij corresponds to moderate or severe retinopathy. Therefore, the AI ​​server 130 corrects the vascular shape data of the fundus image data 102 as shown in Figure 8 or Figure 9 (step S1105).

[0079] In retinopathy II (moderate or severe), a predetermined range of avascular areas exists. Therefore, to prevent the addition of vascular shape data to the avascular areas, the modification in step S1105 does not allow the addition, modification, or deletion of shape data. Furthermore, even if the modification adds vascular shape data to the avascular areas, this is considered an addition of shape data, and such modification is not permitted. The fundus image data modification process (step S702) is then terminated, and the process moves on to the next step.

[0080] In this way, the AI ​​server 130 can prevent changes to the diagnostic result Rij before and after correction by modifying the shape data representing the eye tissue according to the diagnostic result Rij. This ensures the identity of the diagnostic result Rij before and after correction while respecting the privacy of patient i, allowing for the secondary use of the corrected fundus image data 802, which does not change the diagnostic result Rij. Furthermore, this suppresses a decrease in the prediction accuracy of the learning model used for fundus image diagnosis by the AI ​​server 130 when the combination of the corrected fundus image data 802 and its diagnostic result Rij is used as training data.

[0081] <Example of a functional configuration of an information processing system> Figure 13 is a block diagram showing an example of the functional configuration of an information processing system according to Embodiment 1. In Figure 13, the information processing device 1300 is a computer that includes an image acquisition device 400 (terminal 112, in-hospital server 113) and a management server 120. The information processing device 1300 has an acquisition unit 1301, a first transmission unit 1302, and a first reception unit 1303. Specifically, the acquisition unit 1301, the first transmission unit 1302, and the first reception unit 1303 are realized, for example, by causing the processor 201 to execute a program stored in the storage device 202 shown in Figure 2.

[0082] The acquisition unit 1301 acquires various data using the image acquisition device 400. Specifically, for example, as shown in steps S411 to S413 of Figure 4, the acquisition unit 1301 acquires patient information and diagnostic information Dij of the target patient i from the patient information DB 114 using the image acquisition device 400. The diagnostic information Dij includes consent information 103. Furthermore, the acquisition unit 1301 obtains an anonymized ID as unique identification information for the fundus image data 102 by having the management server 120 perform an anonymization process of the patient information as shown in step S422 of Figure 4.

[0083] Furthermore, as shown in steps S425 and S426 of Figure 4, the acquisition unit 1301 acquires patient i's diagnosis result data 105 based on patient information identified from the anonymized ID contained in the anonymized diagnosis result data 107 from the AI ​​server 130 and the diagnosis result Rij contained in the anonymized diagnosis result data 107, and stores it in the management server 120.

[0084] The first transmission unit 1302 transmits various data via the image acquisition device 400. Specifically, for example, as shown in step S415 of Figure 4, if the first consent information 103a indicates permission to transmit, the first transmission unit 1302 transmits diagnostic data 104, including patient information of patient i, fundus image data 102, and second consent information 103b, to the management server 120. Also, as shown in step S423 of Figure 4, the first transmission unit 1302 transmits anonymized diagnostic data 106, including an anonymized ID, fundus image data 102, and second consent information 103b, to the AI ​​server 130 via the management server 120.

[0085] The first receiving unit 1303 receives various data from the image acquisition device 400. Specifically, for example, as shown in step S424 of Figure 4, the first receiving unit 1303 receives anonymized diagnostic result data 107 from the AI ​​server 130. The first receiving unit 1303 also receives completion information from the image acquisition device 400 indicating that the reception of the anonymized diagnostic result data 107 in step S424 has been completed. The first receiving unit 1303 also receives diagnostic result data 105 from the management server 120 from the image acquisition device 400.

[0086] The AI ​​server 130 includes a second receiving unit 1311, an image diagnosis unit 1312, a second transmitting unit 1313, a data control unit 1314, a learning unit 1315, a modification unit 1316, and a learning DB 131. The second receiving unit 1311, the image diagnosis unit 1312, the second transmitting unit 1313, the data control unit 1314, the learning unit 1315, and the modification unit 1316 are specifically implemented, for example, by causing the processor 201 to execute a program stored in the storage device 202 shown in Figure 2. The learning DB 131 is specifically implemented, for example, by the storage device 202 shown in Figure 2.

[0087] The second receiving unit 1311 receives the anonymized diagnostic data 106 transmitted from the first transmitting unit 1302 of the information processing device 1300, as shown in step S431 of Figure 4.

[0088] As shown in step S432 of Figure 4, the image diagnostic unit 1312 inputs fundus image data into a learning model to which the updated learning parameters have been applied to a CNN, performs fundus image diagnosis, and outputs the severity as the diagnostic result Rij to the second transmission unit 1313.

[0089] As shown in step S434 of Figure 4, the second transmission unit 1313 transmits the anonymized diagnostic result data, including the diagnostic result Rij, to the first receiving unit 1303 of the information processing device 1300.

[0090] The data control unit 1314 performs the handling of the fundus image data shown in Figures 7 and 10, as shown in step S433 of Figure 4. Specifically, for example, the data control unit 1314 has a storage unit 1341 and an erasure unit 1342. The storage unit 1341 stores the learning data, which is a combination of the fundus image data 102 diagnosed by the image diagnosis unit 1312 and the diagnosis result Rij, if the secondary use flag is OK, and the erasure unit 1342 erases the fundus image data 102 diagnosed by the image diagnosis unit 1312 if the secondary use flag is NG.

[0091] The learning unit 1315 generates a trained model by providing the training data set in the training DB 131 to the CNN and updating the CNN's training parameters.

[0092] The modification unit 1316 modifies the fundus image data 102 diagnosed by the image diagnosis unit 1312, as shown in step S702 of Figures 7 and 10. In this case, if the secondary use flag is OK, the storage unit 1341 saves the training data, which is a combination of the modified fundus image data 802 and the diagnostic result Rij modified by the modification unit 1316. The modification unit 1316 also obtains the diagnostic result Rij for the modified fundus image data 802 by inputting the modified fundus image data 802 into the latest training model by the learning unit 1315. In this case, the modification unit 1316 may continue modifying the fundus image data 102 until the diagnostic result Rij before and after modification matches.

[0093] Thus, in Example 1, by ensuring the identity of the diagnostic result Rij before and after correction while respecting the privacy of patient i, the corrected fundus image data 802, which does not change the diagnostic result Rij, can be reused. Furthermore, this makes it possible to suppress the decrease in prediction accuracy by the learning model used for fundus image diagnosis by the AI ​​server 130 when the combination of the corrected fundus image data 802 and its diagnostic result Rij is used as training data.

[0094] Furthermore, the image acquisition device 400 may generate first consent information 103a and second consent information 103b when it is the first time for patient i to undergo image diagnosis, and may use the first consent information 103a and second consent information 103b generated the first time when it is the second or subsequent time for patient i to undergo image diagnosis. This eliminates the need to reset the consent information 103, thereby improving convenience.

[0095] Furthermore, if it is the first time for patient i to undergo image diagnosis, the image acquisition device 400 generates first consent information 103a and second consent information 103b. If it is the second or subsequent time for patient i to undergo image diagnosis, it may use the first consent information 103a generated the first time and output an update request for the second consent information 103b. The update request is displayed, for example, on the display screen of terminal 112. When patient i or the doctor sees the update request on the display screen, they set the secondary use flag, which is the second consent information 103b, to "1" (OK) or "0" (NG). After setting, the hospital server 113 refers to the first consent information 103a to determine whether or not to send it to the management server 120, and if it decides to send it, it sends the diagnostic data 104 to the management server 120.

[0096] This eliminates the need to reset the first consent information 103a, thereby improving convenience, and also enhances privacy protection by prompting patient i to reconfirm the second consent information 103b. [Examples]

[0097] Next, we will describe Example 2. In Example 1, the AI ​​server 130 had the learning DB 131, but in Example 2, we will describe an example where the AI ​​server 130 and the learning DB 131 are separated. The learning DB 131 is owned by a DB server 1400, which is different from the AI ​​server 130. The functions of DB server 1400 may be handled by the management server 120. Note that in Example 2, we will focus on the differences from Example 1, so the same reference numerals are used for the same components as in Example 1, and their explanations will be omitted.

[0098] Figure 14 is an explanatory diagram showing an example of the operation sequence of the information processing system 100 according to Embodiment 2. The AI ​​server 130 obtains learning parameters from the DB server 1400 and performs fundus image diagnosis (step S1432). The difference from step S432 is that learning parameters are obtained from the DB server 1400.

[0099] After performing fundus image diagnosis (step S1432), the AI ​​server 130 sends anonymized diagnostic data 106 to the DB server 1400 (step S1401). In this case, the DB server 1400 performs processing on the fundus image data 102 (step S1433). Meanwhile, the AI ​​server 130 deletes the fundus image data 102 to prevent leakage (step S1402). The AI ​​server 130 also obtains the training data set from the training DB 131 and performs training processing (step S1434). The difference from step S434 is that the training data set is obtained from the DB server 1400. The AI ​​server 130 then finishes processing the received fundus image and enters a waiting state to receive the next fundus image.

[0100] Figure 15 is a flowchart showing a detailed example of the processing procedure for handling the fundus image data 102 (step S432) shown in Figure 14. When the DB server 1400 receives the anonymized diagnostic data 106, if the secondary use flag, which is the second consent information 103b, is "0" indicating that secondary use is not permitted (NG) (step S601: No), it deletes the fundus image data 102 and terminates the processing for handling the fundus image data 102 (step S1433).

[0101] On the other hand, if the secondary use flag, which is the second consent information 103b, is "1" indicating permission for secondary use (OK) (step S601: Yes), the DB server 1400 performs the correction process for the fundus image data 102 (step S702). Then, the DB server 1400 sends the corrected fundus image data 802 to the AI ​​server 130 (step S1511).

[0102] When the AI ​​server 130 receives the corrected fundus image data 802 from the DB server 1400, it inputs the corrected fundus image data 802 into the learning model and re-executes the fundus image diagnosis (step S1501). Then, the AI ​​server 130 sends the re-diagnosis result Rij to the DB server 1400 (step S1502).

[0103] The DB server 1400 receives the re-diagnosis result Rij from the AI ​​server 130 (step S1512). The DB server 1400 then determines whether the re-diagnosis result Rij matches the diagnosis result Rij of the fundus image data 102 before correction (step S1002). If they do not match (step S1002: No), the process returns to step S702, and the DB server 1400 corrects the fundus image data 102 again (step S702). If they do match (step S1002: Yes), the DB server 1400 saves the corrected fundus image data 802 and the re-diagnosis result Rij as training data in the training DB 131 (step S704). This completes the processing of the fundus image data 102 (step S1433).

[0104] <Example of a functional configuration of an information processing system> Figure 16 is a block diagram showing an example of the functional configuration of an information processing system according to Embodiment 2. In Figure 16, the data control unit 1600 of the AI ​​server 130 has an erase unit 1342 and a third transmission unit 1601, and the DB server 1400 has a learning DB 131 and a storage unit 1602. The third transmission unit 1601 transmits the corrected fundus image data 802 and the diagnostic result Rij to the DB server 1400 if the secondary use flag is OK. The storage unit 1602 stores the corrected fundus image data 802 and the diagnostic result Rij from the third transmission unit 1601 as learning data in the learning data set of the learning DB 131.

[0105] Thus, in Example 2 as well, by ensuring the identity of the diagnostic result Rij before and after correction while respecting the privacy of patient i, it is possible to reuse the corrected fundus image data 802, which has no change in the diagnostic result Rij. Furthermore, this makes it possible to suppress the decrease in prediction accuracy by the learning model used for fundus image diagnosis by the AI ​​server 130 when the combination of the corrected fundus image data 802 and its diagnostic result Rij is used as training data.

[0106] It should be noted that the present invention is not limited to the above, and may be combined in any way. Furthermore, other embodiments that can be conceivable within the scope of the technical idea of ​​the present invention are also included in the scope of the present invention. [Explanation of symbols]

[0107] 100 Information processing system, 101 In-hospital system, 102 Fundus image data, 103 Consent information, 103a First consent information, 103b Second consent information, 104 Diagnostic data, 105 Diagnostic result data, 106 Anonymized diagnostic data, 107 Anonymized diagnostic result data, 111 Ophthalmic equipment, 112 Terminal, 113 In-hospital server, 114 Patient information DB, 120 Management server, 130 AI server, 131 Learning DB, 201 Processor, 202 Storage device, 400 Image acquisition device, 802 Corrected fundus image data, 1300 Information processing device, 1301 Acquisition unit, 1302 First transmission unit, 1303 First reception unit, 1311 Second reception unit, 1312 Image diagnosis unit, 1313 Second transmission unit, 1314 Data control unit, 1315 Learning unit, 1316 Modification section, 1341 Storage section, 1342 Deletion section, 1400 DB server, 1601 Third transmission section, 1602 Storage section, Dij Diagnostic information, Rij Diagnostic result

Claims

1. An information processing device that can communicate via a management server with an image diagnostic device that performs image diagnosis on a patient's eye image data using artificial intelligence, and for each of the patient, for each image diagnosis of the patient, stores as diagnostic information the following in a database: the eye image data, first consent information indicating permission or denial to transmit the eye image data to the management server, and second consent information indicating permission or denial to reuse the eye image data in the image diagnostic device, An acquisition unit that acquires the consent information for the first image diagnosis of the first eye image data of the patient from the database, If the first consent information is approved, the transmission unit transmits the first eye image data and the second consent information to the management server. A receiving unit that receives the image diagnostic results from the aforementioned image diagnostic device via the management server, The system includes a generation unit that, when new second eye image data is acquired from the ophthalmic device in the second image diagnosis following the first image diagnosis of the patient, registers the first consent information from the first image diagnosis as the first consent information for the second image diagnosis in the database along with the second eye image data, and, as a result of outputting a request to update the second consent information for the first image diagnosis, registers the new second consent information set by the operation input in the database as the consent information for the second image diagnosis, in association with the first consent information for the second image diagnosis. The acquisition unit is an information processing device that acquires consent information in the second image diagnosis.

2. An information processing apparatus according to claim 1, The transmission unit is an information processing device that, if the first consent information is not permitted, does not transmit the eye examination image data to the management server.

3. An information processing apparatus according to claim 1 or 2, The database stores patient information that identifies each patient, associated with the diagnostic information, The transmission unit is an information processing device that transmits the patient information to the management server.

4. The information processing apparatus according to claim 3, The receiving unit is an information processing device that receives the image diagnostic results and the patient information, which is the patient information that has been anonymized when transmitted to the image diagnostic device and then restored.

Images