Deep reinforcement learning method for artificial intelligence applied to visual field testing

Abstract

To provide a visual field testing apparatus capable of performing visual field testing with high accuracy over a short period of time.SOLUTION: A visual field testing apparatus 1 for testing a user's visual field comprises: a display 13 which displays targets; a perceptual information output part 50 which outputs perceptual information concerning whether or not the user perceives the targets displayed on the display 13; and a visual field testing part 60 which successively displays the targets on a predetermined coordinate on the display 13 and tests the targets on the basis of the perceptual information. The visual field testing part 60 comprises a learned model 65 learned by deep learning, that is, the learned model 65 in which the sensitivity of a target for each measurement point, the perceptual information of the target for each measurement point, and the number of measurements for each measurement point are inputted as input data, and from which the coordinate and sensitivity of the target which is a next measurement object are output as output data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a visual field testing device. [Background technology]

[0002] Visual field testing devices for testing the visual field have been provided in the past, and are disclosed, for example, in the following Patent Document 1. The visual field testing device disclosed in the following Patent Document 1 requires the user to input whether or not the visual target is visible using an input device such as a button.

[0003] However, some users may fail to input data because they are not familiar with operating the input device or are unable to operate the input device properly due to nervousness, etc., which can result in an inaccurate visual field test. In addition, the user must input whether or not they can actually see each of the optotypes that are displayed in sequence, which places a heavy burden on the user.

[0004] In response to this, Patent Documents 2 and 3 listed below disclose visual field testing devices that automatically detect the line of sight, which is the direction in which a user is looking, using a line of sight detection device, and automatically determine whether the user can see a visual target. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] WO2017-022757 publication [Patent Document 2] Japanese Patent Application Laid-Open No. 2011-161122 [Patent Document 3] Japanese Patent Publication No. 2020-141848 Summary of the Invention [Problem to be solved by the invention]

[0006] However, even with the gaze detection devices disclosed in Patent Documents 2 and 3, the user must continue to look at the sequentially displayed targets during the visual field test. In visual field tests that generally take several tens of minutes, the longer the test time, the greater the burden on the user.

[0007] The present invention has been made in view of the above-mentioned problems, and has as its object to provide a visual field testing device that can perform a visual field test with high accuracy in a short time. [Means for solving the problem]

[0009] To solve the above problems The learning method using deep reinforcement learning according to the present invention is a learning method using deep reinforcement learning of artificial intelligence applied to a visual field test in which a user's visual field is tested while sequentially displaying visual targets on a display, and includes the steps of: inputting, as input data, the sensitivity of the visual target at each measurement point, perceptual information regarding whether the user perceives the visual target at each measurement point, and the number of measurements at each measurement point; outputting, as output data, the coordinates and sensitivity of the visual target to be measured next; and performing a reaction simulation to calculate the perceptual information based on the output data. wherein the reaction simulation formula includes parameters of a false positive rate and a false negative rate, and is a formula that allows the reaction simulation to be performed while appropriately changing the false positive rate and the false negative rate. and, Based on the results of the reaction simulation, The reward is designed to increase as the number of measurements decreases and the difference between the retinal sensitivity based on the data set and the predicted sensitivity based on the output data decreases. The method is characterized in that the computer executes the steps of: [Effects of the Invention]

[0011] According to the present invention, by applying a trained model learned through deep learning to a visual field testing device, visual field testing can be performed with high accuracy in a short time. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram illustrating the configuration of a visual field testing device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing the display positions of optotypes measured in a visual field test according to an embodiment of the present invention. [Figure 3] FIG. 3 is a cycle flowchart showing the flow of processing during learning according to an embodiment of the present invention. [Figure 4] FIG. 4 is an image diagram that schematically shows the configuration of an agent according to an embodiment of the present invention. [Figure 5] FIG. 5 is a diagram showing the contents of output data of a trained model according to an embodiment of the present invention. [Figure 6] FIG. 6 is a cycle flowchart showing the flow of processing during operation according to an embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing the results of an effectiveness test according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] A visual field testing device 1 according to an embodiment of the present invention will be described in detail below with reference to the drawings. The visual field testing device 1 according to this embodiment is a device that efficiently performs visual field testing by utilizing artificial intelligence (AI), and is characterized by having a trained model (agent) 65 trained by deep reinforcement learning, sequentially displaying targets based on the coordinates and sensitivity (brightness) of the measurement targets output by the agent, and performing a visual field (retinal sensitivity) test with high accuracy in a short time.

[0014] The visual field testing device 1 comprises a head-mounted display (HMD) 10 and a control device 20, which is a computer connected to the HMD 10 via a communication cable. The HMD 10 is a device worn on the user's head during visual field testing, and comprises a display 13 that displays visual targets, a camera 15 that captures images of the user's eyes, and the like.

[0015] The HMD 10 has a gaze detection function that cooperates with a gaze detection unit 25 (described later) to automatically detect the gaze, which is the direction in which the eyes of the user wearing the HMD 10 are looking. The displays 13 and cameras 15 are installed for the right and left eyes, respectively.

[0016] The control device 20 includes an arithmetic device 70 such as a CPU (Central Processing Unit) for performing various calculations, and a storage device 75 such as an HDD (Hard Disc Drive) for storing various information and a RAM (Random Access Memory) used as a work area for calculation processing.

[0017] The storage device 75 includes a gaze log storage unit 76 that sequentially records gaze information detected by the gaze detection unit 25 described later, and an eye target storage unit 77 that records eye target information related to the eye targets to be displayed on the display 13.

[0018] In addition, the control device 20 functionally comprises a gaze detection unit 25, a visual target display unit 30, a visual recognition judgment unit 40, a perception (visibility) judgment unit 50, and a visual field testing unit 60, and these functions are realized by the calculation device 70 executing a predetermined program recorded in the memory device 75.

[0019] The gaze detection unit 25 detects the direction in which the user is looking, i.e., the gaze, based on the captured image of the user's eyes output from the camera 15. Specifically, the gaze detection unit 25 outputs two deviation angles (θ, φ) in polar coordinates (spherical coordinates) as gaze information (gaze angle) that indicates the gaze direction. This gaze detection is performed independently for the right eye and the left eye.

[0020] Based on the optotype information recorded in the optotype memory unit 77, the optotype display unit 30 displays optotypes of a predetermined size at predetermined coordinate positions on the display 13 in sequence with a predetermined sensitivity (brightness).

[0021] FIG. 2 shows the display positions (measurement points) of the optotypes measured in the visual field test according to this embodiment. In this embodiment, the visual field test is performed based on the Humphrey Perimeter Test - Central 24-2 program, and the visual field test is performed while optotypes are displayed at 54 points arranged at the intersections of a grid. The sensitivity (brightness) of each optotype is displayed in 41 stages from 0 to 40 dB. Of course, the visual field test method and the sensitivity level and range when displaying each optotype can be changed as appropriate.

[0022] The visual recognition determination unit 40 determines whether or not the user is actually looking at and visually recognizing the visual target, based on the visual line detected by the visual line detection unit 25 and the coordinates of the visual target displayed on the display 13.

[0023] Furthermore, the perception determination unit 50 determines whether a visual target located away from the user's gaze position is visible to the user, i.e., whether the user perceives it. After the visual target is displayed on the display 13, the perception determination unit 50 performs perception determination based on the trajectory and arrival time of the gaze from the gaze position at the time of display to the visual target.

[0024] The perception determination unit 50 functions as a perception information output unit that outputs the result of the perception determination, ie, whether or not the visual target was perceived, as perception information. Of course, the method of perception determination by the perception determination unit 50 can be changed as appropriate, and the perception information may also be the user's reaction time until recognition or reaction trajectory, which indirectly indicate whether or not the visual target was perceived.

[0025] The visual field testing unit 60 controls the gaze detection unit 25, the target display unit 30, the visual recognition assessment unit 40, and the perception assessment unit 50 to perform a visual field test. In this embodiment, the visual field testing unit 60 includes a deep reinforcement learning unit 63 for performing machine learning, a trained model (trained model) 65 that functions as an agent, and an examination operation unit 67 that operates the trained model 65 during the visual field test.

[0026] In this embodiment, the coordinates and sensitivity (brightness) of the targets to be sequentially displayed during the visual field test are determined and the visual field test is performed by operating a trained model 65 that has undergone machine learning by a deep reinforcement learning unit 63. As the neural network that constitutes the trained model 65, a Transformer model, which has an excellent ability to learn global data relationships, is used. Of course, neural networks other than Transformer may also be used.

[0027] The configuration of the visual field testing device 1 has been described above. Next, the processing flow when the deep reinforcement learning unit 63 learns the trained model 65 will be described. Fig. 3 is a cycle flowchart showing the processing flow during learning. Fig. 4 is an image diagram schematically showing the configuration of the agent.

[0028] During learning, first, in S1, the deep reinforcement learning unit 63 makes the agent observe the state of the environment. Specifically, as shown in Fig. 4, the states of all 54 measurement points are input as input data to the input layer of the learning model 65.

[0029] The following five pieces of information are input for each measurement point as input data: (1) Sensitivity of the target measured at the last time (t): 0 to 40 dB (2) Perceptual information of the visual target measured at the last time (t): -1, 0, 1 (3) Sensitivity of the target measured at the second-to-last time (t-1): 0-40 dB (4) Perceptual information of the visual target measured at the second-to-last time (t-1): -1, 0, 1 (5) The number of times measured t: 0 or more

[0030] The information on each measurement point is input to the agent in the same order each time, and the input order reflects the coordinate information of the measurement points. That is, in this embodiment, the coordinate information of the measurement points is embedded in the input data of the agent, and machine learning is performed taking into account the relationships with surrounding measurement points.

[0031] The perception information is information on whether or not the user, who is the subject, perceived the visual target, and in this embodiment, it is whether or not the user perceived (was able to see) the visual target when it was displayed. If the visual target was perceived, it is 1, and if the visual target was not perceived, it is -1. The perception information during learning uses information calculated by a reaction simulation, which will be described later.

[0032] When t = 0, the perceptual information for t and t-1 is initially set to 0, the measurement sensitivity for t-1 is initially set to 0, and the representative value of a data set relating to a past visual field test obtained separately is input as the measurement sensitivity for t. When t = 1, the perceptual information for t-1 is initially set to 0, and the representative value is input as the measurement sensitivity for t-1.

[0033] This representative value is a value calculated for each user's age and measurement coordinate, and the average value calculated for each user's age and measurement coordinate from the dataset is used. In this embodiment, the dataset used for machine learning is actual measurement data from 139 glaucoma patients who underwent visual field tests (HFA, central 24-2 program) more than 17 times per person over a nine-year period.

[0034] Next, in S2, the deep reinforcement learning unit 63 determines a measurement target based on the output data of the agent. Specifically, the deep reinforcement learning unit 63 causes the trained model 65 to calculate the input data of S1 and output output data, and determines a measurement target by sampling from the output data.

[0035] Fig. 5 is a diagram showing the contents of the output data output from the output layer of the trained model 65. As shown in the figure, the output data is data to which priorities determined by calculation are added to all combinations (2,214 ways) of measurement coordinates (54 points) and measurement sensitivity (41 levels).

[0036] In this way, the output data is a combination of 2,214 different coordinates, sensitivities, and priorities. Here, the priorities are values ​​between 0 and 1, and are set so that the sum of all 2,214 different priorities is 1.

[0037] Furthermore, in S2, the deep reinforcement learning unit 63 multiplies the priority of the output data by a random number and extracts one combination of measurement targets to be measured by sampling. This determines the coordinates and sensitivity of the measurement target to be measured next. Of course, multiple measurement targets may be extracted.

[0038] Next, in S3, the deep reinforcement learning unit 63 calculates perceptual information, which is information regarding whether or not the subject perceives the measurement target determined in S2, by reaction simulation. The reaction simulation is a simulation using a perceptual probability model, and the formula of the perceptual probability model p is the following formula (1).

number

[0039] In equation (1), x is the measured sensitivity, y is the retinal sensitivity, s is the standard deviation for each retinal sensitivity, and G(x, y, s) is the cumulative distribution function of the Gaussian distribution. The retinal sensitivity y uses the measured sensitivity of the above dataset, and the standard deviation s for each retinal sensitivity uses the standard deviation value defined in previous research.

[0040] Furthermore, fp is the false positive rate, and fn is the false negative rate, and in this embodiment, the perception probability p can be calculated by appropriately changing fp and fn. Generally, in an actual visual field test by a subject, there are false negative and false positive reactions, so by including parameters for the false positive rate, false negative rate, and false negative rate in the reaction simulation equation, it is possible to perform a reaction simulation that takes false negatives and false positives into account as appropriate, making it possible to perform deep reinforcement learning in an environment that is similar to an actual visual field test, and improving the reliability of the trained model 65.

[0041] The perception probability p calculated by the above formula (1) is a value between 0 and 1, but the deep reinforcement learning unit 63 multiplies it by a random number and outputs either 1, which indicates a state of perception, or −1, which indicates a state of non-perception, as the result of the reaction simulation. Of course, the formula and method used in the reaction simulation can be changed as appropriate.

[0042] Next, in S4, the deep reinforcement learning unit 63 acquires a reward based on the result of the reaction simulation in S3. Specifically, the reward is calculated using the following reward function formula (2).

number

[0043] In equation (2), y is the retinal sensitivity at each measurement point, as in equation (1), s is the predicted sensitivity at each measurement point, and t is the number of measurements at each measurement point. This reward function is designed so that the reward increases as the number of measurements decreases and the difference between the retinal sensitivity and the predicted sensitivity decreases.

[0044] Here, the predicted sensitivity is determined based on the measured sensitivity of the output data by the following steps (1) to (4). (1) If the measured sensitivity at the tth time is 0 dB and the sensor is not perceptible, the predicted sensitivity is 0 dB. (2) If the measured sensitivity at the tth time is 40 dB and it is perceptible, the predicted sensitivity is 40 dB. (3) When the perceptual information at the t-th time and the t-1 time are the same, or when it is the first measurement (t=1), the predicted sensitivity is the t-th time measurement sensitivity. (4) In cases other than (1) to (3), the predicted sensitivity is the average of the measured sensitivity at the tth and t-1th times.

[0045] Next, in S5, the deep reinforcement learning unit 63 performs deep reinforcement learning so as to maximize the reward calculated in S4, and updates and optimizes the weighting coefficients of the trained model 65.

[0046] Next, in S6, the deep reinforcement learning unit 63 determines whether or not the convergence condition is met. The convergence condition is that measurement has been completed at least once at all measurement points, and the change in the predicted sensitivity is 3 dB or less for two consecutive measurements, at which point convergence is determined and the machine learning is terminated. On the other hand, if at least one of these two conditions is not met, the process returns to S1, and steps S1 to S6 are repeated to continue the machine learning.

[0047] As described above, during deep reinforcement learning by the deep reinforcement learning unit 63, the cycle of S1 to S6 is repeated in order, and the weighting coefficients (parameters) of the trained model 65 are updated and strengthened in order.

[0048] Next, we will explain the processing flow when a visual field examination is performed using the visual field examination device 1, that is, the processing flow when the trained model 65 is operated by the examination operation unit 67. Figure 6 is a cycle flowchart showing the processing flow during operation.

[0049] During operation, first, in S11, the inspection operation unit 67 causes the agent to observe the state of the environment. Specifically, as in the above-mentioned learning, as shown in Fig. 4, the states of all 54 measurement points are input as input data to the input layer of the trained model 65.

[0050] Here, the input data during operation differs in perceptual information from the input data during learning. During learning, perceptual information calculated by simulating the above formula (1) is input as perceptual information, but during operation, the user's reaction in an actual test is input as perceptual information. As in learning, if the user, who is the subject, perceives the optotype, 1 is input, and if not, -1 is input. Furthermore, when t=0, 0 is input as the initial value.

[0051] Next, in S12, the inspection operation unit 67 determines a measurement target. Specifically, the inspection operation unit 67 causes the trained model 65 to calculate the input data of S11 and output output data, and also determines a measurement target by sampling from the output data.

[0052] 5, the output data of the trained model 65 is data to which priorities determined by calculation are assigned to the combinations of all measurement points (54 points) and measurement sensitivities (41 levels). In response to this, the inspection operation unit 67 determines the measurement coordinates and measurement sensitivities of the measurement target by sampling, as in S2 above.

[0053] Next, in S13, the test operation unit 67 performs measurement on the user, who is the subject, using the measurement target determined in S12. Specifically, the test operation unit 67 displays the target of the determined measurement sensitivity at the position of the determined measurement coordinates on the display 13, and then determines whether or not the user perceives the target based on the output of the perception determination unit 50. The test operation unit 67 outputs 1 as perception information if the user perceives the target, and -1 if the user does not perceive the target.

[0054] Next, in S14, the inspection operation unit 67 performs a convergence determination to see if the convergence conditions are met. This convergence determination is performed under the same conditions as the convergence determination during learning (S6) described above. If at least one measurement has been completed at all 54 measurement points and the change in predicted sensitivity is 3 dB or less for two consecutive measurements, convergence is determined and the visual field measurement is terminated. On the other hand, if the convergence conditions are not met, the process returns to S11, and S11 to S14 are repeated to continue the visual field measurement.

[0055] If the test operation unit 67 determines that the results have converged, it terminates the visual field test, using the predicted sensitivity of each measurement point as the measurement result of the user's retinal sensitivity. When the visual field measurement is completed, the visual field test unit 60 outputs the visual field test results to an external display or the like. The visual field test results are displayed as a diagram showing the display positions of the optotypes shown in Figure 2, with the retinal sensitivity of all measurement points colored.

[0056] In this way, by applying machine learning to the trained model 65 used during visual field testing using the deep reinforcement learning unit 63, the trained model 65 can be strengthened so that it can output more appropriate measurement point coordinates and sensitivity, reducing the number of times the measurement target needs to be displayed and making it possible to perform visual field testing more accurately in a shorter time.

[0057] Next, we will explain the effectiveness test of the visual field testing device 1. Fig. 7 is a diagram showing the results of the effectiveness test of the visual field testing device 1. In this effectiveness test, the number of measurements (number of displays) of the optotype during visual field testing using the visual field testing device 1 according to this embodiment and the accuracy of measurement sensitivity were compared with the results of visual field testing using the existing ZEST (Zippy Estimated Sequential Testing).

[0058] In this effectiveness test, a trained model 65 that had been trained for 3,000 epochs (1 epoch is for 100 subjects) was used, and the number of times and accuracy were verified by simulation using 5-way cross-validation.

[0059] Regarding the accuracy of measurement sensitivity, the results of visual field tests using the visual field testing device 1 and ZEST were evaluated by comparing them with the retinal sensitivity of the above-mentioned data set using the mean absolute error per test. In this effectiveness test, similar to the above-mentioned data set, the test was conducted on subjects in three stages of glaucoma: early stage, moderate stage, and advanced stage. Figure 7 shows the average values ​​for all stages.

[0060] In addition, in this effectiveness test, tests were conducted for false positives (fp) and false negatives (fn) in four cases: fp=fn=0%, fp=fn=5%, fp=fn=10%, and fp=fn=15%. It is said that in visual field tests performed by average users, fp and fn are each between 5 and 10%.

[0061] As shown in Figure 7, according to this embodiment, regardless of fp and fn, the accuracy of the measurement sensitivity by the visual field testing device 1 is approximately the same as that of ZEST, and the number of measurements is approximately one-third of the number of measurements by ZEST.

[0062] That is, visual field testing using the visual field testing device 1 can be performed with approximately the same accuracy in one-third the testing time compared to ZEST, and highly accurate visual field testing can be performed in a short time.

[0063] The above describes the visual field testing device 1 according to this embodiment. According to this embodiment, by updating the weighting coefficients of the trained model 65 through deep reinforcement learning and training it appropriately, the trained model 65 can more appropriately determine the coordinates and sensitivity of the measurement target, and more accurate visual field testing can be performed in fewer attempts.

[0064] Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications are possible within the scope of the present invention. For example, in the above embodiments, the learning model is learned by deep reinforcement learning, but the learning method of the learning model is not limited to deep reinforcement learning, and learning may be performed by deep learning with teacher data.

[0065] In this case, a data set including input data (sensitivity of measurement points, perception information, and number of measurements) and output data (measurement coordinates and sensitivity) may be used as training data.

[0066] Furthermore, in the above embodiment, a visual field testing device that automatically determines a user's perceptual information based on gaze information detected by a gaze detection unit was used as an example. However, the present invention can also be applied to a visual field testing device in which the user manually inputs perceptual information indicating whether or not they were able to perceive something into a perceptual information output unit such as a switch. [Explanation of symbols]

[0067] 1. Visual field testing device 10 HMD 13. Display 15 Camera 20 Control device 25 Gaze detection unit 30 Visual target display section 40 Visibility determination section 50 Perception Judgment Section 60 Visual Field Testing Section 63 Deep Reinforcement Learning Department 65 Pre-trained model (training model) 67 Inspection Operations Department 70 Arithmetic unit 75 Storage device 76 Gaze log memory unit 77 Visual target storage unit

Claims

[Claim 1] A learning method using deep reinforcement learning of artificial intelligence applied to a visual field test that tests a user's visual field while sequentially displaying visual targets on a display, inputting, as input data, the sensitivity of the visual target at each measurement point, perception information regarding whether the user perceives the visual target at each measurement point, and the number of measurements at each measurement point; a step of outputting, as output data, the coordinates and sensitivity of the target to be next measured; a step of performing a reaction simulation to calculate the perceptual information based on the output data, wherein an equation for the reaction simulation includes parameters of a false positive rate and a false negative rate, and the reaction simulation can be performed while appropriately changing the false positive rate and the false negative rate; Based on the results of the reaction simulation, a step of performing deep reinforcement learning using a reward function designed so that the reward increases as the number of measurements decreases and the difference between the retinal sensitivity based on the data set and the predicted sensitivity based on the output data decreases; A learning method using deep reinforcement learning, characterized by having a computer execute the following.

Images