Brain evaluation value prediction method and brain evaluation value prediction system

The method uses gaze measurement and interpolation to predict brain diseases and disorders efficiently, addressing the limitations of traditional neuropsychological tests by providing accurate brain evaluation values.

WO2026239733A1PCT designated stage Publication Date: 2026-11-19KYOTO UNIV +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2026/008947
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-12
Filing Date
2026-03-09
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Current neuropsychological tests for cognitive dysfunctions such as dementia and mild cognitive impairment are time-consuming, require trained examiners, and impose mental stress on subjects, necessitating a more efficient and less invasive method for brain evaluation.

Method used

A method using a gaze measurement device to acquire gaze parameters from subjects viewing test content, which are input into a pre-trained prediction model to predict brain evaluation values, incorporating interpolation to handle missing gaze data due to blinking.

Benefits of technology

Enables accurate prediction of brain diseases and disorders like dementia and mild cognitive impairment with improved accuracy by using gaze parameters, reducing the need for traditional testing methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2026008947_19112026_PF_FP_ABST
    Figure JP2026008947_19112026_PF_FP_ABST
Patent Text Reader

Abstract

This brain evaluation value prediction method comprises: a step for acquiring a line-of-sight parameter S based on measurement data D2 output from a line-of-sight measurement device 3 that measures the line of sight of a subject viewing examination content that includes a moving image or a plurality of still images; and a step for acquiring a brain evaluation value E by inputting the acquired line-of-sight parameter S into a prediction model 20 that has been trained in advance to output, in response to an input of a line-of-sight parameter S, a brain evaluation value E pertaining to a disease in the brain, characteristics of the brain, or the presence / absence of a disorder of the brain. The line-of-sight parameter S includes a plurality of data sets DS measured every time a prescribed time elapses while the examination content is played back. Each of the plurality of data sets DS includes a measurement time point and a line-of-sight position in the examination content.
Need to check novelty before this filing date? Find Prior Art

Description

Brain evaluation value prediction method, and brain evaluation value prediction system

[0001] The present disclosure relates to a technology related to brain evaluation values concerning the presence or absence of brain diseases, traits, or disorders.

[0002] Current general initial diagnosis (screening) for cognitive dysfunctions such as dementia, mild dementia, and mild cognitive impairment (MCI) relies on neuropsychological test methods such as the MMSE (Mini-Mental State Exam). The MMSE is a cognitive function evaluation method in the form of an examiner asking questions to a subject, and has points for improvement, such as taking a long time to administer, requiring a trained examiner for accurate evaluation, and imposing significant mental stress on the subject due to being forced to provide responses.

[0003] Patent Document 1 discloses an apparatus that shows a diagnostic video to a subject, detects the subject's viewpoint, and diagnoses the possibility of cognitive function decline.

[0004] International Publication No. 2019 / 098172

[0005] There is a demand for a new technology capable of obtaining brain evaluation values concerning the presence or absence of brain diseases, traits, or disorders including mild dementia and mild cognitive impairment (MCI).

[0006] The present disclosure provides a new technology capable of obtaining brain evaluation values concerning the presence or absence of brain diseases, traits, or disorders.

[0007] The brain evaluation value prediction method of the present disclosure includes a step of acquiring gaze parameters based on measurement data output from a gaze measurement device that measures the gaze of a subject viewing test content including a moving image or a plurality of still images, and a step of inputting the acquired gaze parameters into a prediction model that has been pre-trained to output a brain evaluation value concerning the presence or absence of a brain disease, trait, or disorder with the gaze parameters as an input, to obtain the brain evaluation value. The gaze parameters include a plurality of datasets measured every elapse of a predetermined time during playback of the test content, and each of the plurality of datasets includes a measurement time point and a gaze position in the test content.

[0008] This is a block diagram of the prediction model generation system and brain evaluation value prediction system of the first embodiment. This is a block diagram of the prediction model generation system and brain evaluation value prediction system of the second embodiment. This is a graph showing the relationship between the predicted probability and the actual positive rate for 39 test data after training the prediction model of the first example.

[0009] [First Embodiment] Hereinafter, a first embodiment of the present disclosure will be described with reference to the drawings. Figure 1 is a block diagram showing the prediction model generation system 1 and the brain evaluation value prediction system 2 of the first embodiment.

[0010] <Brain Evaluation Score Prediction System 2> As shown in Figure 1, the brain evaluation score prediction system 2 predicts the brain evaluation score E based on the gaze parameter S. The brain evaluation score E is an evaluation score (score) regarding the presence or absence of brain diseases, characteristics, or disorders, including mild dementia and mild cognitive impairment (MCI). The brain evaluation score prediction system 2 uses a prediction model 20 that has been pre-generated by machine learning using the prediction model generation system 1.

[0011] The brain evaluation value prediction system 2 includes a prediction unit 21, a measurement data acquisition unit 22, an interpolation unit 23, and a gaze parameter acquisition unit 24. The prediction model generation system 1 includes a training data acquisition unit 10, training data D1, a learning unit 11, a measurement data acquisition unit 12, and an interpolation unit 13. The training data D1 is stored in memory 1a. The training data acquisition unit 10, the learning unit 11, the measurement data acquisition unit 12, and the interpolation unit 13 are implemented by processor 1b. The prediction unit 21, the measurement data acquisition unit 22, the interpolation unit 23, and the gaze parameter acquisition unit 24 are implemented by processor 2b. In this embodiment, processors 1b and 2b in one device implement each unit, but this is not limited to this. For example, each process may be distributed using a network, and multiple processors 1b and 2b may be configured to execute the processing of each unit. That is, one or more processors 1b and 2b execute the processing.

[0012] The gaze parameters S input to the prediction model 20 are data based on measurement data D2 output from the gaze measurement device 3, which measures the gaze of the subject viewing the inspection content. The inspection content includes moving images or multiple still images. The inspection content in this embodiment does not include instructions to the subject in the form of text or voice.

[0013] The eye-tracking device 3 of this embodiment is head-mounted and can be worn on the head of a subject. The eye-tracking device 3 includes a left-eye camera and a right-eye camera capable of detecting the subject's gaze, and a display capable of displaying the examination content shown to the subject. The measurement data D2 output by the eye-tracking device 3 includes a plurality of sampling data M measured at predetermined time intervals during the playback of the examination content. Each sampling data M includes the gaze position (X coordinate, Y coordinate) on the examination content (display). In this embodiment, the gaze position of the right eye is used, but it is not limited to this; the gaze position of the left eye may be used, or the gaze positions of both eyes may be used. An example of a predetermined time (sampling period) is 1 / 30 second (30 fps), but it is not limited to this. For example, if the playback length of the examination content is 3 minutes and 40 seconds and the sampling period is 30 fps, the total number of sampling data M will be 220 seconds × 30 fps = 6600. The sampling data M may include, for example, information indicating the measurement time (frame number, timestamp [ms]). In other words, multiple sample data M constitute time-series data containing 6600 gaze positions.

[0014] As shown in Figure 1, the gaze parameter S input to the prediction model 20 includes multiple datasets DS measured at predetermined time intervals during playback of the inspection content. Each of the multiple datasets DS corresponds to the sampling data M in the measurement data D2 and corresponds to each measurement point. The number of datasets DS is 6600, which is the same as the number of sampling data M in the measurement data D2. Each of the multiple datasets DS includes the measurement point (at least one of the frame number or timestamp [ms]) and the gaze position in the inspection content at the measurement point (at least one of the X coordinate or Y coordinate). All of the multiple datasets DS constituting the gaze parameter S include the gaze position in the inspection content.

[0015] During playback of the test content, the subject blinks. While blinking, the eye-tracking device 3 cannot detect the gaze position. As a result, the gaze position (X, Y coordinates) in the measurement data D2 is randomly missing. As shown in Figure 1, the multiple sampling data M that make up the measurement data D2 include multiple Type 1 sampling data M1 that include the gaze position in the test content, and multiple Type 2 sampling data M2 that do not include the gaze position in the test content due to blinking. Type 1 sampling data M1 is data measured when the subject is not blinking. Type 2 sampling data M2 is data measured when the subject is blinking. Because Type 2 sampling data M2 is mainly due to blinking, it occurs continuously. The presence of Type 2 sampling data M2 that does not include the gaze position results in missing data in the time-series data of the gaze position, which may affect the prediction accuracy of the brain evaluation value E by the prediction model 20.

[0016] Therefore, to address missing data on gaze position, the brain evaluation value prediction system 2 has an interpolation unit 23. The interpolation unit 23 is capable of performing interpolation processing. The interpolation processing is the process of interpolating gaze positions in test content not included in the second type sampling data M2 based on at least a portion of a plurality of first type sampling data M1. Various methods can be employed in the interpolation processing as long as they can interpolate the gaze positions of the second type sampling data M2. In this embodiment, the first processing is performed for consecutive second type sampling data M2 of less than a predetermined number of frames (e.g., 10 frames). The second processing is performed for consecutive second type sampling data M2 of a predetermined number of frames or more. The third processing is performed to perform a correction based on a state-space model on the interpolated values ​​interpolated in the first or second processing.

[0017] The first process is linear interpolation. This is based on the idea that the gaze position moves along a straight line over a short period of time less than a predetermined number of frames. The gaze position of each of the second type sampling data M2 groups to be interpolated is interpolated by assuming that it lies on a straight line connecting the gaze position of the first type sampling data M1 that is in front of the second type sampling data M2 group to be interpolated and the gaze position of a certain first type sampling data M1 that is behind the second type sampling data M2 group to be interpolated.

[0018] The second process is the hotdeck method. The hotdeck method is a technique for interpolating missing data between measurement data D2s that have similar overall gaze position patterns. A specific example is briefly explained. Several other measurement data D2s besides the measurement data D2 to be interpolated are stored in advance. The interpolation unit 23 calculates the similarity between the measurement data D2 to be interpolated and the other several measurement data D2s. The interpolation unit 23 uses the gaze position of the i-frame (same time point) of the other measurement data D2 that has the highest similarity to the measurement data D2 to be interpolated, as the gaze position of the second type sampling data M2 at the i-frame (same time point) in the measurement data D2 to be interpolated. The similarity is a value based on the cumulative difference of the gaze positions at the same time point.

[0019] The third process corrects the interpolated measurement data D2, which was interpolated in the first or second process, to a better-fitting value. The state of the i-th frame of the measurement data D2 to be corrected is predicted from the data observed before the i-th frame, and the observed value in the i-th frame is predicted from the predicted state. When new data (including the interpolated value) is observed in the i-th frame, the current state is corrected based on the error with the model's predicted value. This is repeated sequentially, and by iteratively repeating the prediction and correction of the state and predicted value, the interpolated value is corrected to a value that fits the pattern of the actually observed data of the measurement data D2 to be corrected.

[0020] As shown in Figure 1, the interpolated measurement data D3, as a result of the interpolation unit 23, becomes first-type sampling data M1, which includes the line of sight position, for all the sampling data M that make up the interpolated measurement data D3.

[0021] The gaze parameter acquisition unit 24 shown in Figure 1 generates and acquires gaze parameters S based on the interpolated measurement data D3. The gaze parameters S in the first example include a timestamp [ms] indicating the measurement time, a frame number indicating the measurement time, an X coordinate indicating the gaze position at the measurement time, and whether or not blinking occurred at the measurement time [bool]. The X coordinate is the horizontal axis coordinate of the display of the gaze measurement device 3, and is obtained by dividing the X coordinate (pixel) of the gaze position by the size of the X axis of the display. The presence or absence of blinking can be set to 0 or 1 depending on the presence or absence of the gaze position (coordinate), but is not limited to this. For example, it may be set to 0 if the eye is closed in the image of the left eye camera or the right eye camera, and to 1 if the eye is open.

[0022] The gaze parameter acquisition unit 24 can generate and acquire gaze parameters S for the second to fifth examples, in addition to the gaze parameter S for the first example. Second example: Includes a timestamp indicating the measurement time and an X-coordinate indicating the gaze position. Third example: Includes a frame number indicating the measurement time and an X-coordinate indicating the gaze position. Fourth example: Includes a timestamp indicating the measurement time, an X-coordinate indicating the gaze position, and whether or not blinking occurred. Fifth example: Includes a frame number indicating the measurement time, an X-coordinate indicating the gaze position, and whether or not blinking occurred.

[0023] The prediction unit 21 shown in Figure 1 takes gaze parameters S as input to the prediction model 20 generated by the prediction model generation system 1 and obtains a brain evaluation value E. The prediction model 20 is a model that has been pre-trained to output a brain evaluation value E using gaze parameters S as input. The prediction model 20 in this embodiment uses HIVE-COTE 2.0, but is not limited to this. HIVE-COTE 2.0 is a hierarchical voting algorithm using a heterogeneous meta-ensemble composed of multiple state-of-the-art algorithms in time series classification. In this embodiment, a brain evaluation value E of 0 indicates that there is no brain disease, characteristic, or disorder, and a brain evaluation value E of 1 indicates that there is a brain disease, characteristic, or disorder. The brain evaluation value E output by the prediction model 20 is, in principle, a number that includes a decimal between 0 and 1. If the value output by the prediction model 20 exceeds the threshold of 0.5, the brain evaluation value E output by the prediction unit 21 is set to 1. If the value output by the prediction model 20 does not exceed the threshold of 0.5, the brain evaluation value E output by the prediction unit 21 is set to 0.

[0024] <Predictive Model Generation System 1> As shown in Figure 1, the predictive model generation system 1 constructs a predictive model 20 using machine learning with training data D1. First, in order to acquire (generate) the training data D1, the examination content is viewed by multiple subjects (N) whose presence or absence of brain diseases, characteristics, or disabilities is known. The measurement data acquisition unit 12 acquires measurement data D2 from the eye-tracking device 3 for multiple subjects (N) and stores it in memory 1a. Since the eye-gaze position is missing from the measurement data D2 due to blinking, the interpolation unit 13 of the predictive model generation system 1 interpolates the eye-gaze position using the same processing as the interpolation unit 23 of the brain evaluation value prediction system 2. The training data D1 shown in Figure 1 consists of eye-gaze parameters S (S) for multiple subjects (N) that match the format of the eye-gaze parameters S of the first to fifth examples above. 1 ~S N ) and the associated brain evaluation value E (E 1 ~E N ) has. Brain evaluation value E (E) in training data D1 1 ~E N) is a value based on the diagnosis of a neurologist or psychiatrist. The learning unit 11 of the predictive model generation system 1 trains the predictive model 20 using the training data D1 to obtain a trained predictive model 20.

[0025] <Performance Evaluation> For training data (training data D1), measurement data D2 from 57 patients with cognitive impairment and 96 control patients was used. For test data, measurement data D2 from 15 patients with cognitive impairment and 24 control patients was used. Performance evaluation was performed for each of the gaze parameters S in the above examples 1 to 5. The results are as follows: Average accuracy rate by stratified 3-fold CV at the time of model training; accuracy rate for unknown data Example 1 (X, timestamp, frame, blink): 77%; 69% Example 2 (X, timestamp): 77%; 59% Example 3 (X, frame): 75%; 64% Example 4 (X, timestamp, blink): 76%; 69% Example 5 (X, frame, blink): 75%; 67%

[0026] In all of the above examples 1 through 5, the accuracy rate for unknown data was 59%, demonstrating that the brain evaluation value E could be predicted. Furthermore, in examples 1, 4, and 5, where the presence or absence of blinking was added as a gaze parameter S (explanatory variable of the prediction model 20), the accuracy rate for unknown data improved compared to data without blinking (examples 2 and 3), demonstrating the usefulness of blinking. In addition, it is possible to predict the brain evaluation value E using time-series data of gaze position without converting gaze position to another feature. Therefore, it is possible to avoid the risk of selecting a feature that is not necessarily optimal by converting it to another feature based on gaze position (e.g., attention time, gaze movement distance, gaze movement speed, etc.), demonstrating the effectiveness of the method disclosed herein.

[0027] Figure 3 is a graph showing the relationship between the predicted probability and the actual positive rate for 39 test data after training the first example prediction model 20. The predicted probability, which is the output of the prediction model 20, is plotted in 0.1 increments, with the actual positive rate of the corresponding sample group in each increment on the vertical axis and the model's predicted positive probability on the horizontal axis. The 0.1 increments represent half-open intervals, which are represented by "[" indicating a closed interval including the endpoint and ")" indicating an open interval not including the endpoint. Each point in Figure 3 represents the actual positive rate vs. the predicted positive probability in each increment. The bar plot in Figure 3 represents the number of people in each increment. As shown in Figure 3, the actual positive rate increases as the model's predicted positive probability increases. Furthermore, all samples judged as positive in the graph shown in Figure 3 (those with a predicted positive probability exceeding 0.5) were actually positive. From this, it can be said that the brain evaluation value E obtained by the prediction model 20 in this disclosure is useful.

[0028] [1] As in the above embodiment, the brain evaluation value prediction method is a method executed by one or more processors 2b, and includes the steps of: acquiring gaze parameters S based on measurement data D2 output from a gaze measurement device 3 that measures the gaze of a subject viewing examination content including moving images or multiple still images; and obtaining a brain evaluation value E by inputting the acquired gaze parameters S to a prediction model 20 that has been pre-trained to output a brain evaluation value E regarding the presence or absence of brain disease, characteristics, or impairment, with the gaze parameters S as input, wherein the gaze parameters S may include a plurality of datasets DS measured at predetermined time intervals during playback of the examination content, and each of the plurality of datasets DS may include the measurement time and the gaze position in the examination content. With this configuration, the brain evaluation value E can be predicted by inputting a plurality of datasets DS (time-series data of gaze positions) including the gaze position in the gaze content to the prediction model 20.

[0029] [2] The brain evaluation value prediction method described in [1] above, wherein each of the multiple datasets DS may include whether or not blinking occurred at the time of measurement. By including whether or not blinking occurred at the time of measurement in the features (explanatory variables) input to the prediction model 20, the prediction accuracy can be improved compared to the case where the presence or absence of blinking at the time of measurement is not included in the features.

[0030] [3] A brain evaluation value prediction method according to [1] or [2] above, wherein the measurement data D2 includes a plurality of sampling data M measured at predetermined time intervals during playback of the test content, the plurality of sampling data M includes a plurality of first type sampling data M1 including the gaze position in the test content, and second type sampling data M2 that does not include the gaze position in the test content due to the subject blinking, and the method includes a step of performing interpolation processing to interpolate the gaze position in the test content that is not included in the second type sampling data M2 based on at least a portion of the plurality of first type sampling data M1. In this way, while it is not possible to appropriately predict the brain evaluation value E in data where the gaze position is missing due to blinking, by performing interpolation processing to interpolate the gaze position, it becomes possible to appropriately predict the brain evaluation value E even for test content where it is unknown when the subject will blink.

[0031] [4] As in the above embodiment, the brain evaluation value prediction system 2 may include one or more processors 2b that perform the method described in any of [1] to [3] above.

[0032] [5] As in the above embodiment, the program may cause one or more processors 2b to execute the method described in any of [1] to [3] above.

[0033] The computer-readable temporary recording medium according to the above embodiment stores the above program.

[0034] Although embodiments of this disclosure have been described above with reference to the drawings, it should be understood that the specific configurations are not limited to these embodiments. The scope of this disclosure is indicated not only by the description of the embodiments above but also by the claims, and further includes all modifications within the meaning and scope equivalent to the claims.

[0035] (A) For example, in the first embodiment described above, the length of the inspection content and the sampling period (predetermined time) are examples and can be changed as appropriate.

[0036] (B) [Second Embodiment] In the first embodiment, the interpolation unit 23 is mounted on a device other than the eye-tracking device 3, but is not limited thereto. In the second embodiment, as shown in Figure 2, the interpolation unit 23 may be mounted on the eye-tracking device 3. In this case, the eye-tracking device 3 outputs the interpolated measurement data D3.

[0037] (C) In the above embodiment, the line of sight position in the line of sight parameter S input to the prediction model 20 is only the X coordinate, but is not limited to this. For example, it may be only the Y coordinate orthogonal to the X coordinate, or it may include both the X coordinate and the Y coordinate.

[0038] (D) In ​​the above embodiment, the prediction model 20 is HIVE-COTE 2.0, but is not limited thereto. It may be time series classification or time series regression.

[0039] (E) Regarding the interpolation process in the above embodiment, if there are no missing line-of-sight positions in the measurement data D2, the interpolation process does not need to be performed, and the interpolation units 13 and 23 can be omitted. The interpolation process in the above embodiment performs a first process, a second process, and a third process, but this can be changed as appropriate. For example, at least one of the first or second process may be performed and the third process may not be performed, or only the first process may be performed, or only the second process may be performed, or the first and second processes may be performed and the third process may not be performed.

[0040] The structures adopted in each of the above embodiments can be adopted in any other embodiment. The specific configuration of each part is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of this disclosure.

[0041] 2: Brain evaluation value prediction system 2b: Processor 3: Eye-tracking device 20: Prediction model D2: Measurement data DS: Dataset E: Brain evaluation value M: Sampling data M1: Type 1 sampling data M2: Type 2 sampling data S: Eye-tracking parameters

Claims

1. A method for predicting brain evaluation values, comprising the steps of: acquiring gaze parameters based on measurement data output from a gaze measuring device that measures the gaze of a subject viewing examination content including moving images or multiple still images; and inputting the acquired gaze parameters to a predictive model that has been pre-trained to output brain evaluation values ​​regarding the presence or absence of brain diseases, characteristics, or disorders, with the gaze parameters comprising a plurality of datasets measured at predetermined time intervals during playback of the examination content, and each of the plurality of datasets comprising the measurement time and the gaze position in the examination content.

2. The brain evaluation value prediction method according to claim 1, wherein each of the plurality of datasets includes whether or not blinking occurred at the time of measurement.

3. The brain evaluation value prediction method according to claim 1, wherein the measurement data includes a plurality of sampling data measured at predetermined time intervals during playback of the test content, the plurality of sampling data includes a plurality of first type sampling data including the gaze position in the test content and second type sampling data that does not include the gaze position in the test content due to the subject blinking, and the method includes the step of performing an interpolation process to interpolate the gaze position in the test content that is not included in the second type sampling data based on at least a portion of the plurality of first type sampling data.

4. A brain evaluation value prediction system comprising one or more processors that perform the method according to any one of claims 1 to 3.