Three-dimensional cognitive ability evaluation system, three-dimensional cognitive ability evaluation device, three-dimensional cognitive ability evaluation program, and three-dimensional cognitive ability evaluation method

Abstract

The present invention achieves an objective evaluation of three-dimensional cognitive ability. A three-dimensional cognitive ability evaluation system (100) is provided with an extraction unit (101b) and a determination unit (101e). The extraction unit (101b) extracts, from time-series data of a moving subject being measured, data of time intervals during which the subject being measured performs at least one of an action to react to a target object and an action for attempting to maintain the certain state of motion. The determination unit (101e) determines the three-dimensional cognitive ability of the subject being measured using at least one feature amount calculated from the time interval data. With the three-dimensional cognitive ability evaluation system (100), it is possible to achieve an objective evaluation of three-dimensional cognitive ability.

Description

Stereocognition ability evaluation system, stereocognition ability evaluation device, stereocognition ability evaluation program, and stereocognition ability evaluation method

[0001] The present disclosure relates to a stereocognition ability assessment system, a stereocognition ability assessment device, a stereocognition ability assessment program, and a stereocognition ability assessment method.

[0002] Conventionally, systems for assessing cognitive function of a subject based on the subject's responses have been known. For example, Japanese Patent Publication No. 6000968 (Patent Document 1) discloses a system using a portable touchscreen personal computing device as a system for assessing cognitive function using visual measurements. This system performs cognitive assessment of an individual based on the response speed to displayed cognitive assessment stimuli. In the cognitive assessment test, the reaction time from when a character appears on the display of the personal computing device to when the user responds by pressing a button is measured.

[0003] JP 2015-502238 A (Patent Document 2) discloses a system that provides a video game for mapping a subject's peripheral visual field, including a test in which the subject must find a visual stimulus presented for a short period of time. In this system, targets are displayed on a display, and the visual field defect caused by glaucoma is measured based on the user's response to the targets.

[0004] Japanese Patent No. 6000968 Special Publication No. 2015-502238 International Publication No. 2023 / 032806

[0005] Stereocognitive ability is an index that indicates the appropriateness of the subject's reaction to a target that is approaching the subject, and can be expressed, for example, as a sense of perspective. In recent years, it has been known that stereocognitive ability tends to decline as cognitive function declines in elderly people and others. Therefore, evaluating stereocognitive ability may have an effect similar to that of evaluating cognitive function.

[0006] However, in the system disclosed in Patent Document 1, the displayed object is moved, but the user's perspective relative to the object is not taken into consideration. Furthermore, in this system, the cognitive function measurement results are affected by various attributes such as the subject's concentration level or proficiency in operation, making it difficult to objectively evaluate the user's three-dimensional cognitive ability. Furthermore, in the system disclosed in Patent Document 2, the distance between the subject and the monitor is measured, but the subject's perspective relative to the displayed target is not taken into consideration.

[0007] The present disclosure has been made to solve the above-mentioned problems, and its purpose is to realize an objective evaluation of stereoscopic ability.

[0008] According to one aspect of the present disclosure, a system for assessing stereoscopic ability includes an extracting unit and a determining unit. The extracting unit extracts data for a time interval during which the subject is performing at least one of an action for reacting to a target and an action for maintaining a constant state of motion from time-series data of the subject. The determining unit determines the subject's stereoscopic ability using at least one feature calculated from the data for the time interval.

[0009] According to another aspect of the present disclosure, a stereocognitive ability assessment device includes an extraction unit, a determination unit, and a housing. The extraction unit extracts data for a time interval during which the subject is performing at least one of an action to react to a target and an action to maintain a constant state of motion from time-series data of the subject as he or she moves. The determination unit determines the subject's stereocognitive ability using at least one feature extracted from the data for that time interval. The housing includes a housing that houses the extraction unit and the determination unit.

[0010] A stereocognitive ability assessment program according to another aspect of the present disclosure is a program that, when executed by a computer, causes the computer to configure a stereocognitive ability assessment device for assessing the stereocognitive ability of a moving subject. The stereocognitive ability assessment device includes an extraction unit and a determination unit. The extraction unit extracts data from the time-series data of the subject, the data being included in a time interval during which the subject is performing at least one of an action to react to a target and an action to maintain a constant state of motion. The determination unit determines the stereocognitive ability using at least one feature extracted from the data included in the time interval.

[0011] A stereocognitive ability assessment method according to another aspect of the present disclosure is a method for assessing the stereocognitive ability of a moving subject, which includes the steps of extracting data from time-series data of the subject, the data including a time interval during which the subject is performing at least one of an action to react to a target and an action to maintain a constant state of motion, and determining the stereocognitive ability using at least one feature extracted from the data included in the time interval.

[0012] According to the stereoscopic cognitive ability assessment system, stereoscopic cognitive ability assessment device, stereoscopic cognitive ability assessment program, and stereoscopic cognitive ability assessment method disclosed herein, the stereoscopic cognitive ability of the subject can be determined using at least one feature calculated from data on the time interval during which the subject is performing at least one of the following actions: an action to react to a target; and an action to maintain a constant state of movement; thereby, an objective assessment of the subject's stereoscopic cognitive ability can be achieved.

[0013] 1 is a block diagram showing the configuration of a stereocognition ability assessment system according to embodiment 1. FIG. 2 is a diagram showing the hardware configuration of the stereocognition ability assessment system of FIG. 1. FIG. 3 is a functional block diagram showing the configuration of a stereocognition ability assessment function of the processor of FIG. 2. FIG. 4 is a functional block diagram showing the configuration of a machine learning function of the processor of FIG. 2. FIG. 5 is a plan view of a road on which automobiles can travel, viewed from the weight direction. FIG. 6 is a diagram showing the relationship between a feature amount related to the acceleration of a subject and the subject's spatial cognition score, and the relationship between a feature amount related to the curve radius of the subject and the subject's spatial cognition score. FIG. 7 is a diagram showing the relationship between a feature amount related to the curve radius of the subject and the subject's spatial cognition score, and the relationship between a feature amount related to the angular velocity of the subject and the subject's spatial cognition score. FIG. 8 is a diagram showing the relationship between a feature amount related to the speed of the subject and the subject's spatial cognition score, and the relationship between a feature amount related to the acceleration of the subject and the subject's spatial cognition score. FIG. 9 is a histogram comparing, for each of a plurality of subjects, the spatial cognition scores obtained by an existing stereocognition ability assessment method with the spatial cognition scores obtained by the stereocognition ability assessment system according to embodiment 1. FIG. 10 is a flowchart showing the processing flow in a stereocognition ability assessment method performed by a processor executing the stereocognition ability assessment program of FIG. 2. 15 is a block diagram showing the configuration of a stereocognition ability assessment system according to a modified example of embodiment 1. It is a diagram showing the hardware configuration of the stereocognition ability assessment system of FIG. 11. It is a graph showing the correspondence between the output values ​​(predicted values) of a trained random forest model and the actual values ​​of the feature quantities corresponding to the output values. It is a flowchart showing the processing flow of a stereocognition ability assessment method performed by a processor that executes a stereocognition ability assessment program according to embodiment 2. It is a diagram showing the correlation between the predicted value and the actual measured value of the spatial cognition score of a subject determined by a stereocognition ability assessment system according to embodiment 3 based on time-series data of the subject driving a car. It is a diagram showing the coefficient of determination for each of curve driving, sudden acceleration, sudden deceleration, sudden steering (lateral acceleration), and sudden steering (lateral angular velocity) of FIG. It is a diagram showing the correlation between the predicted value and the actual measured value of the spatial cognition score of a subject when the number of judgment sections is 1, 3, 5, 10, and 30.21 is a diagram showing the relationship between the number of judgment intervals and prediction accuracy (coefficient of determination) in FIG. 17. FIG. 22 is a diagram showing how the stereocognitive ability of subject Sb2 is measured by a measurement test using a driving simulation in a stereocognitive ability assessment system according to embodiment 4. FIG. 23 is a diagram showing an example of a screen displayed on the subject Sb2 via a head-mounted display in a driving simulation using the stereocognitive ability assessment system of FIG. 19. FIG. 24 is a perspective view of the appearance of the head-mounted display of FIG. 19. FIG. 25 is a diagram showing the hardware configuration of the head-mounted display of FIG. 21. FIG. 26 is a diagram showing the concept of a visual target used in measuring eye function. FIG. 27 is a diagram showing the relationship between visual target distance (the distance between the subject's eye and the visual target) and pupil diameter due to the pupillary short-distance reflex. FIG. 28 is a diagram showing the relationship between visual target distance and pupil position (convergence and divergence movement). FIG. 29 is a functional block diagram showing the configuration of the stereocognitive ability evaluation function of the processor of FIG. 24.

[0014] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals, and their description will not be repeated in principle.

[0015] [First Embodiment] FIG. 1 is a block diagram showing the configuration of a stereocognitive ability assessment system 100 according to the first embodiment. As shown in FIG. 1, the stereocognitive ability assessment system 100 includes an information processing device 110 (a stereocognitive ability assessment device), a terminal device 800 of a subject Sb1, and a measurement device 900 mounted on a vehicle Cs1 (a moving object). The vehicle Cs1 is operated by the subject Sb1 and moves along a road (track). The measurement device 900 may include, for example, a drive recorder of the vehicle Cs1 or a terminal device 800 (e.g., a smartphone, tablet, smartwatch, or smart glasses) carried by the subject Sb1 inside the vehicle Cs1. The information processing device 110, the terminal device 800, the vehicle Cs1, and the measurement device 900 are connected to one another via a network NW. The network NW may include, for example, the Internet, a local area network (LAN), or a cloud system. The information processing device 110 may be included in the cloud system. Note that the moving object is not limited to a vehicle. The moving object can be anything, as long as its position changes based on the subject's actions (e.g., operations), such as a motorbike, bicycle, airplane, drone, train, or ship, or it can even be the subject themselves.

[0016] The subject Sb1 is a subject whose stereocognitive ability is measured by the stereocognitive ability assessment system 100. The subject Sb1 rides in and drives a car Cs1. The measurement device 900 measures physical quantities (e.g., position, coordinates, velocity, angular velocity, acceleration, deviation of the subject's actual trajectory from the ideal trajectory, or operation amounts based on CAN (Controller Area Network) data) of the subject Sb1 moving with the car Cs1 at each sampling time and records the time-series data of the physical quantities. Note that the operation amounts based on the CAN data are the operation amounts of the subject Sb1 on an operating unit (e.g., at least one of the accelerator, brake, transmission, and steering) for operating the car Cs1. For example, the operation amounts of the subject Sb1 on the accelerator or brake may be used instead of the acceleration of the subject Sb1. Furthermore, the operation amounts of the subject Sb1 on the steering wheel may be used instead of the angular velocity of the subject Sb1.

[0017] The information processing device 110 acquires time-series data of the subject Sb1 from the measuring device 900. The information processing device 110 may create time-series data from physical quantities transmitted from the measuring device 900 at each sampling time. From the time-series data, the information processing device 110 extracts data (determination section data) included in a time interval during which the subject Sb1 is performing an action (curve driving) to react to a curve (target), which is a bend in the road. The information processing device 110 determines the subject Sb1's three-dimensional cognitive ability using at least one feature calculated from the determination section data. For example, the information processing device 110 uses a regression model trained by decision tree-based machine learning to determine the importance (e.g., priority or contribution) of each of multiple feature values ​​derived from the determination section data, and can use N feature values ​​(N is a natural number) in descending order of importance as inputs (explanatory variables) for determining the subject Sb1's three-dimensional cognitive ability (target variable). The information processing device 110 transmits the evaluation result of the stereoscopic ability to the terminal device 800 of the subject Sb1. The information processing device 110 may function as a web server that displays the evaluation result of the stereoscopic ability of the subject Sb1 on a browser running on the terminal device 800 or another device of the subject Sb1 (for example, a personal computer). The information processing device 110 includes, for example, a personal computer or a workstation. The information processing device 110 may be mounted on the automobile Cs1. Note that "response" means recognizing and responding to the proximity of a moving object.

[0018] The subject Sb1 can check the evaluation result by referring to the terminal device 800 wherever he or she is located as long as a connection between the terminal device 800 and the network NW can be established.

[0019] The behavior for which the information processing device 110 can evaluate the stereoscopic ability of the subject Sb1 may be any behavior in which the subject Sb1 reacts to a target, and is not limited to behavior in which the subject Sb1 reacts to a bend in the road. For example, the information processing device 110 may evaluate the stereoscopic ability of the subject Sb1 based on judgment data acquired while the subject Sb1 is performing an action in which the subject Sb1 reacts to a staircase (a target).

[0020] Fig. 2 is a diagram showing the hardware configuration of the stereoscopic perception ability assessment system 100 of Fig. 1. As shown in Fig. 2, the information processing device 110 includes a processor 101, a RAM (Random Access Memory) 102, a storage 103, a communication unit 104, a memory interface 105, and a housing Hs1. The housing Hs1 houses the processor 101, the RAM (Random Access Memory) 102, the storage 103, the communication unit 104, and the memory interface 105.

[0021] The processor 101 is a processing circuit for executing various functions that control the operation of the information processing device 110. The processor 101 includes a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) that operates an information device such as a computer.

[0022] The RAM 102 includes a volatile memory and is used as a work area when the processor 101 operates, a temporary data storage area, and the like.

[0023] The storage 103 includes a non-volatile memory such as a flash read-only memory (ROM) or a solid-state drive (SSD). The storage 103 stores computer programs and data referenced during execution of the programs. For example, the storage 103 stores an operating system (OS) program (not shown), a stereoscopic perception ability assessment program 103a, and a machine learning program 103b as the computer programs. The storage 103 also stores a machine learning model 103c and training data 103d as data referenced during execution of the computer programs.

[0024] The machine learning model 103c evaluates the stereocognitive ability of the subject Sb1 based on feature values ​​calculated using judgment interval data extracted from the time-series data of the subject Sb1. The machine learning model 103c may be a classification model that classifies the stereocognitive ability of the subject Sb1 into one of a plurality of predetermined levels, or may be a regression model that outputs a numerical value corresponding to the stereocognitive ability. The machine learning model 103c is referenced by the stereocognitive ability evaluation program 103a. The machine learning model 103c may include any model that corresponds to a machine learning algorithm based on regression analysis, such as a linear regression model, a decision tree, a random forest, a gradient boosting decision tree, a neural network, or a Light Gradient Boosting Machine (LightGBM). The learning data 103d is referenced by the machine learning program 103b. The machine learning program 103b executes, for example, linear regression, decision tree regression (e.g., CART (Classification and Regression Trees)), random forest regression, gradient boosting regression, or deep learning as a machine learning algorithm based on regression analysis corresponding to the machine learning model 103c.

[0025] The training data 103d includes time-series data (teacher data) of the subject, labeled with a label corresponding to the subject's stereoscopic ability. Examples of labels that can be attached to the time-series data include scores from other stereoscopic ability assessment systems, depth perception tests, mental rotation tasks, visual ability assessment tests, "seeing ability" comprehension tests, existing visuospatial cognition assessment batteries such as the Kaufman Assessment Battery for Children (K-ABC), visual perceptual skill tests, and motor-exclusion visual cognition tests, as well as the level of stereoscopic ability determined in advance from the driver's driving skills, occupation, age, driving history, and the like (information about the subject's attributes).

[0026] The communication unit 104 communicates with each of the terminal device 800, the automobile Cs1, and the measuring device 900 via the network NW. The communication unit 104 and the network NW may be connected by wired communication (e.g., Ethernet (registered trademark)) or wireless communication (e.g., Wi-Fi (registered trademark)).

[0027] The memory interface 105 enables access to an external storage medium of the information processing device 110. The memory interface 105 is compatible with a Secure Digital (SD) card and a Universal Serial Bus (USB).

[0028] The information processing device 110 may be connected to an input / output unit (not shown) that receives input from a user and displays to the user the processing results of the processor 101. The input / output unit includes, for example, a display, a touch panel, a keyboard, a speaker, a lamp, and a mouse.

[0029] The measuring device 900 includes a processor 901, a camera 902, a RAM 903, a communication unit 904, a storage 905, a memory interface 906, and a sensor unit 907. The processor 901, the RAM 903, the communication unit 904, and the memory interface 906 have substantially the same functions as the processor 101, the RAM 102, the communication unit 104, and the memory interface 105 of the information processing device 110, respectively, and therefore descriptions of the functions of the processor 901, the RAM 903, the communication unit 904, and the memory interface 906 will not be repeated.

[0030] The camera 902 generates image data. For example, if the measuring device 900 is a drive recorder, the camera 902 generates image data of a view from the driver's seat of the automobile Cs1 in the direction of travel of the automobile Cs1.

[0031] The sensor unit 907 measures physical quantities of the subject Sb1 traveling with the vehicle Cs1 at each sampling time and records time-series data 905a of the physical quantities. The sensor unit 907 includes, for example, a GPS (Global Positioning System) sensor, a speed sensor, an acceleration sensor, and a gyro sensor. The sensor unit 907 may acquire CAN data from the vehicle Cs1. The time-series data 905a is stored in the storage 905. The time-series data 905a is transmitted to the information processing device 110 by the communication unit 904. The sensor unit 907 may transmit the physical quantities measured at each sampling time to the information processing device 110 via the communication unit 904.

[0032] The automobile Cs1 may transmit data regarding the reaction of the subject Sb1 operating the automobile Cs1 to the information processing device 110. Specifically, the automobile Cs1 may transmit data regarding the reaction of the subject Sb1 to the information processing device 110 based on input information from the subject to the steering wheel, accelerator pedal, brake pedal, etc. (for example, the rotation angle of the steering wheel, the depression degree of the accelerator pedal, and the depression degree of the brake pedal).

[0033] The stereocognitive ability evaluation function of the stereocognitive ability evaluation system 100 is realized by the processor 101 reading out the stereocognitive ability evaluation program 103a stored in the storage 103 and executing it using the work area of ​​the RAM 102. By executing the stereocognitive ability evaluation program 103a, modules that realize various functions related to the stereocognitive ability evaluation are formed, and the modules perform operations that realize the functions.

[0034] Fig. 3 is a functional block diagram showing the configuration of the stereocognition ability evaluation function of the processor 101 in Fig. 2. In the stereocognition ability evaluation system 100, a stereocognition ability evaluation program 103a stored in the storage 103 is executed by the processor 101 to configure modules forming functional blocks, such as a section extraction unit 101b and a stereocognition ability determination unit 101e. Therefore, Fig. 3 shows functional blocks realized by the processor 101 and the stereocognition ability evaluation program 103a in Fig. 2 instead. These functional blocks will be described below.

[0035] The interval extraction unit 101b acquires time-series data of the subject Sb1 from the measurement device 900. The interval extraction unit 101b extracts, from the time-series data, data that satisfy a condition indicating that the subject is engaging in at least one of an action to react to a target and an action to maintain a certain state of motion as judgment interval data, and outputs the extracted data to the stereoscopic ability assessment unit 101e. The condition includes, for example, a condition that, for a predetermined time interval (e.g., 3 seconds), a change in angular velocity of 1 degree / second or more occurs continuously in a constant rotational direction (rightward (clockwise) or leftward (counterclockwise)) relative to the direction of travel, and the difference in angle between the start and end times of the time interval is 60 degrees or more. Alternatively, the condition may include a condition that, for a predetermined travel distance (e.g., 5 m), a change in angular velocity of 1 degree / second or more occurs continuously in a constant rotational direction relative to the direction of travel, and the difference in angle between the start and end points of the travel distance is 60 degrees or more.

[0036] The stereocognitive ability assessment unit 101e derives feature quantities from the assessment section data. These feature quantities include, for example, the speed, acceleration, angular velocity, and curve radius of the subject Sb1, the deviation of the subject's actual trajectory from the subject's ideal trajectory, the count of physical quantities that satisfy specific conditions among multiple physical quantities included in the assessment section, the proportion (occupancy rate) of the physical quantities that satisfy the conditions among the multiple physical quantities, and feature quantities related to at least one of the time intervals of the assessment section data. The stereocognitive ability assessment unit 101e uses the machine learning model 103c to assess the stereocognitive ability of the subject Sb1 from the feature quantities. The stereocognitive ability assessment unit 101e transmits the assessment result of the stereocognitive ability to the terminal device 800 of the subject Sb1 via the communication unit 104.

[0037] In the stereoscopic perception ability assessment system 100, the processor 101 executes the machine learning program 103b, and the information processing device 110 functions as a learning device that generates a trained machine learning model 103c.

[0038] Figure 4 is a functional block diagram showing the configuration of the machine learning function of the processor 101 in Figure 2. As shown in Figure 4, in the stereoscopic perception ability assessment system 100, modules forming functional blocks, namely, a section extraction unit 111b and a learning unit 101f, are configured by executing the machine learning program 103b by the processor 101. Therefore, in Figure 4, instead of the processor 101 and the machine learning program 103b in Figure 2, functional blocks realized by them are shown. These functional blocks will be described below.

[0039] The interval extraction unit 111b acquires labeled time-series data from the learning data 103d. Similar to the interval extraction unit 101b in Fig. 3, the interval extraction unit 111b extracts, from the time-series data, data that satisfies a condition indicating that the subject is performing an action to react to a target, as judgment interval data, and outputs the data together with the label to the learning unit 101f.

[0040] The learning unit 101f calculates feature amounts from the determination interval data acquired from the interval extraction unit 111b. The learning unit 101f performs machine learning on the machine learning model 103c using the feature amounts and the labels acquired from the interval extraction unit 111b, and sets the machine learning model 103c as a trained model. For example, the learning unit 101f performs supervised learning on the machine learning model 103c, in which the labels are used as the correct answers for the output of the machine learning model 103c.

[0041] FIG. 5 is a plan view of a road Rd on which the automobile Cs1 can travel, viewed from the gravity direction. In FIG. 5, the x-axis, y-axis, and z-axis are perpendicular to one another, and the negative direction of the z-axis corresponds to the direction of gravity. As shown in FIG. 5, the road Rd extends from a start point Ps to a destination point Pg and includes left curves Cv11 and Cv12 and right curves Cv21, Cv22, and Cv23. For example, if determination section data is extracted from time-series data acquired from the automobile Cs1 traveling on the road Rd using the conditions that an angular velocity change of 1 degree / second or more occurs continuously in a constant rotational direction relative to the traveling direction during a predetermined time interval (e.g., 3 seconds) and the difference in angle between the start time and the end time of the time interval is 60 degrees or more, data while the automobile Cs1 is traveling around each of the left curves Cv11 and Cv12 and the right curves Cv21, Cv22, and Cv23 is extracted from the time-series data as the determination section data. Even if the conditions are used that there is a continuous change in angular velocity of 1 degree / second or more in a constant rotational direction relative to the direction of travel for a predetermined travel distance (for example, 5 m), and the difference between the angle at the start point and the angle at the end point of the travel distance is 60 degrees or more, data while the car Cs1 is traveling around each of the left curves Cv11, Cv12 and the right curves Cv21, Cv22, Cv23 can be extracted from the time series data as judgment section data.

[0042] 6 to 8, examples of the subject's feature values ​​calculated from the judgment section data and the relationship between the feature values ​​and an index (spatial cognition score) representing the subject's spatial cognition ability evaluated by an existing stereoscopic cognition ability evaluation method (see Patent Document 3) will be described. Note that in this specification, spatial cognition is used as a term having the same meaning as stereoscopic cognition.

[0043] FIG. 6 shows the relationship between a feature value x1 related to the subject's acceleration and the subject's spatial cognition score, as well as the relationship between a feature value x2 related to the subject's curve radius and the subject's spatial cognition score. Specifically, feature value x1 is the average acceleration in the subject's direction of travel near the start of the curve. Feature value x2 is the standard deviation of the subject's curve radius near the start of the curve. Note that the feature value of the subject near a point refers to a feature value derived from subset data including data on the time the subject passes through the point when the determination section data is divided into multiple subset data. For example, the feature value of the subject near the start of the curve refers to a feature value derived from subset data of the determination section data including the time the subject passes through the start of the curve.

[0044] As shown in Figure 6, the relationship between the feature value x1 and the spatial cognition score is expressed as a regression line Lc1 that falls within the confidence interval Ic1. Because the slope of the regression line Lc1 is positive, the relationship between the feature value x1 and the spatial cognition score is a positive correlation. In other words, the larger the feature value x1, the higher the spatial cognition score. This suggests that subjects with higher spatial cognition ability tend to accelerate in the direction of travel at the start of a curve.

[0045] The relationship between the feature value x2 and the spatial cognition score is expressed as the regression line Lc2, which is included in the confidence interval Ic2. Because the slope of the regression line Lc2 is negative, the relationship between the feature value x2 and the spatial cognition score is a negative correlation. In other words, the smaller the feature value x2, the higher the spatial cognition score. This suggests that subjects with higher spatial cognition ability tend to have smaller changes in the curve radius at the starting point of the curve.

[0046] 7 shows the relationship between the subject's curve radius feature value x3 and the subject's spatial cognition score, and the relationship between the subject's angular velocity feature value x4 and the subject's spatial cognition score. Specifically, feature value x3 is the standard deviation of the subject's curve radius near the center of the curve. Feature value x4 is the standard deviation of the angular velocity near the start of the curve.

[0047] As shown in Figure 7, the relationship between the feature value x3 and the spatial cognition score is represented by the regression line Lc3, which is included in the confidence interval Ic3. Because the slope of the regression line Lc3 is positive, the relationship between the feature value x3 and the spatial cognition score is a positive correlation. In other words, the larger the feature value x3, the higher the spatial cognition score. This suggests that subjects with higher spatial cognition ability tend to experience greater changes in the curve radius at the center of the curve.

[0048] The relationship between the feature value x4 and the spatial cognition score is expressed as a regression line Lc4, which is included in the confidence interval Ic4. Because the slope of the regression line Lc4 is positive, the relationship between the feature value x4 and the spatial cognition score is a positive correlation. In other words, the larger the feature value x4, the higher the spatial cognition score. This suggests that subjects with higher spatial cognition ability tend to have a larger change in angular velocity at the starting point of the curve.

[0049] 8 shows the relationship between the subject's speed feature value x5 and the subject's spatial cognition score, and the relationship between the subject's acceleration feature value x6 and the subject's spatial cognition score. Specifically, feature value x5 is the standard deviation of the subject's speed near the end of the curve. Feature value x6 is the standard deviation of the subject's acceleration in the direction of travel near the start of the curve.

[0050] As shown in Figure 8, the relationship between the feature value x5 and the spatial cognition score is represented by the regression line Lc5, which is included in the confidence interval Ic5. Because the slope of the regression line Lc5 is negative, the relationship between the feature value x5 and the spatial cognition score is a negative correlation. In other words, the smaller the feature value x5, the higher the spatial cognition score. This suggests that subjects with higher spatial cognition ability tend to have smaller changes in speed at the end of the curve.

[0051] The relationship between the feature value x6 and the spatial cognition score is expressed as a regression line Lc6 included in the confidence interval Ic6. Because the slope of the regression line Lc6 is negative, the relationship between the feature value x6 and the spatial cognition score is a negative correlation. In other words, the smaller the feature value x6, the higher the spatial cognition score. This suggests that subjects with higher spatial cognition ability tend to have smaller changes in acceleration at the starting point of the curve.

[0052] 9 is a histogram comparing, for each of multiple subjects Sb11 to Sb31, the spatial cognition scores obtained by existing stereoscopic ability evaluation methods with the spatial cognition scores obtained by the stereoscopic ability evaluation system 100 according to embodiment 1. In FIG. 9, the hatched bar graph on the left for each subject represents the spatial cognition score obtained by existing stereoscopic ability evaluation methods, and the unhatched bar graph on the right represents the spatial cognition score obtained by the stereoscopic ability evaluation system 100 according to embodiment 1.

[0053] 9, when determining that a subject with a spatial cognition score of 90 or less is a driving risk, the determination result based on the spatial cognition score by the stereoscopic cognitive ability assessment system 100 for subjects other than subject Sb20 is the same as the determination result based on the spatial cognition score by existing stereoscopic cognitive ability assessment methods. The stereoscopic cognitive ability assessment system 100 has almost the same accuracy as existing stereoscopic cognitive ability assessment methods.

[0054] Fig. 10 is a flowchart showing the flow of processing in the stereocognition ability evaluation method performed by the processor 101 that executes the stereocognition ability evaluation program 103a in Fig. 2. In the following, steps will be simply abbreviated as S.

[0055] 10 , in S101, the processor 101 selects the subject Sb1 as the driver of the vehicle Cs1 and proceeds to S102. In S102, the processor 101 reads calculated data related to the subject Sb1 (e.g., features derived from previous time-series data) from the storage 103 and proceeds to S103. In S103, the processor 101 acquires current time-series data from the measuring device 900 and proceeds to S104. In S104, the processor 101 extracts determination section data from the time-series data and proceeds to S105. In S105, the processor 101 extracts features from the determination section data and proceeds to S106.

[0056] In S106, the processor 101 determines whether the driver has changed. If the driver has not changed (NO in S106), the processor 101 returns to S102 and writes the calculation data including the feature values ​​calculated in S104 to the storage 103. If the driver has changed (YES in S106), the processor 101 writes the calculation data including the feature values ​​calculated in S104 to the storage 103 in S107 and proceeds to S108. In S108, the processor 101 uses the machine learning model 103c to determine the stereocognitive ability of the subject Sb1 based on the feature values ​​derived from the judgment section data, and proceeds to S109. In S109, the processor 101 transmits the evaluation result of the stereocognitive ability to the terminal device of the subject Sb1, and ends the process.

[0057] According to the stereoscopic perception ability assessment system 100, the trained machine learning model 103c generated by machine learning can be used to objectively assess the stereoscopic perception ability of the subject based on quantified features.

[0058] [Variation of First Embodiment] In the first embodiment, a configuration has been described in which the information processing device 110 has both a function of evaluating three-dimensional perception ability and a learning function of training a machine learning model. In a variation of the first embodiment, a configuration will be described in which the device having the function of evaluating three-dimensional perception ability and the device having the learning function are separate devices.

[0059] 11 is a block diagram showing the configuration of a stereoscopic perception ability assessment system 100A according to a modified example of the first embodiment. In the configuration of the stereoscopic perception ability assessment system 100A, the information processing device 110 in FIG. 1 is replaced with 110A, and a learning device 600 is added. Other than this, the configuration of the stereoscopic perception ability assessment system 100A is the same as the configuration of the stereoscopic perception ability assessment system 100 in FIG. 1, and therefore a description of this configuration will not be repeated. As shown in FIG. 11, the information processing device 110A, the learning device 600, the terminal device 800, the automobile Cs1, and the measurement device 900 are connected to each other via a network NW.

[0060] Fig. 12 is a diagram showing the hardware configuration of the stereocognition ability assessment system 100A of Fig. 11. The hardware configuration of the information processing device 110A is a configuration in which the machine learning program 103b and the training data 103d are removed from the storage 103 of Fig. 2. The hardware configuration of the information processing device 110A other than these is the same as the hardware configuration of the information processing device 110 of Fig. 2, and therefore description of this hardware configuration will not be repeated.

[0061] 12, the learning device 600 includes a processor 601, a RAM 602, a storage 603, a communication unit 604, a memory interface 605, and a housing Hs11. The housing Hs11 houses the processor 601, the RAM 602, the storage 603, the communication unit 604, and the memory interface 605. The processor 601, the RAM 602, the storage 603, the communication unit 604, and the memory interface 605 have similar functions to the processor 101, the RAM 102, the storage 103, the communication unit 104, and the memory interface 105, respectively, of FIG. 2, and therefore, a description of the similar functions of the processor 601, the RAM 602, the storage 603, the communication unit 604, and the memory interface 605 will not be repeated.

[0062] The storage 603 stores a machine learning program 103b, a machine learning model 103c, and training data 103d. The processor 601 generates a trained machine learning model 103c by executing the machine learning program 103b. The processor 601 provides the trained machine learning model 103c to the information processing device 110A via the communication unit 604.

[0063] As described above, the three-dimensional cognitive ability assessment system, three-dimensional cognitive ability assessment device, three-dimensional cognitive ability assessment program, and three-dimensional cognitive ability assessment method according to embodiment 1 and its modifications can achieve an objective assessment of three-dimensional cognitive ability.

[0064] [Embodiment 2] In embodiment 2, a configuration is described in which the risk of an accident when the subject drives a moving object (e.g., a car, motorcycle, or bicycle) is further determined based on the stereoscopic perception ability determined in embodiment 1 and the modified example. The accident risk may be any indicator that indicates the likelihood of an accident occurring when the subject drives a car. For example, the accident risk may include the possibility that the subject will have a car accident within a predetermined period (e.g., one month, six months, or one year), the number of car accidents the subject will have, or the instability of the subject's own body control (the number of falls or the likelihood of falling).

[0065] In the following, we first compare the spatial cognition score, which represents the subject's stereoscopic ability, with other features obtained from a questionnaire about driving a car, in terms of importance (contribution rate) to accident risk. Table 1 below shows examples of questionnaire items (features) about driving a car and sample answers.

[0066]

[0067] Numbers 13 to 16 in Table 1 are examples of features related to accident risk. Note that a "minor" accident in the example response to number 14, "Severity of accident," is, for example, an accident in which the subject inflicts damage on the vehicle, causing a dent in the vehicle. A "moderate" accident is, for example, an accident involving another vehicle, in which repairs are required for either the subject or the vehicle owner, passengers, or the other party, or medical treatment is required. A "severe" accident is, for example, an accident that requires a police investigation.

[0068] By performing supervised learning using each of the numbers 13 to 16 in Table 1 as the correct answer, a random forest model can be generated that outputs the numbers 13 to 16 in Table 1 based on stereoscopic ability and other feature quantities. Figure 13 is a graph showing the correspondence between the output values ​​(predicted values) of a trained random forest model and the actual values ​​of the feature quantities corresponding to the output values. In Figure 2, RMSE stands for root mean square error.

[0069] The following Tables 2, 3, 4, and 5 show the importance and correlation of the spatial cognition score with respect to the predicted values ​​of (a), (b), (c), and (d) in Figure 13, as well as the importance and correlation of other features. The rankings shown in Tables 2 to 5 are based on the importance of each feature.

[0070]

[0071] As shown in Table 2, the importance of spatial awareness score to the presence or absence of a history of accidents exceeds 0.15, ranking second only to age.

[0072]

[0073] As shown in Table 3, the importance of spatial awareness score to the severity of accidents is almost the same as that of age, which is ranked first, and is second only to age.

[0074]

[0075] As shown in Table 4, the importance of the spatial cognition score to the number of falls within the last month exceeds 0.25, ranking first.

[0076]

[0077] As shown in Table 5, the importance of the spatial cognition score to the number of trips made within the last month exceeds 0.10, ranking second only to table tennis experience.

[0078] As shown in Tables 2 to 5, the spatial awareness score ranks highly among multiple feature quantities in terms of importance to accident risk. Because the spatial awareness score has such a high importance to accident risk, it is possible to further predict accident risk from the spatial awareness score.

[0079] Fig. 14 is a flowchart showing the flow of processing in a method for evaluating stereoscopic ability performed by a processor executing a stereoscopic ability evaluation program according to embodiment 2. The flowchart shown in Fig. 14 is a flowchart obtained by adding S118 to the flowchart of Fig. 10 and replacing S109 with S119. As shown in Fig. 14, the processor 101 executes S101 to S108 in the same manner as in embodiment 1, and then determines the risk of an accident in S118 and proceeds to S119. In S119, the processor 101 transmits the evaluation results of the stereoscopic ability and the accident risk to the terminal device of the subject Sb1, and ends the processing.

[0080] As described above, the stereoscopic cognitive ability assessment system, stereoscopic cognitive ability assessment device, stereoscopic cognitive ability assessment program, and stereoscopic cognitive ability assessment method of embodiment 2 can achieve an objective assessment of stereoscopic cognitive ability, and can also determine the risk of an accident when the subject is driving a moving object based on the stereoscopic cognitive ability.

[0081] [Embodiment 3] The behavior of the subject reacting to a target is not limited to curve driving, which is the subject's reaction to a curve. Therefore, in embodiment 3, a configuration for evaluating the subject's three-dimensional cognitive ability will be described, focusing on behaviors other than curve driving. Hereinafter, the time interval during which the condition indicating that the subject is performing an action to react to a target is satisfied will also be referred to as a judgment interval, and the number of judgment intervals will be denoted as a natural number N.

[0082] FIG. 15 shows the correlation between predicted and measured spatial cognition scores of a subject who drives a car, as determined by the stereoscopic cognition ability assessment system according to embodiment 3, based on time-series data of the subject. FIG. 15 shows the subjects' behaviors in response to targets, including curve driving, sudden acceleration, sudden deceleration, sudden steering (lateral acceleration), and sudden steering (lateral angular velocity), and the regression lines Rc, Rac, Rbc, Rs, and Ran for each case. Targets in behaviors other than curve driving include, for example, preceding vehicles, traffic signals, signs, pedestrians, obstacles, or the road ahead of the subject. The time-series data is, for example, a driving record of approximately 15 minutes, including 30 or more judgment sections. The curve driving, sudden acceleration, sudden deceleration, sudden steering (lateral acceleration), and sudden steering (lateral angular velocity) shown in FIG. 15 are behaviors that occurred in one of the multiple judgment sections included in the time-series data.

[0083] The lateral acceleration (specific physical quantity) is the acceleration of the vehicle driven by the subject in a direction perpendicular to the vehicle's direction of travel. The lateral angular velocity (specific physical quantity) is the angular velocity of the vehicle driven by the subject, with an axis parallel to the direction of gravity as its axis of rotation. Multiple accelerations (specific physical quantities), multiple lateral accelerations, and multiple lateral angular velocities are derived from the time-series data of the vehicle driven by the subject. These physical quantities may be directly measured data or may be derived from other data included in the time-series data. The stereocognitive ability assessment system according to the third embodiment determines the subject's stereocognitive ability using at least one feature calculated from these physical quantities.

[0084] A sudden acceleration is an action that occurs during a time interval (determination section) in which, when time series data is divided into multiple time intervals (e.g., 15-second intervals), a positive acceleration having an absolute value exceeding a threshold (within a predetermined range) is obtained. A sudden deceleration is an action that occurs during a time interval in which a negative acceleration having an absolute value exceeding the threshold is obtained. The threshold is, for example, a statistical value obtained by adding the standard deviation to the average value of the multiple accelerations included in the determination section.

[0085] The sudden steering (lateral acceleration) is an action taken during a time interval (determination section) in which, when time-series data is divided into multiple time intervals, a lateral acceleration having an absolute value exceeding a threshold is obtained. The threshold is, for example, a statistical value obtained by adding 0.3 times the standard deviation to the average value of the multiple lateral accelerations included in the determination section.

[0086] The sudden steering (lateral angular velocity) is an action performed during a time interval (determination section) in which, when time-series data is divided into multiple time intervals, a lateral angular velocity having an absolute value exceeding a threshold is obtained. The threshold is, for example, a statistical value obtained by adding 0.3 times the standard deviation to the average value of the multiple lateral angular velocities included in the determination section.

[0087] The actual measured value of the subject's spatial cognition score, like the label attached to the training data, can include, for example, the score of another stereoscopic cognition ability assessment system, a depth perception test, a mental rotation task, a visual ability assessment test, a "seeing ability" comprehension test, an existing visual-spatial cognition assessment battery such as the K-ABC (Kaufman Assessment Battery for Children) test, a visual perceptual skills test, or a motor-exclusion visual cognition test, or the level or score of stereoscopic cognition determined in advance from the driver's driving skill, occupation, age, or driving history (information about the subject's attributes).

[0088] As shown in Figure 15, a positive correlation is observed between the predicted and measured spatial awareness scores for each of curve driving, sudden acceleration, sudden deceleration, sudden steering (lateral acceleration), and sudden steering (lateral angular velocity).

[0089] FIG. 16 shows the coefficient of determination R for each of the curve driving, sudden acceleration, sudden deceleration, sudden steering (lateral acceleration), and sudden steering (lateral angular velocity) shown in FIG. ２ The coefficient of determination R ２ is equal to the square of the correlation coefficient. ２ The closer to 1, the greater the explanatory power of the regression line.

[0090] As shown in FIG. 16, the coefficient of determination R ２ is 0.158. The coefficient of determination for sudden acceleration, R ２ is 0.13. The coefficient of determination for sudden deceleration, R ２ is 0.155. The coefficient of determination R for sudden steering (lateral acceleration) ２ is 0.141. The coefficient of determination R for sudden steering (lateral angular velocity) ２ is 0.102. The coefficient of determination R ２ is greater than 0.1. The explanatory power of the regression line for sudden deceleration is comparable to that of the regression line for curve driving. The explanatory power of the regression line for sudden acceleration is comparable to that of the regression line for sudden steering (lateral acceleration).

[0091] 17 is a diagram showing the correlation between the predicted value and the actual measured value of the spatial cognition score of the subject when the number of judgment sections N is 1, 3, 5, 10, and 30. FIG. 18 is a diagram showing the correlation between the number of judgment sections N in FIG. 17 and the prediction accuracy (coefficient of determination R ２ ) is a diagram showing the relationship between

[0092] 17 shows regression lines Rn1, Rn3, Rn5, Rn10, and Rn30 when the number of judgment sections N is 1, 3, 5, 10, and 30. When the number of judgment sections N is 2 or more, the multiple actions performed in each of the multiple judgment sections may include multiple different actions (for example, curve driving, sudden acceleration, sudden deceleration, sudden steering (lateral acceleration), and sudden steering (lateral angular velocity)).

[0093] As shown in Figure 17, a positive correlation is observed between the predicted value and the actual measured value of the spatial cognition score when the number of judgment sections N is 1, 3, 5, 10, and 30. The coefficient of determination R when the number of judgment sections N is 1 ２is 0.17. The coefficient of determination R when the number of judgment intervals N is 3 ２ is 0.33. The coefficient of determination R when the number of judgment intervals N is 5 ２ is 0.41. The coefficient of determination R when the number of judgment intervals N is 10 ２ is 0.52. The coefficient of determination R when the number of judgment intervals N is 30 ２ is 0.62. As shown in FIG. 18 , the greater the number of judgment intervals N, the higher the prediction accuracy. By increasing the number of judgment intervals in this way, the spatial cognitive ability of the subject can be determined based on data resulting from the subject's behavior, including various patterns. As a result, a multifaceted evaluation of the subject's spatial cognitive ability becomes possible, thereby improving the prediction accuracy of the stereoscopic cognitive ability evaluation system.

[0094] At least one of the conditions indicating that the subject is reacting to a target and the conditions indicating that the subject is attempting to maintain a constant state of motion may include the following: the subject's average speed during a predetermined time interval (e.g., 5 seconds) is within a predetermined range (speed range); the subject's trajectory is straight during a predetermined time interval (e.g., the curvature of the trajectory is within a predetermined range); the subject is moving at a constant speed (e.g., the subject's multiple accelerations during a predetermined time interval include both positive and negative accelerations); or the subject is accelerating or decelerating at a constant speed (e.g., the subject's acceleration is positive, negative, or the absolute value of the subject's acceleration is less than a threshold during a predetermined time interval). The existence of these conditions includes, for example, the subject running stably at a substantially constant speed as a reaction to a state in which free running is possible. The above speed ranges include, for example, an ultra-low speed range (a range of 0.1 m / sec or more and less than 1 m / sec), a low speed range (a range of 1 m / sec or more and less than 5 m / sec), a medium speed range (a range of 5 m / sec or more and less than 15 m / sec), and a high speed range (a range of 15 m / sec or more).

[0095] As described above, the three-dimensional cognitive ability assessment system, three-dimensional cognitive ability assessment device, three-dimensional cognitive ability assessment program, and three-dimensional cognitive ability assessment method according to embodiment 3 can achieve an objective assessment of three-dimensional cognitive ability.

[0096] [Embodiment 4] In the first to third embodiments, a configuration was described in which the stereoscopic cognitive ability of a subject is evaluated based on judgment interval data included in time-series data of the subject moving in the real world. In the fourth embodiment, a configuration is described in which the stereoscopic cognitive ability of a subject is evaluated based on judgment interval data included in time-series data of the subject moving in virtual reality (virtual space). Note that, as in the first embodiment, the fourth embodiment also describes a case in which the subject operates a car as a moving object, but the behavior that can be used to evaluate the stereoscopic cognitive ability of the subject may be any behavior that allows the subject to react to a target. Furthermore, virtual reality includes the metaverse.

[0097] 19 is a diagram showing how the stereocognitive ability of subject Sb2 is measured by a driving simulation measurement test in stereocognitive ability assessment system 200 according to embodiment 4. Similar to the stereocognitive ability assessment systems according to embodiments 1 to 3, stereocognitive ability assessment system 200 evaluates the stereocognitive ability of subject Sb2 using at least one feature calculated from data on the time interval during which subject Sb2 is taking action to react to a target.

[0098] 19, the stereocognitive ability assessment system 200 includes a terminal device 820 for the subject Sb2, a head-mounted display 210 (stereocognitive ability assessment device), and a control device Cd (virtual reality interface unit). The head-mounted display 210 and the control device Cd are connected to each other wirelessly or via a wire. The control device Cd includes a steering wheel Hd, an accelerator pedal Ap, and a brake pedal Bp.

[0099] Fig. 20 is a diagram showing an example of a screen displayed on the subject Sb2 via the head-mounted display 210 in a driving simulation using the stereoscopic perception ability assessment system 200 of Fig. 19. As shown in Fig. 20, the subject Sb2 is shown a view from the driver's seat of a car Cs2 (moving object) driven by the subject Sb2 via the control device Cd in a virtual space. A preceding car Ca2 (target) traveling ahead of the car Cs2 is displayed in the view. The car Cs2 moves in the virtual space in response to inputs from the subject Sb2 to the control device Cd of Fig. 19. The target is not limited to a car and may be, for example, a curve.

[0100] Fig. 21 is an external perspective view of the head-mounted display 210 of Fig. 19. In Fig. 21, the components indicated by dashed lines are housed inside the housing Hs2 of the head-mounted display 210 and cannot be seen from the outside. Details of this configuration will be described later with reference to Fig. 22. The head-mounted display 210 allows the subject to view a target (for example, a curve) and evaluates the subject's reaction to it, thereby evaluating the subject's stereoscopic perception ability.

[0101] The head-mounted display 210 is typically goggles equipped with an electronic display that displays moving images representing three-dimensional virtual reality, and takes the form of a virtual reality headset. A wearing band Rb, typically a rubber band, is attached to the head-mounted display 210. The user (subject) wears the head-mounted display 210 around the eyes by placing the head-mounted display 210 so that it covers the area around the eyes and wrapping the wearing band Rb around the head.

[0102] Fig. 22 is a diagram showing the hardware configuration of head-mounted display 210 of Fig. 21. As shown in Fig. 22, head-mounted display 210 includes processor 201, RAM 202, storage 203, communication unit 204, memory interface 205, electronic display 211, and gaze / pupil sensor 212. Processor 201, RAM 202, storage 203, communication unit 204, memory interface 205, and terminal device 820 of subject Sb2 have similar functions to processor 101, RAM 102, storage 103, communication unit 104, memory interface 105, and terminal device 800 of Fig. 2, respectively, and therefore description thereof will not be repeated.

[0103] The storage 203 stores an OS (Operating System) program (not shown), a stereoscopic perception ability assessment program 203a, a machine learning program 203b, a machine learning model 203c, and training data 203d. The stereoscopic perception ability assessment program 203a, the machine learning program 203b, the machine learning model 203c, and the training data 203d correspond to the stereoscopic perception ability assessment program 103a, the machine learning program 103b, the machine learning model 103c, and the training data 103d, respectively, in FIG. 2 . That is, in the stereoscopic perception ability assessment system 200, the processor 201 executes the machine learning program 203b, causing the head-mounted display 210 to function as a learning device that generates a trained machine learning model 203c. As in the modification of the first embodiment, machine learning for the machine learning model 203c may be performed by a learning device separate from the head-mounted display 210.

[0104] The electronic display 211 includes a flat panel display such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display. The electronic display 211 displays a moving image of a moving object moving in a virtual space to the subject, who wears the head-mounted display 210 around the subject's eyes, via eyepieces located on the subject's side. When moving image data is transferred from the processor 201 to the data buffer area of ​​the electronic display 211, the electronic display 211 reads the image data from the data buffer area and displays the moving image represented by the data. The electronic display 211 is provided separately for the right and left eyes, and the user views each of the electronic displays 211 through eyepieces. When an object displayed on the electronic display 211 is located at infinity, it is displayed at the same position on the electronic displays 211 for the right and left eyes. As a result, no parallax occurs between the left and right eyes, and the left and right eyes are diverged, giving the subject the sensation of being at infinity. As the object moves closer to the user, it is displayed closer to the center on the right-eye electronic display 211 and the left-eye electronic display 211. As a result, parallax occurs between the left and right eyes, causing the left and right eyes to converge, giving the subject the sensation that the object is nearby.

[0105] The gaze / pupil sensor 212 is positioned facing the subject, such as above the electronic display 211, and detects the gaze direction and pupil size of each of the left and right eyes. The gaze / pupil sensor 212 acquires images of each eye using an image acquisition device, such as a camera, and determines the gaze direction and pupil size by identifying the pupil position and pupil size in the image. The sensor outputs these as gaze information to the processor 201. A visible light camera or an infrared camera can be used as the camera. To determine whether the subject is viewing a target, the gaze direction is an important piece of data. By confirming that the gazes of both eyes (normals to the center of the pupils) accurately pass through the target, the subject's visibility can be confirmed. If the subject is viewing a nearby target, the gazes of the left and right eyes will converge due to parallax. The pupil diameter can also be used to determine whether the subject is continuously viewing an approaching target. When the subject is continuously viewing an approaching target, the pupil diameter gradually decreases due to the pupillary short-distance reflex, and by detecting this, it is possible to confirm whether the target has been viewed successfully.

[0106] Stereocognitive ability is based on the assumption that the eyes have normal functions for acquiring visual information (for example, pupil accommodation or eye movement). Therefore, stereocognitive ability is defined as the ability to accurately grasp the positional relationship of a target object based on visual information when the subject looks at the object, and to perform appropriate and accurate actions in response to the target object based on the grasped positional relationship.

[0107] Known methods for verifying normal eye function include pupillometry and near point distance measurement. These measurements can be performed using a device called TriIris. Pupillometry involves measuring the pupil's response to visible light stimuli from a visual target (light response) and measuring pupillary changes while viewing a moving visual target. Specifically, pupillary changes occur due to the pupillary near-distance reflex, in which the pupil constricts as the visual target moves closer. Near point distance measurement involves the subject pressing a handheld switch when the visual target approaches at a constant refraction velocity and becomes blurred. The target's position at that time is recorded as the near point distance. These measurements are based on the state of the eye (pupil) and the pressing of a switch at the point when the image becomes blurred, but their primary purpose is to measure the near point distance.

[0108] Stereocognition ability particularly includes the ability to accurately track a target with the eyes, correctly recognize its position, and respond appropriately to it. To measure the ability to track a target with the eyes, methods for measuring pupil accommodation and convergence response are considered particularly useful. These measurement methods are described below with reference to Figures 23 to 25.

[0109] Figure 23 is a diagram showing the concept of the visual target used to measure eye function. The visual target (a preceding vehicle Ca2) moves between receding and approaching directions so that it is visible in front of the subject's eyes. It is desirable to repeat the target and check the state of the subject's eyes each time.

[0110] Figure 24 shows the relationship between the target distance (the distance between the subject's eye and the target) and pupil diameter due to the pupillary near-distance reflex. Figure 24(A) shows that the pupil diameter becomes smaller when the target distance is close. Figure 24(B) shows that the pupil diameter becomes larger when the target distance is far. Figure 24(C) shows a graph with the target distance on the horizontal axis and the pupil diameter on the vertical axis. The solid line shows the graph for the left eye, and the dashed line shows the graph for the right eye. These graphs show that the pupil diameter becomes smaller when the target distance is close and larger when the target distance is far. They also show that the pupil diameter of the right eye and the left eye is approximately the same regardless of the target distance.

[0111] FIG. 25 shows the relationship between visual target distance and pupil position (convergence and divergence movement). FIG. 25(A) shows that when the visual target distance is close, the left and right eyes converge inward. FIG. 25(B) shows that when the visual target distance is far, the left and right eyes diverge and are parallel. FIG. 25(C) shows a graph with the visual target distance on the horizontal axis and the pupil position on the vertical axis. The solid line shows the graph for the left eye, and the dashed line shows the graph for the right eye. These graphs show that when the visual target distance is close, the distance between the pupils of the left and right eyes decreases, resulting in a state of convergence, and when the visual target distance is far, the distance between the pupils of the left and right eyes increases, resulting in a state of divergence.

[0112] When the index is moved to change the perspective of the visual target viewed by the subject, the subject's reactions, such as the aforementioned change in pupil diameter and convergence / divergence movements, occur. If the subject's visual function is impaired, these reactions decrease. Therefore, visual function can be measured by measuring the change in pupil diameter and convergence / divergence movements of the subject when the perspective of the visual target is changed. The stereocognition ability assessment system 200 does not require a large-scale device to change the perspective of the visual target, allowing for a compact stereocognition ability assessment system. Furthermore, since there is no need to go outside to acquire time-series data, objective evaluation of stereocognition ability can be performed indoors. Furthermore, the ability to set various targets in virtual reality allows for objective evaluation of stereocognition ability from multiple angles.

[0113] Figure 26 is a functional block diagram showing the configuration of the stereocognition ability evaluation function of the processor 201 in Figure 24. In the stereocognition ability evaluation system 200, a stereocognition ability evaluation program 203a stored in the storage 203 is executed by the processor 201 to configure modules forming functional blocks including a virtual reality display unit 201a, a section extraction unit 201b, a visibility determination unit 201c, a virtual reality control unit 201d, and a stereocognition ability determination unit 201e. Therefore, Figure 26 shows functional blocks realized by the processor 201 and the stereocognition ability evaluation program 203a in Figure 24 instead. These functional blocks will be described below.

[0114] The virtual reality display unit 201a continuously generates images of the scenery ahead seen by the subject Sb2 from the driver's seat of the vehicle Cs2 by three-dimensional rendering in accordance with the progress of the vehicle Cs2, and transfers these images as moving images to the data buffer area of ​​the electronic display 211. The moving images are data for the right and left eyes of the electronic display 211. The moving images create a parallax between the moving images for the right eye and the moving images for the left eye in the scenery seen by the subject Sb2 from the vehicle Cs2, depending on the position of the moving object. Therefore, the subject viewing the moving images on the electronic display 211 can see the ball with a realistic perspective. The virtual reality display unit 201a transmits driving data of the vehicle Cs2 based on the moving images to the section extraction unit 201b.

[0115] The section extraction unit 201b acquires running data (time-series data) of the subject Sb2 from the virtual reality display unit 201a. From the running data, the section extraction unit 201b extracts data that satisfy a condition indicating that the subject Sb2 is taking action to react to a target as judgment section data, and outputs the data to the stereoscopic perception ability assessment unit 201e. The section extraction unit 201b outputs information regarding the position of the car Cs2 to the visibility assessment unit 201c.

[0116] The visibility determination unit 201c determines whether the subject Sb2 visually and spatially recognizes the vehicle Cs2 by determining whether the gaze direction correctly corresponds to the position of the vehicle Cs2. The visibility determination unit 201c receives data on the gaze directions of the subject Sb2's left and right eyes detected by the gaze / pupil sensor 212, and determines whether the gaze directions of each of the left and right eyes correspond to the position of the vehicle Cs2 transmitted from the virtual reality display unit 201a, thereby determining whether the subject Sb2 spatially recognizes the vehicle Cs2. The visibility determination unit 201c may further receive data on the pupil diameters of the subject Sb2's left and right eyes detected by the gaze / pupil sensor 212. The visual recognition determination unit 201c may operate to determine that the subject is spatially recognizing an object when it further determines that the pupil diameter of both eyes is gradually decreasing while the position of the preceding vehicle Ca2 approaches a predetermined viewpoint and the distance between the subject and the object is decreasing (when a pupillary short-distance reflex is occurring in response to the decreasing distance between the subject and the object).The visual recognition determination unit 201c outputs the determination result to the stereoscopic perception ability determination unit 201e as visual recognition information.

[0117] The virtual reality control unit 201d receives input of the subject Sb2's active reaction corresponding to the three-dimensional position in the virtual space of the vehicle Cs2 recognized by the subject Sb2. The virtual reality control unit 201d identifies the reaction of the subject Sb2 driving the vehicle Cs2 based on the input information of the subject Sb2 to the control device Cd. The virtual reality control unit 201d outputs information related to the reaction to the virtual reality display unit 201a. The virtual reality display unit 201a moves the position of the subject Sb2 in the virtual reality based on the information.

[0118] The stereocognitive ability assessment unit 201e derives feature quantities from the assessment section data. These feature quantities include, for example, feature quantities related to at least one of the speed, acceleration, angular velocity, curve radius, and time interval of the assessment section data of the subject Sb2. The stereocognitive ability assessment unit 201e assesses the stereocognitive ability of the subject Sb2 from the feature quantities using the machine learning model 203c. The stereocognitive ability assessment unit 201e transmits the evaluation result of the stereocognitive ability to the terminal device 820 of the subject Sb2 via the communication unit 204.

[0119] In the stereoscopic perception ability assessment system 200, the processor 201 executes the machine learning program 203b, causing the head-mounted display 210 to function as a learning device that generates a trained machine learning model 203c. As in the modification of the first embodiment, machine learning for the machine learning model 203c may be performed by a learning device separate from the head-mounted display 210.

[0120] As described above, the three-dimensional cognitive ability assessment system, three-dimensional cognitive ability assessment device, three-dimensional cognitive ability assessment program, and three-dimensional cognitive ability assessment method according to embodiment 4 can achieve an objective assessment of three-dimensional cognitive ability.

[0121] [Additional Notes] The above-described embodiment according to the present disclosure includes the following configurations.

[0122] [Configuration 1] A system for assessing three-dimensional cognitive ability, comprising: an extraction unit that extracts data on time intervals from time series data of a moving subject during which the subject is performing at least one of the following actions: reacting to a target object and maintaining a constant state of motion; and a determination unit that determines the subject's three-dimensional cognitive ability using at least one feature calculated from the data on the time intervals.

[0123] [Configuration 2] The stereoscopic perception ability assessment system according to Configuration 1, wherein the time-series data includes at least one of a position, a velocity, an angular velocity, and an acceleration of a moving object moving along a trajectory based on the behavior of the subject, and an operation amount of the subject on an operation unit for operating the moving object, and the target is a curved portion of the trajectory.

[0124] [Configuration 3] The stereocognition ability evaluation system according to Configuration 2, wherein the at least one feature amount corresponds to at least one of the entire curved portion, a start point, a middle point, and an end point.

[0125] [Configuration 4] The stereoscopic ability assessment system according to any one of configurations 1 to 3, wherein the determination unit determines the stereoscopic ability from the at least one feature amount using a trained machine learning model.

[0126] [Configuration 5] The stereocognition ability evaluation system according to Configuration 4, wherein the machine learning model includes a model corresponding to a machine learning algorithm based on regression analysis.

[0127] [Configuration 6] The stereoscopic ability assessment system according to any one of Configurations 1 to 6, wherein the determining unit further determines an accident risk when the subject drives a moving object based on the stereoscopic ability.

[0128] [Configuration 7] The stereoscopic cognition ability assessment system according to any one of Configurations 1 to 6, wherein the at least one feature amount is derived from a plurality of specific physical quantities obtained from the data of the time interval, and at least one of the at least one feature amount is included in a predetermined range.

[0129] [Configuration 8] The stereoscopic perception ability assessment system described in Configuration 7, wherein each of the plurality of specific physical quantities includes at least one of a first acceleration, a second acceleration, and an angular velocity of a moving object that is operated by the subject to move along a trajectory, the direction of the first acceleration being parallel to the direction of travel of the moving object, and the direction of the second acceleration being perpendicular to the direction of travel.

[0130] [Configuration 9] The stereocognition ability evaluation system according to Configuration 7 or 8, wherein the threshold value included in the predetermined range is derived from statistical values ​​of a plurality of specific physical quantities included in the time-series data.

[0131] [Configuration 10] A stereoscopic cognitive ability assessment system according to any one of configurations 1 to 9, wherein the determination unit determines the stereoscopic cognitive ability of the subject using a feature calculated from data of multiple time intervals during which the subject is taking action to react to a target.

[0132] [Configuration 11] A stereoscopic cognitive ability assessment system according to any one of configurations 1 to 10, further comprising a virtual reality display unit that displays a virtual reality moving image including the target to the subject, wherein the virtual reality display unit moves the position of the subject in the virtual reality in response to an input from the subject.

[0133] [Configuration 12] The stereoscopic cognitive ability assessment system according to any one of configurations 1 to 11, further comprising: a terminal device for the subject; and a communication unit that transmits the determination result of the determination unit to the terminal device.

[0134] [Configuration 13] A stereoscopic ability evaluation program that, when executed by a computer, causes the computer to configure a stereoscopic ability evaluation device for evaluating the stereoscopic ability of a moving subject, the stereoscopic ability evaluation device comprising: an extraction unit that extracts, from the time series data of the subject, data included in a time interval during which the subject is performing at least one of an action to react to a target and an action to maintain a constant state of movement; and a determination unit that determines the stereoscopic ability using at least one feature extracted from the data included in the time interval.

[0135] [Configuration 14] The stereoscopic perception ability evaluation program according to Configuration 13, wherein the time series data includes at least one of a position, a velocity, an angular velocity, and an acceleration of a moving object moving along a trajectory based on the behavior of the subject, and an operation amount of the subject on an operation unit for operating the moving object, and the target is a curved portion of the trajectory.

[0136] [Configuration 15] The stereocognition ability evaluation program according to Configuration 14, wherein the at least one feature amount corresponds to at least one of the entire curved portion, a start point, a middle point, and an end point.

[0137] [Configuration 16] The stereoscopic ability evaluation program according to any one of Configurations 13 to 15, wherein the determination unit determines the stereoscopic ability from the at least one feature amount using a trained machine learning model.

[0138] [Configuration 17] The stereocognition ability evaluation program according to Configuration 16, wherein the machine learning model includes a model corresponding to a machine learning algorithm based on regression analysis.

[0139] [Configuration 18] The stereoscopic ability evaluation program according to any one of Configurations 13 to 17, wherein the determination unit further determines an accident risk when the subject drives a moving object based on the stereoscopic ability.

[0140] [Configuration 19] The stereocognition ability evaluation program according to any one of Configurations 13 to 18, wherein the at least one feature amount is derived from a plurality of specific physical quantities obtained from the data of the time interval, and at least one of the at least one feature amount is included in a predetermined range.

[0141] [Configuration 20] The stereoscopic perception ability assessment program according to Configuration 19, wherein each of the plurality of specific physical quantities includes at least one of a first acceleration, a second acceleration, and an angular velocity of a moving object operated and moved by the subject, the direction of the first acceleration being parallel to the direction of travel of the moving object, and the direction of the second acceleration being perpendicular to the direction of travel.

[0142] [Configuration 21] The stereocognition ability evaluation program according to Configuration 19, wherein the threshold value defining the predetermined range is derived from statistical values ​​of the plurality of specific physical quantities.

[0143] [Configuration 22] The stereoscopic cognitive ability evaluation program according to any one of Configurations 13 to 21, wherein the determination unit determines the stereoscopic cognitive ability of the subject using a feature calculated from data of a plurality of time intervals during which the subject is taking action to react to a target.

[0144] [Configuration 23] The stereoscopic perception ability evaluation program according to any one of Configurations 13 to 22, further comprising a virtual reality display unit that displays a virtual reality moving image including the target to the subject, wherein the virtual reality display unit moves the position of the subject in the virtual reality in response to an input from the subject.

[0145] [Configuration 24] The stereoscopic cognitive ability evaluation program according to any one of Configurations 13 to 23, further comprising: a terminal device for the subject; and a communication unit that transmits the determination result of the determination unit to the terminal device.

[0146] The embodiments disclosed herein are intended to be implemented in appropriate combinations within the scope of compatibility. The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not the above description, and is intended to include all modifications within the meaning and scope of the claims.

[0147] 100, 100A, 200 Stereocognition ability evaluation system, 101, 201, 601, 901 Processor, 101b, 111b, 201b Section extraction unit, 101e, 201e Stereocognition ability determination unit, 101f Learning unit, 102, 202, 602, 903 RAM, 103, 203, 603, 905 Storage, 103a, 203a Stereocognition ability evaluation program, 103b, 203b Machine learning program, 103c, 203c Machine learning model, 103d, 203d Learning data, 104, 204, 604, 904 Communication unit, 105, 205, 605, 906 Memory interface, 110, 110A Information processing device, 201a Virtual reality display unit, 201c Visibility determination unit, 201d Virtual reality control unit, 210 head-mounted display, 211 electronic display, 212 pupil sensor, 600 learning device, 800, 820 terminal device, 900 measuring device, 902 camera, 905a time series data, 907 sensor unit, Ap accelerator pedal, Bp brake pedal, Ca2 preceding vehicle, Cd control device, Cs1, Cs2 automobile, Cv11, Cv12 left curve, Cv21 to Cv23 right curve, Hd steering wheel, Hs1, Hs2, Hs11 housing, Ic1 to Ic6 confidence interval, Lc1 to Lc6, Rac, Ran, Rbc, Rc, Rn1, Rn3, Rn5, Rn10, Rn30, Rs regression line, NW network, Pg destination point, Ps starting point, Rb wearing band, Rd Road, Sb1, Sb2, Sb11, Sb20, Sb31 Person to be measured, x1 to x6 Feature amount.

Claims

1. A system for evaluating stereoscopic cognition comprising: an extraction unit that extracts data on time intervals during which a moving subject is performing at least one of the following actions: reacting to a target and maintaining a constant state of motion, from time series data; and a judgment unit that judges the subject's stereoscopic cognition ability using at least one feature calculated from the data on the time intervals.

2. The stereoscopic perception ability assessment system of claim 1, wherein the time series data includes at least one of the position, velocity, angular velocity, and acceleration of a moving object moving along a trajectory based on the subject's behavior, and the amount of operation of the subject on an operating unit for operating the moving object, and the target is a curved portion of the trajectory.

3. The stereocognitive ability assessment system according to claim 2, wherein the at least one feature amount corresponds to at least one of the entirety, the start point, the middle point, and the end point of the curved portion.

4. A stereoscopic perception ability assessment system according to any one of claims 1 to 3, wherein the judgment unit judges the stereoscopic perception ability from the at least one feature amount using a trained machine learning model.

5. The stereocognitive ability assessment system according to claim 4, wherein the machine learning model includes a model corresponding to a machine learning algorithm based on regression analysis.

6. A stereoscopic cognitive ability assessment system as described in any one of claims 1 to 5, wherein the judgment unit further judges the risk of an accident when the subject drives a moving object based on the stereoscopic cognitive ability.

7. A stereoscopic cognition ability assessment system according to any one of claims 1 to 6, wherein the at least one feature amount is derived from a plurality of specific physical quantities obtained from the data of the time interval, and at least one of the at least one feature amount is included within a predetermined range.

8. The stereoscopic perception ability assessment system of claim 7, wherein each of the plurality of specific physical quantities includes at least one of a first acceleration, a second acceleration, and an angular velocity of a moving object operated and moved by the subject, the direction of the first acceleration being parallel to the direction of travel of the moving object, and the direction of the second acceleration being perpendicular to the direction of travel.

9. The stereocognitive ability assessment system according to claim 7 or 8, wherein the threshold value defining the predetermined range is derived from statistical values ​​of the plurality of specific physical quantities.

10. A stereoscopic cognition ability assessment system as described in any one of claims 1 to 9, wherein the judgment unit judges the stereoscopic cognition ability of the subject using features calculated from data of multiple time intervals during which the subject is taking action to react to a target.

11. A stereoscopic perception ability assessment system as described in any one of claims 1 to 10, further comprising a virtual reality display unit that displays a virtual reality moving image including the target to the subject, the virtual reality display unit moving the position of the subject in the virtual reality in response to input from the subject.

12. A stereoscopic cognition ability assessment system according to any one of claims 1 to 11, further comprising: a terminal device for the person to be measured; and a communication unit for transmitting the judgment result of the judgment unit to the terminal device.

13. A stereoscopic cognitive ability assessment device comprising: an extraction unit that extracts data on a time interval during which a moving subject is performing at least one of the following actions: reacting to a target and maintaining a constant state of movement, from time series data of the subject; a judgment unit that judges the subject's stereoscopic cognitive ability using at least one feature extracted from the time interval data; and a housing that houses the extraction unit and the judgment unit.

14. A stereoscopic cognitive ability evaluation program that, when executed by a computer, causes the computer to configure a stereoscopic cognitive ability evaluation device for evaluating the stereoscopic cognitive ability of a moving subject, the stereoscopic cognitive ability evaluation device comprising: an extraction unit that extracts, from the time series data of the subject, data included in a time interval during which the subject is performing at least one of an action to react to a target and an action to maintain a constant state of movement; and a judgment unit that judges the stereoscopic cognitive ability using at least one feature extracted from the data included in the time interval.

15. The stereoscopic perception ability assessment program of claim 14, wherein the time series data includes at least one of the position, velocity, angular velocity, and acceleration of a moving object moving along a trajectory based on the subject's behavior, and the amount of operation of the subject on an operating unit for operating the moving object, and the target is a curved portion of the trajectory.

16. The stereocognition ability evaluation program according to claim 15, wherein the at least one feature amount corresponds to at least one of the entirety, the start point, the middle point, and the end point of the curved portion.

17. A stereoscopic perception ability evaluation program according to any one of claims 14 to 16, wherein the judgment unit judges the stereoscopic perception ability from the at least one feature amount using a trained machine learning model.

18. The stereocognitive ability evaluation program according to claim 17, wherein the machine learning model includes a model corresponding to a machine learning algorithm based on regression analysis.

19. A stereoscopic cognitive ability evaluation program as described in any one of claims 14 to 18, wherein the judgment unit further judges an accident risk when the subject drives a moving object based on the stereoscopic cognitive ability.

20. A stereoscopic cognition ability assessment program according to any one of claims 14 to 19, wherein the at least one feature amount is derived from a plurality of specific physical quantities obtained from the data of the time interval, and at least one of the at least one feature amount is included within a predetermined range.

21. The stereoscopic perception ability assessment program of claim 20, wherein each of the plurality of specific physical quantities includes at least one of a first acceleration, a second acceleration, and an angular velocity of a moving object operated and moved by the subject, the direction of the first acceleration being parallel to the direction of travel of the moving object, and the direction of the second acceleration being perpendicular to the direction of travel.

22. The stereocognitive ability evaluation program according to claim 20 or 21, wherein the threshold value defining the predetermined range is derived from statistical values ​​of the plurality of specific physical quantities.

23. A stereoscopic cognition ability evaluation program as described in any one of claims 14 to 22, wherein the judgment unit judges the stereoscopic cognition ability of the subject using features calculated from data of multiple time intervals during which the subject is taking action to react to a target.

24. A stereoscopic perception ability assessment program as described in any one of claims 14 to 23, further comprising a virtual reality display unit that displays to the subject a virtual reality moving image including the target, the virtual reality display unit moving the position of the subject in the virtual reality in response to an input from the subject.

25. A stereoscopic cognition ability evaluation program according to any one of claims 14 to 24, further comprising: a terminal device of the person to be measured; and a communication unit that transmits the judgment result of the judgment unit to the terminal device.

26. A method for evaluating the stereoscopic ability of a subject who is moving, comprising the steps of: extracting data from time-series data of the subject that is included in a time interval during which the subject is performing at least one of an action to react to a target and an action to maintain a constant state of movement; and determining the stereoscopic ability using at least one feature extracted from the data included in the time interval.

Images