Cognitive function prediction system

Abstract

A cognitive function prediction system (100) comprises a detection unit (20) and a prediction unit (30). The detection unit (20) detects, from an evaluation subject (SJ) carrying out a prescribed task, an action performed by the evaluation subject (SJ) and / or a response from the evaluation subject (SJ) with respect to a cognitive task. The prediction unit (30) predicts a change in the future cognitive function of the evaluation subject (SJ) on the basis of detection data indicating the result of the detection performed by the detection unit (20). The prescribed task includes a dual task in which a first exercise task and a first cognitive task are simultaneously imposed. The prediction unit (30) includes a machine learning model (ML1) that outputs a prediction result for a future change in the cognitive function of the evaluation subject (SJ). The machine learning model (ML1) is constructed by performing machine learning of training data including label data that indicates a change in the evaluation value for the cognitive function of a subject, the label data being collected within a first prescribed period.

Description

Cognitive function prediction system

[0001] The present invention relates to a cognitive function prediction system.

[0002] A method for predicting future changes in cognitive function using technology such as deep learning has been proposed (see, for example, Non-Patent Document 1). Specifically, MRI (Magnetic Resonance Imaging) data, biomarkers, clinical test records, and the like are learned (machine learned) by deep learning or the like.

[0003] BH Vieira et al., "Predicting future cognitive decline from non-brain and multimodal brain imaging data in healthy and pathological aging", Neurobiology of Aging, Vol. 118, pp. 55-65, 2022.

[0004] However, MRI data, biomarkers, clinical test records, etc. cannot be measured on a daily basis, and therefore it is not easy to predict future changes in the cognitive function of a subject by measuring the subject's MRI data, biomarkers, clinical test records, etc.

[0005] The present invention has been made in view of the above-mentioned problems, and its purpose is to provide a cognitive function prediction system that can easily predict future changes in cognitive function.

[0006] According to one aspect of the present invention, a cognitive function prediction system includes a detection unit and a prediction unit. The detection unit detects at least one of the subject's movements or the subject's responses to cognitive tasks from a subject performing a predetermined task. The prediction unit predicts future changes in the subject's cognitive function based on detection data indicating the results of the detection by the detection unit. The predetermined task includes a dual task that simultaneously assigns a first motor task and a first cognitive task. The subject's movements include the subject's movements performing the dual task. The subject's responses include the subject's responses to the first cognitive task. The prediction unit includes a machine learning model that outputs a prediction result of future changes in the subject's cognitive function. The machine learning model is constructed by machine learning training data including label data indicating changes in the subject's cognitive function assessment values ​​collected within a first predetermined period.

[0007] In one embodiment, the prediction unit predicts, as the future change in the cognitive function of the person to be evaluated, whether or not the cognitive function of the person to be evaluated will decline in the future.

[0008] In one embodiment, the prediction unit predicts whether the subject will develop mild cognitive impairment or dementia in the future as a change in the subject's cognitive function in the future.

[0009] In one embodiment, the subject performs the predetermined task a plurality of times within a second predetermined period, and the prediction unit predicts a future change in the cognitive function of the subject based on the detection data for each of the plurality of times.

[0010] In one embodiment, the predetermined task further includes a motor task that imposes a second motor task, and the movement of the subject further includes a movement of the subject performing the motor task.

[0011] In one embodiment, the predetermined task further includes a cognitive task that imposes a second cognitive task, and the response of the subject further includes a response of the subject to the second cognitive task.

[0012] According to the cognitive function prediction system of the present invention, future changes in cognitive function can be easily predicted.

[0013] 1 is a diagram showing a cognitive function prediction system according to a first embodiment of the present invention. FIG. 1 is a diagram showing an example of a skeleton included in a skeleton sequence. (a) to (d) are diagrams showing an example of a predetermined task that is presented to an evaluation subject by a task presenter included in the cognitive function prediction system according to the first embodiment of the present invention. FIG. 1 is a diagram showing another example of a cognitive task that is presented to an evaluation subject by a task presenter included in the cognitive function prediction system according to the first embodiment of the present invention. FIG. 1 is a block diagram showing the configuration of a cognitive function prediction system according to the first embodiment of the present invention. FIG. 1 is a diagram showing a process executed by a prediction unit included in the cognitive function prediction system according to the first embodiment of the present invention. FIG. 1 is a diagram showing the flow of ...(a) is a diagram showing an example of time-series data of MMSE scores and a simple regression line. FIG. 1(b) is a diagram showing another example of time-series data of MMSE scores and a simple regression line. FIG. 1 is a block diagram showing the configuration of a cognitive function prediction system according to a second embodiment of the present invention. FIG. 1 is a diagram showing the flow of a process executed by a prediction unit included in the cognitive function prediction system according to the second embodiment of the present invention.

[0014] Hereinafter, embodiments of the cognitive function prediction system of the present invention will be described with reference to the drawings (FIGS. 1 to 12). However, the present invention is not limited to the following embodiments, and can be implemented in various forms without departing from the spirit of the present invention. Note that where explanations are redundant, they may be omitted as appropriate. Furthermore, in the drawings, identical or equivalent parts are designated by the same reference symbols, and explanations will not be repeated.

[0015] [Embodiment 1] First, a cognitive function prediction system 100 of this embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing the cognitive function prediction system 100 of embodiment 1. The cognitive function prediction system 100 is a system that predicts future changes in the cognitive function of the subject SJ. Specifically, the cognitive function prediction system 100 may predict changes in the cognitive function of the subject SJ two years from now. For example, the cognitive function prediction system 100 may predict whether the cognitive function of the subject SJ will have declined two years from now.

[0016] As shown in FIG. 1 , the cognitive function prediction system 100 includes a task setting unit 10 , a detection unit 20 , and a prediction unit 30 .

[0017] The task presenting unit 10 presents a task to be performed by the evaluation subject SJ. Specifically, the task presenting unit 10 may have a display such as a liquid crystal display or an organic electroluminescence (EL) display. The task presenting unit 10 displays a screen showing the task to be performed by the evaluation subject SJ on the display. The display may be installed, for example, in front of the evaluation subject SJ.

[0018] The tasks (predetermined tasks) to be performed by the subject SJ include at least one dual task. A dual task refers to a task in which the subject SJ is required to perform a motor task and a cognitive task simultaneously. A motor task is a task that imposes a motor task on the subject SJ. A cognitive task is a task that imposes a cognitive task on the subject SJ. Examples of motor tasks include stepping, walking, running, or skipping. Examples of cognitive tasks include calculation problems, location memory problems, or rock-paper-scissors problems.

[0019] The tasks (predetermined tasks) to be performed by the subject SJ may further include at least one single task. The single task is a motor task or a cognitive task. In this embodiment, the subject SJ performs a cognitive task (single task), a motor task (single task), and a dual task in this order.

[0020] Specifically, the subject SJ performs a cognitive task for a predetermined time (e.g., 30 seconds). Following the cognitive task, the subject SJ performs a motor task for a predetermined time (e.g., 20 seconds). Following the motor task, the subject SJ performs a dual task for a predetermined time (e.g., 30 seconds).

[0021] Hereinafter, the time during which the subject SJ performs a cognitive task may be referred to as "cognitive task performance time." Similarly, the time during which the subject SJ performs a motor task may be referred to as "motor task performance time," and the time during which the subject SJ performs a dual task may be referred to as "dual task performance time." The length of the cognitive task performance time may be arbitrarily set (determined). Similarly, the length of the motor task performance time and the length of the dual task performance time may be arbitrarily set (determined).

[0022] In this embodiment, the cognitive task (second cognitive task) assigned to the subject SJ in the cognitive task (single task) is the same as the cognitive task (first cognitive task) of the dual task. Furthermore, the motor task (second motor task) assigned to the subject SJ in the motor task (single task) is the same as the motor task (first motor task) of the dual task. Note that the cognitive task of the cognitive task may be different from the cognitive task of the dual task. Similarly, the motor task of the motor task may be different from the motor task of the dual task.

[0023] The detection unit 20 detects at least one of the actions of the subject SJ performing a predetermined task or the subject SJ's response to a cognitive task, and generates detection data. The detection data indicates the results of detection by the detection unit 20.

[0024] In this embodiment, the detection unit 20 detects the movements of the subject SJ who is performing a cognitive task, a motor task, and a dual task in this order, and the responses of the subject SJ to the cognitive tasks, and generates movement detection data and response detection data. Specifically, the detection unit 20 includes a movement detection unit 21 and a response detection unit 22.

[0025] The movement detection unit 21 detects the movement of the evaluation subject SJ performing a predetermined task and generates movement detection data. In this embodiment, the movement detection unit 21 detects the movement of the evaluation subject SJ performing a cognitive task, a motor task, and a dual task in this order. The movement detection data includes data indicating the movement of the evaluation subject SJ performing the motor task and data indicating the movement of the evaluation subject SJ performing the dual task.

[0026] For example, the motion detection unit 21 captures an image of the evaluation subject SJ performing a predetermined task and generates a skeleton sequence SD. The skeleton sequence SD is an example of "motion detection data." The skeleton sequence SD indicates time-series data of a skeleton SK (human skeletal model) representing the evaluation subject SJ. Specifically, the motion detection unit 21 may include an imaging unit 211 and a motion capture unit 212.

[0027] The imaging unit 211 captures an image of the evaluation subject SJ performing a predetermined task and generates an imaging signal. In this embodiment, the imaging unit 211 captures an image of the evaluation subject SJ performing a cognitive task, a motor task, and a dual task in this order. The imaging unit 211 may capture an image of the evaluation subject SJ from the start of the motor task to the end of the dual task. The imaging unit 211 includes, for example, a charge-coupled device (CCD) image sensor, a complementary metal oxide semiconductor (CMOS) image sensor, a range sensor, or a depth sensor. The imaging unit 211 is disposed, for example, in front of the evaluation subject SJ.

[0028] The motion capture unit 212 converts the movements of each part of the evaluation subject SJ into vector data to generate motion capture data that reflects the movements of each part of the evaluation subject SJ (movements of the evaluation subject SJ). The motion capture data is a series of frames showing a skeleton SK that moves in accordance with the movements of the evaluation subject SJ captured by the imaging unit 211. In other words, the motion capture data shows time-series data (skeleton sequence SD) of the skeleton SK. The motion capture data is an example of "movement detection data."

[0029] The motion capture unit 212 includes, for example, a processor and a storage unit. The processor includes, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit). The storage unit stores computer programs executed by the processor. The computer programs include a computer program for generating motion capture data from an imaging signal. The storage unit includes, for example, semiconductor memory (semiconductor storage device) such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit may further include a VRAM (Video RAM).

[0030] The answer detection unit 22 detects answers to the cognitive tasks of the evaluation subject SJ who is performing a predetermined task. In this embodiment, the answer detection unit 22 detects answers to the cognitive tasks of the cognitive task of the evaluation subject SJ and answers to the cognitive tasks of the dual task of the evaluation subject SJ.

[0031] For example, the answer detection unit 22 includes an answer switch for the left hand and an answer switch for the right hand. The subject SJ answers the cognitive task presented by the task presentation unit 10 by pressing the answer switch for the left hand or the answer switch for the right hand.

[0032] The answer switch for the left hand may be held in the left hand of the subject SJ or may be fixed to a handrail installed on the left side of the subject SJ. Similarly, the answer switch for the right hand may be held in the right hand of the subject SJ or may be fixed to a handrail installed on the right side of the subject SJ.

[0033] The prediction unit 30 predicts future changes in the cognitive function of the subject SJ based on the detection data output from the detection unit 20. Specifically, the prediction unit 30 extracts at least one of the characteristics of the subject SJ's movements and the characteristics of the subject SJ's answers to the cognitive tasks based on the detection data, and predicts future changes in the cognitive function of the subject SJ based on the extracted characteristics. In this embodiment, the prediction unit 30 extracts the characteristics of the subject SJ's movements and the characteristics of the subject SJ's answers to the cognitive tasks based on the movement detection data and the answer detection data. The prediction unit 30 may predict changes in the cognitive function of the subject SJ after a predetermined period (e.g., two years) has elapsed since the date and time the subject SJ last performed a predetermined task. For example, the prediction unit 30 may predict whether the subject SJ's cognitive function is declining.

[0034] Next, the skeletons SK included in the skeleton sequence SD (time-series data of skeletons SK) will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the skeletons SK included in the skeleton sequence SD.

[0035] As shown in FIG. 2 , the skeleton SK has a graph structure including a plurality of joints J. Specifically, the skeleton SK represents the evaluation subject SJ using a tree structure in which adjacent joints J are linked based on the structure of the human body. In this embodiment, the skeleton SK represents the entire body of the evaluation subject SJ. The skeleton SK illustrated in FIG. 2 represents the entire body of the evaluation subject SJ using a tree structure in which 20 joints J are linked. The skeleton SK may be a two-dimensional human skeletal model or a three-dimensional human skeletal model. FIG. 2 illustrates an example of a two-dimensional human skeletal model. Note that the joints J that make up the skeleton SK do not have to correspond to human joints.

[0036] Next, the task presenter 10 and the answer detector 22 will be described with reference to Figures 3(a) to 3(d) and 4. Figures 3(a) to 3(d) are diagrams showing an example of a predetermined task presented to the subject SJ by the task presenter 10 included in the cognitive function prediction system 100 of this embodiment. In the example shown in Figures 3(a) to 3(d), the motor task assigned to the subject SJ is "tapping," and the cognitive task (cognitive problem) assigned to the subject SJ is a "calculation problem."

[0037] As shown in (a) of Figure 3, the task presenter 10 displays a first notification screen 11 on the display to notify the subject SJ of the start of the task. Next, as shown in (b) of Figure 3, the task presenter 10 presents a cognitive task. Specifically, the task presenter 10 displays a problem presenting screen 12a on the display showing a calculation problem (cognitive assignment). For example, the task presenter 10 displays a subtraction problem on the display as the calculation problem.

[0038] In the example shown in FIG. 3B, the calculation problem is a subtraction problem, but the calculation problem is not limited to a subtraction problem. The calculation problem may be an addition problem. Alternatively, the calculation problem may include a subtraction problem and an addition problem.

[0039] The task presenter 10 terminates the display of the cognitive task (calculation problem) at a predetermined timing. Specifically, it erases the problem presenter screen 12a from the display. After erasing the problem presenter screen 12a from the display, the task presenter 10 displays an answer candidate presenter screen 12b showing two answer candidates on the display. The subject SJ presses one of the two answer switches of the answer detector 22 to select an answer.

[0040] When the subject SJ selects an answer, the task presenter 10 displays the next calculation problem (next cognitive task) on the display. Specifically, a calculation problem different from the previous calculation problem is displayed on the display as the next calculation problem. Thereafter, the presentation of calculation problems (cognitive tasks) to be answered by the subject SJ is repeated until a predetermined cognitive task performance time has elapsed.

[0041] 3(c), after a predetermined cognitive task performance time has elapsed, the task presenter 10 presents the motor task to the subject SJ. Specifically, the task presenter 10 displays on the display an exercise presentation screen 13 showing the motor task (stepping) to be assigned to the subject SJ.

[0042] In this embodiment, the motor task of the dual task is the same as the motor task. Furthermore, the cognitive task of the dual task is the same as the cognitive task. Therefore, after a predetermined motor task performance time has elapsed, the task presenter 10 repeatedly presents the calculation problem (cognitive task) to be answered by the subject SJ, as described with reference to FIG. 3B, until a predetermined dual task performance time has elapsed.

[0043] As shown in FIG. 3(d), after a predetermined dual task performance time has elapsed, the task presenter 10 displays a second notification screen 14 on the display to notify the subject SJ of the completion of the task.

[0044] The cognitive task is not limited to a calculation problem. For example, the cognitive task may be a location memory problem or a rock-paper-scissors problem.

[0045] 4 is a diagram showing another example of a cognitive task presented to the subject SJ by the task presenter 10 included in the cognitive function prediction system 100 of this embodiment. In the example shown in FIG. 4, the cognitive task is a "location memory task." As shown in FIG. 4, when the cognitive task is a location memory task, the task presenter 10 displays on the display a task presentation screen 12a in which a figure is placed in one of four areas in which figures can be placed.

[0046] The task presenter 10 erases the question presentation screen 12a from the display, and then displays an answer candidate presentation screen 12b on the display, in which a figure is placed in one of the four areas in which figures can be placed. The answer candidate presentation screen 12b displays a question statement along with two answer candidates, "yes" and "no." The question statement indicates a question that can be answered with "yes" or "no." Here, the subject SJ is asked whether the position of the figure is the same between the question presentation screen 12a and the answer candidate presentation screen 12b.

[0047] If the evaluation subject SJ determines that the position of the figure is the same between the question presentation screen 12a and the answer candidate presentation screen 12b, he / she presses the answer switch for the left hand to select "Yes." Alternatively, if the evaluation subject SJ determines that the position of the figure is different between the question presentation screen 12a and the answer candidate presentation screen 12b, he / she presses the answer switch for the right hand to select "No."

[0048] Next, the "Rock-Paper-Scissors Problem" will be explained. When the cognitive task assigned to the evaluation subject SJ is a "Rock-Paper-Scissors Problem," for example, the task presenter 10 displays one of "Rock," "Scissors," and "Paper" on the problem presentation screen 12a. Then, on the answer candidate presentation screen 12b, one of "Rock," "Scissors," and "Paper," a question statement, and "Yes" and "No" are displayed. On the answer candidate presentation screen 12b, for example, the evaluation subject SJ is asked whether the finger pose displayed on the answer candidate presentation screen 12b can beat the finger pose displayed on the problem presentation screen 12a.

[0049] If the evaluation subject SJ determines that the finger pose displayed on the answer candidate presentation screen 12b is better than the finger pose displayed on the question presentation screen 12a, he / she presses the answer switch for the left hand to select "Yes." Alternatively, if the evaluation subject SJ determines that the finger pose displayed on the answer candidate presentation screen 12b is not better than the finger pose displayed on the question presentation screen 12a, he / she presses the answer switch for the right hand to select "No."

[0050] Next, the cognitive function prediction system 100 of this embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the cognitive function prediction system 100 of the first embodiment.

[0051] 5, the cognitive function prediction system 100 of this embodiment further includes an input unit 40, an identification information acquisition unit 50, and a result presentation unit 60. The prediction unit 30 includes a processing unit 101 and a storage unit 102. The prediction unit 30 may be configured, for example, by a general-purpose computer or a dedicated computer.

[0052] The input unit 40 includes a user interface device operated by the worker. For example, the input unit 40 may include a keyboard and a mouse as user interface devices. The worker can operate the input unit 40 to input various instructions (or commands) and various information (or data) to the prediction unit 30 (processing unit 101). Specifically, the input unit 40 inputs a signal corresponding to the worker's operation to the processing unit 101. For example, the worker operates the input unit 40 to cause the prediction unit 30 to predict future changes in the cognitive function of the subject SJ.

[0053] Each evaluation subject SJ who uses the cognitive function prediction system 100 is assigned unique identification information in advance. The identification information acquisition unit 50 acquires the identification information assigned to each evaluation subject SJ and inputs it to the processing unit 101. The identification information acquisition unit 50 includes, for example, a card reader. Each evaluation subject SJ has the card reader read the identification information carried on the card. As a result, the identification information assigned to each evaluation subject SJ is input to the processing unit 101. Note that an operator may operate the input unit 40 to input the identification information assigned to each evaluation subject SJ.

[0054] The storage unit 102 has a storage device. The storage device includes, for example, a semiconductor memory (semiconductor storage device). The storage unit 102 may include, for example, a ROM and a RAM as the semiconductor memory. The storage unit 102 may further include a VRAM as the semiconductor memory. The storage unit 102 may also have a storage device. The storage unit 102 may include, for example, at least one of an HDD (Hard Disk Drive) and an SSD (Solid State Drive) as the storage device. The storage unit 102 may further include removable media.

[0055] The storage unit 102 stores various computer programs and various data. Specifically, the storage unit 102 stores basic characteristic data BD, which will be described later with reference to Fig. 7, in association with identification information assigned to each evaluation subject SJ.

[0056] The storage unit 102 also stores a first machine learning model ML1. The first machine learning model ML1 is a computer program that predicts future changes in the cognitive function of the subject SJ and outputs the prediction result. Specifically, the prediction result may indicate whether the cognitive function of the subject SJ has changed after a predetermined period (e.g., two years) has elapsed since the measurement end date and time, which is the date and time the subject SJ last performed a predetermined task, compared to the cognitive function at the measurement end date and time. Alternatively, the prediction result may indicate whether the cognitive function of the subject SJ has changed after a predetermined period has elapsed since the specified date and time, which is the date and time specified by the operator using the input unit 40, compared to the cognitive function at the specified date and time. For example, the prediction result may indicate whether the cognitive function of the subject SJ in the future will decline compared to the cognitive function at the measurement end date and time or the specified date and time.

[0057] The first machine learning model ML1 is constructed by machine learning the training data. The algorithm of the first machine learning model ML1 is not particularly limited as long as it is supervised learning, and may be, for example, a decision tree, a nearest neighbor method, a naive Bayes classifier, or a support vector machine. For example, LightGBM may be used as the first machine learning model ML1. The algorithm of LightGBM is gradient boosting with boosted decision trees.

[0058] Alternatively, the algorithm of the first machine learning model ML1 may be a neural network. The neural network includes an input layer, one or more hidden layers, and an output layer. Specifically, the neural network includes at least one of a deep neural network (DNN), a recurrent neural network (RNN), a convolutional neural network (CNN), a transformer, or a graph convolutional neural network such as a spatio-temporal graph convolutional neural network (ST-GCN), and performs deep learning. For example, a deep neural network includes an input layer, multiple hidden layers, and an output layer.

[0059] The processing unit 101 includes a processor. The processor may include, for example, a CPU, a GPU, or an NPU (Neural Network Processing Unit). Alternatively, the processing unit 101 may include a quantum computer.

[0060] The processing unit 101 executes various computer programs stored in the storage unit 102 to perform various processes such as numerical calculations, information processing, and device control.

[0061] Specifically, when the identification information acquisition unit 50 acquires the identification information, the processing unit 101 causes the task presentation unit 10 to display various screens, as described with reference to Figures 3(a) to 3(d) and Figure 4.

[0062] Furthermore, when the worker operates the input unit 40 to issue a command to output the prediction result, the processing unit 101 acquires the prediction result from the first machine learning model ML1 and causes the result presentation unit 60 to present the prediction result. The result presentation unit 60 includes, for example, a printer. The processing unit 101 controls the printer to output a sheet on which the prediction result is printed from the printer. Note that the cognitive function prediction system 100 may include a communication device as the result presentation unit 60. In this case, the processing unit 101 may control the communication device to send an email indicating the prediction result to, for example, an email address registered in advance by the subject SJ.

[0063] Next, the processing executed by the prediction unit 30 will be described with reference to Fig. 5 and Fig. 6. Fig. 6 is a diagram showing the processing executed by the prediction unit 30 included in the cognitive function prediction system 100 of this embodiment. In detail, Fig. 6 shows the processing executed by the processing unit 101.

[0064] As shown in Fig. 6, the processing unit 101 predicts future changes in the cognitive function of the subject SJ based on the action detection data and the response detection data (step S1). Specifically, the processing unit 101 acquires a prediction result from the first machine learning model ML1. Then, the processing unit 101 controls the result presentation unit 60 to present the prediction result (step S2). As a result, the process shown in Fig. 6 ends.

[0065] In this embodiment, the processing unit 101 extracts the characteristics of the movement of the evaluation subject SJ and the characteristics of the answer to the cognitive task from the movement detection data and the answer detection data, and inputs the extracted characteristics as explanatory variables to the first machine learning model ML1. As a result, a prediction result (objective variable) is output from the first machine learning model ML1. More specifically, the processing unit 101 calculates the characteristics of the movement of the evaluation subject SJ and the characteristics of the answer to the cognitive task based on the movement detection data and the answer detection data.

[0066] The processing executed by the prediction unit 30 (processing unit 101) will be described in detail below with reference to Figs. 5, 7, and 8. Figs. 7 and 8 are diagrams showing the flow of processing executed by the prediction unit 30 (processing unit 101) included in the cognitive function prediction system 100 of embodiment 1. In detail, Fig. 7 shows the flow of processing for collecting data used for prediction. Fig. 8 shows the flow of processing for prediction.

[0067] The process shown in Fig. 7 is executed when the subject SJ performs a predetermined task. As shown in Fig. 7, when the subject SJ starts performing the predetermined task, the subject SJ performs a cognitive task (step S11), a motor task (step S12), and a dual task (step S13) in this order, as described with reference to Figs. 3(a) to 3(d).

[0068] While the subject SJ is performing the cognitive task (step S11), the processing unit 101 acquires answer detection data from the answer detection unit 22 and stores it in the storage unit 102 (step S21).

[0069] While the subject SJ is performing the motor task (step S12), the processing unit 101 acquires movement detection data (skeleton sequence SD) from the movement detection unit 21 and stores it in the memory unit 102 (step S22).

[0070] The processing unit 101 acquires dual task data when the evaluation subject SJ is performing the dual task (step S13) and stores the data in the storage unit 102 (step S23). The dual task data indicates the action detection data and the answer detection data output from the action detection unit 21 and the answer detection unit 22, respectively, when the evaluation subject SJ is performing the dual task.

[0071] The processing unit 101 calculates basic feature amounts based on the detection data acquired while the subject SJ is performing a predetermined task, and stores basic feature data BD (see FIG. 5) indicating the calculated basic feature amounts in the storage unit 102 (step S24). As a result, the processing shown in FIG. 7 ends. In this embodiment, the basic feature amounts include feature amounts of the movements of the subject SJ and feature amounts of the responses of the subject SJ to the cognitive tasks. In other words, the basic feature data BD indicates feature amounts of the movements of the subject SJ and feature amounts of the responses of the subject SJ to the cognitive tasks.

[0072] Specifically, the processing unit 101 calculates basic feature amounts of the movement of the evaluation subject SJ based on the movement detection data acquired from the movement detection unit 21. In this embodiment, the processing unit 101 calculates basic feature amounts of the movement of the evaluation subject SJ based on the skeleton sequence SD (time-series data of the skeleton SK) acquired from the movement detection unit 21. Furthermore, the processing unit 101 calculates basic feature amounts of the answer of the evaluation subject SJ to the cognitive task based on the answer detection data acquired from the answer detection unit 22. Then, the processing unit 101 stores basic feature data BD indicating the basic feature amounts of the movement and the basic feature amounts of the answer in the storage unit 102.

[0073] Specifically, the subject SJ performs a predetermined task multiple times during a predetermined task data acquisition period. The processing unit 101 associates the basic characteristic data BD from each time with identification information assigned to each subject SJ and stores it in the memory unit 102. In other words, the basic characteristic data BD of the subject SJ is collected during the predetermined task data acquisition period. The task data acquisition period is, for example, six months. The task data acquisition period is an example of a "second predetermined period." The prediction unit 30 (processing unit 101) predicts future changes in the cognitive function of the subject SJ based on the basic characteristic data BD acquired multiple times during the predetermined task data acquisition period. Note that the task data acquisition period is not limited to six months. The task data acquisition period can be set to any value.

[0074] An example of the basic feature amounts is shown in Table 1 below. Table 1 shows an example of the basic feature amounts calculated when the subject SJ is assigned stepping as a motor task and a calculation problem as a cognitive task. As shown in Table 1, the processing unit 101 may calculate nine types of basic feature amounts (first to ninth basic feature amounts).

[0075]

[0076] Specifically, the movement detection data includes first movement detection data indicating the movement of the evaluation subject SJ performing a motor task (single task) and second movement detection data indicating the movement of the evaluation subject SJ performing a dual task. The processing unit 101 calculates a first basic feature amount and a third basic feature amount based on the first movement detection data. The processing unit 101 calculates a second basic feature amount and a fourth basic feature amount based on the second movement detection data. The first basic feature amount and the second basic feature amount indicate the average number of times the evaluation subject SJ steps per unit time. The third basic feature amount and the fourth basic feature amount indicate the standard deviation of the number of times the evaluation subject SJ steps per unit time. The unit time is, for example, one second.

[0077] The processing unit 101 further calculates the average size of the thigh raise (fifth basic feature) of the subject SJ performing the stepping (motor task) based on the first movement detection data and the second movement detection data.

[0078] The answer detection data includes first answer detection data indicating the result of the answer of the subject SJ to the cognitive task (single task) and second answer detection data indicating the result of the answer of the subject SJ to the cognitive task (dual task). The processing unit 101 calculates sixth and eighth basic feature amounts based on the first answer detection data, and calculates seventh and ninth basic feature amounts based on the second answer detection data. The sixth and seventh basic feature amounts indicate the correct answer rate. The eighth and ninth basic feature amounts indicate the average answer time.

[0079] The correct answer rate indicates the ratio between the number of answers and the number of correct answers. The number of answers indicates the number of times the subject SJ answered. The number of correct answers indicates the number of correct answers. The average answer time indicates the average length of time it took the subject SJ to answer one cognitive task. For example, the average answer time may indicate the average length of time it took the subject SJ to press the answer switch after the answer candidate presentation screen 12b described with reference to Figures 3(b) and 4 was presented by the task presentation unit 10.

[0080] The process shown in Fig. 8 starts, for example, when a worker operates the input unit 40 to issue a command to output a prediction result. When the process shown in Fig. 8 starts, the processing unit 101 first creates time-series data from the basic feature data BD acquired during a predetermined task data acquisition period (e.g., six months) (step S31). Specifically, the processing unit 101 executes a process of connecting the basic feature data BD in the time-series direction.

[0081] For example, when the nine types of basic feature amounts shown in Table 1 are used, the processing unit 101 creates multivariate time-series data by connecting each of the first to ninth basic feature amounts in the time-series direction. Specifically, the processing unit 101 creates time-series data of a nine-dimensional vector. The time-series data of the nine-dimensional vector indicates a nine-dimensional feature vector at each measurement time point. Each measurement time point indicates the date and time when the subject SJ performed a predetermined task.

[0082] The processing unit 101 may sample the basic feature data BD acquired during a predetermined task data acquisition period (e.g., six months) from the basic feature data BD stored in the storage unit 102. For example, the processing unit 101 may perform sampling by sliding window processing.

[0083] Specifically, the processing unit 101 may sample the basic characteristic data BD for a predetermined task data acquisition period, going back from the date and time when the subject SJ last performed a predetermined task (measurement end date and time). For example, the processing unit 101 may sample the basic characteristic data BD acquired within the past six months from the measurement end date and time. In this case, the prediction unit 30 predicts changes in the cognitive function of the subject SJ after a predetermined period (e.g., two years) has elapsed since the measurement end date and time. The measurement end date and time may be input (set) by the operator by operating the input unit 40.

[0084] Alternatively, the processing unit 101 may sample the basic characteristic data BD acquired during a predetermined task data acquisition period (e.g., six months) from the date and time when the subject SJ first performed a predetermined task (measurement start date and time). In this case, the prediction unit 30 uses the date on which the predetermined task data acquisition period (e.g., six months) has elapsed since the measurement start date and time as a reference date, and predicts changes in the cognitive function of the subject SJ after a predetermined period (e.g., two years) has elapsed since the reference date and time. The measurement start date and time may be input (set) by the operator by operating the input unit 40.

[0085] Alternatively, the processing unit 101 may sample the basic feature data BD for a specified task data acquisition period going back in time from an arbitrary date and time input (set) by the worker by operating the input unit 40, or may sample the basic feature data BD acquired from the set date and time until the specified task data acquisition period has elapsed.

[0086] When sampling the basic feature data BD going back in time from the date and time input by the worker, the prediction unit 30 predicts a change in the cognitive function of the person to be evaluated SJ after a predetermined period (e.g., two years) has passed since the date and time input by the worker. Also, when sampling the basic feature data BD from the date and time input by the worker until a predetermined task data acquisition period has passed, the prediction unit 30 uses the day on which the predetermined task data acquisition period (e.g., six months) has passed since the date and time input by the worker as a reference date, and predicts a change in the cognitive function of the person to be evaluated SJ after a predetermined period (e.g., two years) has passed since the reference date.

[0087] Furthermore, before creating the time-series data, the processing unit 101 may perform preprocessing to remove basic feature data BD that includes predefined outliers from the basic feature data BD stored in the storage unit 102. Performing the preprocessing improves prediction accuracy. An example of the definition of the outliers is shown in Table 2 below. Table 2 shows an example of the definition of the outliers for the first to ninth basic feature amounts. The processing unit 101 may remove basic feature data BD, for which at least one value of the nine types of basic feature amounts corresponds to a corresponding outlier, from the basic feature data BD stored in the storage unit 102.

[0088]

[0089] Next, the processing unit 101 calculates multiple types of statistics from the created time-series data (step S32). For example, the processing unit 101 calculates a lag feature of the first basic feature (average stepping speed), a lag feature of the second basic feature (average stepping speed), the number of peaks of the eighth basic feature (average response time), the number of peaks of the ninth basic feature (average response time), etc. tsfresh, for example, may be used to calculate the features.

[0090] Next, the processing unit 101 executes a filtering process on the calculated multiple types of statistics, and selects some of the calculated multiple types of statistics (step S33).

[0091] Specifically, when obtaining explanatory variables of the learning data used to train the first machine learning model ML1, if a statistic highly correlated with the objective variable (a statistic effective for predicting changes in cognitive function) is selected from among multiple types of statistics, the processing unit 101 may filter the multiple types of statistics calculated in step S32 by referring to the name of the statistic selected when obtaining explanatory variables of the learning data.

[0092] Next, the processing unit 101 inputs the selected statistics as explanatory variables into the first machine learning model ML1 (step S34). As a result, a prediction result (objective variable) is output from the first machine learning model ML1. The selected statistics are an example of the features (features) of the actions extracted from the action detection data and the features (features) of the answers extracted from the answer detection data.

[0093] Finally, the processing unit 101 controls the result presentation unit 60 to present the prediction result obtained from the first machine learning model ML1 (step S35), thereby completing the process shown in FIG.

[0094] Next, the training data used to train the first machine learning model ML1 will be described. The training data may be created by the cognitive function prediction system 100 described with reference to Figures 1 to 5, or may be created using another computer system. Below, the training data will be described using the cognitive function prediction system 100 described with reference to Figures 1 to 5 as an example.

[0095] The statistical quantities described with reference to FIG. 8 are used as explanatory variables for the training data. Specifically, multiple subjects are made to perform predetermined tasks according to the procedure described with reference to FIG. 7, and basic feature data BD is collected for each subject. For example, the basic feature data BD may be collected by having multiple subjects perform predetermined tasks over a predetermined task data acquisition period (e.g., six months), or the basic feature data BD may be collected from multiple subjects without a set period. The operator operates the input unit 40 to cause the processing unit 101 to create time-series data of the basic feature data BD, calculate multiple types of statistical quantities from the time-series data, and select some of the multiple types of statistical quantities, as described with reference to FIG. 8. As a result, explanatory variables to be used for the training data are acquired.

[0096] For example, tsfresh may be used for the selection process of selecting some of the multiple types of statistics. By selecting statistics highly related to the target variable from the multiple types of statistics and having the first machine learning model ML1 learn them, overfitting is less likely to occur.

[0097] If the basic feature data BD is collected from multiple subjects without setting a time period, when creating time-series data, the processing unit 101 samples the basic feature data BD for a predetermined task data acquisition period (e.g., six months) from the basic feature data BD stored in the storage unit 102. For example, the processing unit 101 may perform sampling by sliding window processing.

[0098] Specifically, the processing unit 101 may sample the basic feature data BD for a predetermined task data acquisition period (e.g., six months) going back from the date and time when the subject last performed the predetermined task (measurement end date and time). Alternatively, the processing unit 101 may sample the basic feature data BD acquired during the period from the date and time when the subject first performed the predetermined task (measurement start date and time) until the predetermined task data acquisition period (e.g., six months) has elapsed. The measurement end date and time or the measurement start date and time may be input (set) by the operator operating the input unit 40. Alternatively, the processing unit 101 may sample the basic feature data BD for the predetermined task data acquisition period going back from any date and time input (set) by the operator operating the input unit 40, or may sample the basic feature data BD acquired during the period from the set date and time until the predetermined task data acquisition period has elapsed.

[0099] In addition, the process of selecting statistics may be omitted from the process of acquiring explanatory variables of learning data, in which case step S33 (filtering process) in FIG.

[0100] The label data (correct labels) assigned to the subject are used as the objective variable of the training data. The label data indicates changes in the assessment values ​​of the subject's cognitive function collected over a predetermined period (e.g., two years). Hereinafter, the period during which data is collected to create the label data (objective variable) may be referred to as the "label data collection period." The label data collection period is an example of a "first predetermined period." Note that the cognitive function prediction system 100 predicts changes in the cognitive function of the subject SJ after a period corresponding to the label data collection period has elapsed from the measurement end date and time, for example.

[0101] In this embodiment, the label data indicates negative or positive. Therefore, the prediction result output from the first machine learning model ML1 indicates negative or positive. Specifically, positive label data is assigned to subjects whose degree of cognitive decline is greater than a reference value, and negative label data is assigned to subjects whose degree of cognitive decline is equal to or less than the reference value. The reference value may be a fixed value or may be arbitrarily set by the operator.

[0102] For example, the label data may be generated based on time-series data of cognitive function test scores measured during a label data collection period (e.g., two years). The cognitive function test may be, for example, the Mini-Mental State Examination (MMSE), the Revised Hasegawa Cognitive Assessment, or the Montreal Cognitive Assessment (MoCA). The subject's cognitive function test score is an example of an "evaluation value of the subject's cognitive function."

[0103] Specifically, the label data may indicate whether the slope of a line (simple regression line) obtained by performing simple regression analysis on time-series data of cognitive function test scores is below a threshold. In this case, positive label data is assigned to subjects whose slope of the simple regression line is below the threshold, and negative label data is assigned to subjects whose slope is not below the threshold. The threshold is set to, for example, -0.60 points / year. -0.60 points / year indicates a decrease of 0.6 points in the cognitive function test score in one year (a decrease of 1.2 points in two years).

[0104] The threshold may be set arbitrarily. For example, the threshold may be selected from the range of -1.0 [points / year] to 0.0 [points / year]. Specifically, the threshold may be set according to the purpose.

[0105] For example, if the threshold is set to 0.0 points / year, subjects whose cognitive function test scores have declined even slightly over two years will be judged as positive. As a result, if the threshold is set to 0.0 points / year, the first machine learning model ML1 will output a positive prediction result for the subject SJ, whose cognitive function is predicted to decline even slightly in the future. Therefore, the cognitive function prediction system 100 can predict cognitive function decline relatively early.

[0106] Furthermore, when the threshold is set to -1.0 points / year, subjects whose cognitive function test scores have declined significantly over two years are determined to be positive. As a result, when the threshold is set to -1.0 points / year, the first machine learning model ML1 outputs a positive prediction result for the subject SJ, whose cognitive function is predicted to decline significantly in the future. Therefore, the cognitive function prediction system 100 is able to predict a more severe decline in cognitive function.

[0107] 9A is a diagram showing an example of time-series data of MMSE scores and a simple regression line. FIG. 9B is a diagram showing another example of time-series data of MMSE scores and a simple regression line. In FIG. 9A and FIG. 9B, the horizontal axis indicates the day on which the MMSE scores were measured. The vertical axis indicates the MMSE scores.

[0108] In the example shown in Figure 9(a), the MMSE score has decreased by about 5 points over a period of about two years. Therefore, the slope of the simple regression line is about -2.5 [points / year]. Therefore, if the threshold is set to -0.60 [points / year], positive label data will be assigned to the subject whose MMSE score shown in Figure 9(a) was measured.

[0109] In the example shown in Figure 9(b), the MMSE score has remained almost unchanged over a period of approximately two years, so the slope of the simple regression line is approximately 0.0 [points / year]. Therefore, if the threshold is set to -0.60 [points / year], negative label data is assigned to the subject whose MMSE score shown in Figure 9(b) was measured.

[0110] Embodiment 1 of the present invention has been described above with reference to FIGS. 1 to 8, 9(a) and 9(b). According to embodiment 1, future changes in cognitive function of an evaluation subject SJ can be predicted using data acquired from the evaluation subject SJ performing a predetermined task. Because it is not difficult for the evaluation subject SJ to perform predetermined tasks on a daily basis, data used to predict changes in cognitive function can be easily acquired compared to MRI data, biomarkers, clinical test records, etc. Therefore, according to embodiment 1, future changes in cognitive function can be more easily predicted compared to methods that predict changes in cognitive function using MRI data, biomarkers, clinical test records, etc.

[0111] Furthermore, according to embodiment 1, since features are extracted from multivariate time series data, prediction accuracy can be stabilized even if there is variation in the number of times a specific task is performed for each evaluation subject SJ or the length of the period from the measurement start date and time to the measurement end date and time.

[0112] In the first embodiment, the learning data is created using as explanatory variables data acquired from subjects who performed a predetermined task multiple times within a predetermined task data acquisition period, but the subject may be made to perform the predetermined task only once. In this case, the cognitive function prediction system 100 (prediction unit 30) predicts future changes in the cognitive function of the subject SJ using data acquired from the subject SJ who performed the predetermined task once.

[0113] Specifically, when a predetermined task is performed once, for example, the nine types of basic feature values ​​shown in Table 1 may be used as explanatory variables of the training data. Furthermore, the nine types of basic feature values ​​shown in Table 1 may be used as data to be input to the first machine learning model ML1 when predicting a change in cognitive function. In this case, steps S31 to S33 shown in FIG. 8 may be omitted.

[0114] In addition, in embodiment 1, the nine types of basic features shown in Table 1 were used to predict future changes in cognitive function of the subject SJ, but it is also possible to predict future changes in cognitive function of the subject SJ using some of the nine types of basic features.

[0115] Furthermore, in the first embodiment, the evaluation subject SJ is imaged and the motion of the evaluation subject SJ is detected, but the configuration for detecting the motion of the evaluation subject SJ is not particularly limited. For example, the motion detection unit 21 may have a mat-shaped pressure-sensitive switch instead of the image capture unit 211 and the motion capture unit 212. In this case, the mat-shaped pressure-sensitive switch is laid on the floor and outputs a signal corresponding to the foot movement of the evaluation subject SJ. The processing unit 101 calculates the first to fourth basic feature amounts based on the output of the pressure-sensitive switch.

[0116] Furthermore, in embodiment 1, future changes in the cognitive function of the subject SJ are predicted using the characteristics of the subject SJ's movements and the characteristics of the subject SJ's answers to the cognitive tasks, but future changes in the cognitive function of the subject SJ may be predicted using only the characteristics of the subject SJ's movements, or may be predicted using only the characteristics of the subject SJ's answers to the cognitive tasks. When only the characteristics of the subject SJ's movements are used, the answer detection unit 22 may be omitted. When only the characteristics of the subject SJ's answers are used, the movement detection unit 21 may be omitted.

[0117] Furthermore, in the first embodiment, nine types of basic feature amounts are exemplified, but the basic feature amounts are not limited to those shown in Table 1. For example, the average size of the raised thighs when performing a motor task and the average size of the raised thighs when performing a dual task may be used as the basic feature amounts. Furthermore, the standard deviation of the size of the raised thighs of the subject SJ performing stepping (motor task) may be used as the basic feature amounts. Alternatively, the standard deviation of the size of the raised thighs when performing a motor task and the standard deviation of the size of the raised thighs when performing a dual task may be used as the basic feature amounts.

[0118] In addition, in embodiment 1, future changes in cognitive function of the subject SJ are predicted using basic features, but future changes in cognitive function of the subject SJ may also be predicted using time series data of each joint J that constitutes the skeleton SK.

[0119] [Embodiment 2] Next, a second embodiment of the present invention will be described with reference to Figures 10 to 12. However, differences from embodiment 1 will be described, and descriptions of the same aspects as embodiment 1 will be omitted. In embodiment 2, the machine learning model is different from embodiment 1.

[0120] Fig. 10 is a block diagram showing the configuration of a cognitive function prediction system 100 according to embodiment 2. Figs. 11 and 12 are diagrams showing the flow of processing executed by the prediction unit 30 (processing unit 101) included in the cognitive function prediction system 100 according to embodiment 2. In detail, Fig. 11 shows the flow of processing for collecting data to be used for prediction. Fig. 12 shows the flow of processing for making prediction.

[0121] As shown in FIG. 10, in the second embodiment, the storage unit 102 stores answer feature data FD and skeleton sequence SD, which will be described later with reference to FIG. 11, in association with identification information assigned to each evaluation target SJ.

[0122] The storage unit 102 also stores a second machine learning model ML2. The second machine learning model ML2 is a computer program that predicts future changes in the cognitive function of the subject SJ and outputs the prediction results. The second machine learning model ML2 is constructed by machine learning training data. In the second embodiment, a neural network is used as the algorithm of the second machine learning model ML2. The neural network is, for example, a neural network that performs deep learning.

[0123] In the second embodiment, the answer feature data FD and the skeleton sequence SD are input to the second machine learning model ML2. The neural network of the second machine learning model ML2 is constructed to extract features of the answer of the subject SJ to the cognitive task from the answer feature data FD, extract features of the movement of the subject SJ from the skeleton sequence SD, predict future changes in the cognitive function of the subject SJ based on the extracted answer features and movement features, and output the prediction result.

[0124] Steps S11 to S13 and steps S41 to S43 in Fig. 11 are substantially the same as steps S11 to S13 and steps S21 to S23 in Fig. 8, and therefore detailed description thereof will be omitted. In the second embodiment, a skeleton sequence SD (time-series data of skeleton SK) is acquired as the motion detection data (steps S42 and S43).

[0125] 11, after acquiring the dual-task data, the processing unit 101 creates answer feature data FD based on the answer detection data (step S44). The answer feature data FD indicates features of the answer of the subject SJ to the cognitive task. For example, the answer feature data FD may indicate the sixth to ninth basic feature amounts shown in Table 1.

[0126] As in the first embodiment, the subject SJ performs a predetermined task multiple times during a predetermined task data acquisition period. The processing unit 101 associates the skeleton sequence SD and answer feature data FD from each execution with identification information assigned to each subject SJ and stores them in the storage unit 102 (see FIG. 10 ). In other words, the skeleton sequence SD and answer feature data FD of the subject SJ are collected during a predetermined task data acquisition period (e.g., six months). The prediction unit 30 (processing unit 101) predicts future changes in the cognitive function of the subject SJ based on the skeleton sequence SD and answer feature data FD acquired multiple times during the predetermined task data acquisition period.

[0127] 12 starts in response to, for example, a command to output a prediction result by an operator operating the input unit 40. When the processing unit 101 starts the processing shown in FIG. 12 , the processing unit 101 inputs the skeleton sequence SD and answer feature data FD acquired during a predetermined task data acquisition period (e.g., six months) into the second machine learning model ML2 (step S51). As a result, a prediction result is output from the second machine learning model ML2.

[0128] Finally, the processing unit 101 controls the result presentation unit 60 to present the prediction result obtained from the second machine learning model ML2 (step S52), thereby completing the processing shown in FIG.

[0129] As in embodiment 1, the prediction unit 30 (second machine learning model ML2) may predict future changes in the cognitive function of the subject SJ based on data obtained from the subject SJ who has performed a specified task once.

[0130] Next, the second machine learning model ML2 will be described. The neural network of the second machine learning model ML2 may include a component that extracts answer features from the answer feature data FD, a component that extracts action features from the skeleton sequence SD, and a component that combines or fuses the extracted answer features and action features to output a prediction result.

[0131] Specifically, the neural network of the second machine learning model ML2 may include a component that convolves the answer feature data FD and extracts features of the answer of the subject SJ to the cognitive task. A convolutional neural network (CNN) may be used to convolve the answer feature data FD.

[0132] Furthermore, when the skeleton SK is a three-dimensional human skeletal model, the neural network of the second machine learning model ML2 may include a component that extracts, as the movement characteristics of the evaluation subject SJ, movement characteristics that indicate the characteristics of the spatial positional relationships between the multiple joints J that constitute the skeleton SK and the characteristics of the temporal fluctuations of each of the multiple joints J that constitute the skeleton SK. In other words, the neural network of the second machine learning model ML2 may include a component that extracts the spatiotemporal characteristics of the multiple joints J that constitute the skeleton SK. A graph convolutional neural network such as a spatiotemporal graph convolutional neural network is used to extract the movement characteristics of the evaluation subject SJ.

[0133] Specifically, when the skeleton SK is a three-dimensional human skeletal model, the skeleton sequence SD indicates the three-dimensional coordinates of each joint J of the skeleton SK in time series. In other words, the skeleton sequence SD indicates time-series data of the three-dimensional coordinates of each joint J.

[0134] The second machine learning model ML2 generates, for each frame included in the skeleton sequence SD, a spatial graph showing the spatial (three-dimensional) positional relationships between the multiple joints J that make up the skeleton SK, and also generates a time graph showing the fluctuations of the same joint J between adjacent frames (i.e., the temporal fluctuations of the joint J).The second machine learning model ML2 then extracts features of the spatial positional relationships between the multiple joints J that make up the skeleton SK by convolving the spatial graph.The second machine learning model ML2 also extracts features of the fluctuations between each joint J that make up the skeleton SK by convolving the time graph.As a result, spatiotemporal features of the multiple joints J that make up the skeleton SK are extracted as movement features.

[0135] Alternatively, when the skeleton SK is a three-dimensional human skeletal model, the processing unit 101 may divide the skeleton sequence SD into a skeleton sequence SD1 for performing a motor task and a skeleton sequence SD2 for performing a dual task, and input them into the second machine learning model ML2.

[0136] In this case, the neural network of the second machine learning model ML2 may include a component that extracts first movement features that are spatiotemporal features of the multiple joints J that make up the skeleton SK from the skeleton sequence SD1 when performing a motor task, a component that extracts second movement features that are spatiotemporal features of the multiple joints J that make up the skeleton SK from the skeleton sequence SD2 when performing a dual task, and a component that acquires a feature difference that indicates the difference between the first movement feature and the second movement feature. Furthermore, the neural network of the second machine learning model ML2 may further include a component that combines or fuses the answer feature extracted by convolving the answer feature data FD, the first movement feature, the second movement feature, and the feature difference to output a prediction result.

[0137] Alternatively, if the movement required by the motor task is periodic (e.g., stepping), the processing unit 101 may decompose the time-series data (skeleton sequence SD) of the skeleton SK representing the two-dimensional human skeletal model or the three-dimensional human skeletal model into periods (e.g., for each stepping period) to generate skeleton data for each period, align the phase of the skeleton data for each period, and generate input data to be input to the second machine learning model ML2. In this case, the neural network of the second machine learning model ML2 may include a component that extracts movement characteristics of the evaluation subject SJ by convolving the input data into spatial, phase, and period dimensions.

[0138] Embodiment 2 of the present invention has been described above with reference to Figures 10 to 12. According to Embodiment 2, similar to Embodiment 1, it is possible to predict future changes in cognitive function of an evaluation subject SJ using data acquired from the evaluation subject SJ performing a predetermined task. Therefore, similar to Embodiment 1, it is possible to more easily predict future changes in cognitive function compared to methods that predict changes in cognitive function using MRI data, biomarkers, clinical test records, etc.

[0139] In embodiment 2, the second machine learning model ML2 extracted the movement characteristics of the subject SJ using information on all joints J that make up the skeleton SK, but the second machine learning model ML2 may also extract the movement characteristics of the subject SJ using information on some of the joints J that make up the skeleton SK.

[0140] The embodiments of the present invention have been described above with reference to the drawings (FIGS. 1 to 12). However, the present invention is not limited to the above embodiments and can be implemented in various forms without departing from the spirit of the present invention. Furthermore, the components disclosed in the above embodiments can be modified as appropriate. For example, some of the components shown in one embodiment may be added to the components of another embodiment, or some of the components shown in one embodiment may be deleted from the embodiment.

[0141] The drawings mainly show each component in a schematic manner to facilitate understanding of the invention, and the thickness, length, number, spacing, etc. of each component shown in the drawings may differ from the actual ones due to the convenience of creating the drawings. Furthermore, the configuration of each component shown in the above embodiment is merely an example and is not particularly limited, and it goes without saying that various modifications are possible within a range that does not substantially deviate from the effects of the present invention.

[0142] For example, in the embodiment described with reference to Figures 1 to 12, the change in cognitive function of the subject SJ after two years was predicted, but there is no particular limitation as to how far into the future the change in cognitive function is predicted. For example, the change in cognitive function after one year or after three years may be predicted.

[0143] 1 to 12, whether or not the cognitive function of the subject SJ is declining is predicted, but the prediction of changes in the cognitive function of the subject SJ is not limited to the prediction of cognitive decline. For example, it may be possible to predict whether or not the future subject SJ will develop dementia or mild cognitive impairment (MCI).

[0144] In this case, the label data may be created based on time-series data of the scores of the cognitive function test, or may be created based on time-series data of MRI data, biomarkers, clinical test records, definitive diagnoses, etc. The subject's MRI data, biomarkers, clinical test records, definitive diagnoses, etc. are examples of "evaluation values ​​of the subject's cognitive function."

[0145] For example, if the MMSE score is 23 points or less, it is determined that there is a suspicion of dementia, and if the MMSE score is more than 23 points but less than 27 points, it is determined that there is a suspicion of mild cognitive impairment. Furthermore, if the MMSE score is higher than 27 points, it is determined that the subject is non-demented (healthy). Therefore, for example, by assigning positive label data to a subject whose MMSE score on a certain measurement day is 28 points or more and whose MMSE score two years later is 27 points or less, and by assigning negative label data to a subject whose MMSE score on a certain measurement day is 28 points or more and whose MMSE score two years later is 28 points or more, it is possible to predict whether the subject SJ will develop mild cognitive impairment or dementia two years later. Alternatively, by assigning positive label data to subjects whose MMSE score on a certain measurement day is 24 points or more and whose MMSE score two years later is 23 points or less, and by assigning negative label data to subjects whose MMSE score on a certain measurement day is 24 points or more and whose MMSE score two years later is 24 points or more, it becomes possible to predict whether or not the subject SJ will develop dementia two years later.

[0146] Alternatively, positive label data may be assigned to a subject whose MRI data on a certain measurement day indicates a healthy individual or mild cognitive impairment, and whose MRI data two years later indicates dementia, and negative label data may be assigned to a subject whose MRI data on a certain measurement day indicates a healthy individual or mild cognitive impairment, and whose MRI data two years later indicates a healthy individual or mild cognitive impairment. In this case, it is possible to predict whether or not the subject SJ will develop dementia two years later. Alternatively, positive label data may be assigned to a subject whose MRI data on a certain measurement day indicates a healthy individual, and whose MRI data two years later indicates mild cognitive impairment or dementia, and negative label data may be assigned to a subject whose MRI data on a certain measurement day indicates a healthy individual, and whose MRI data two years later indicates a healthy individual. In this case, it is possible to predict whether or not the subject SJ will develop mild cognitive impairment or dementia two years later. Similarly, by creating label data based on time-series data such as MRI data, biomarkers, clinical test records, or definitive diagnoses, it is possible to predict whether the subject SJ will develop mild cognitive impairment or dementia, for example, two years from now.

[0147] In addition, in the embodiment described with reference to Figures 1 to 12, in order to predict future changes in the cognitive function of the subject SJ, characteristics of the answers were extracted from the answers of the subject SJ performing a cognitive task and the answers of the subject SJ performing a dual task, but the prediction unit 30 may extract characteristics of the answers of the subject SJ to the cognitive task from only one of these answers.

[0148] Furthermore, in the embodiment described with reference to Figures 1 to 12, in order to predict future changes in the cognitive function of the subject SJ, movement characteristics were extracted from the movement of the subject SJ performing a motor task and the movement of the subject SJ performing a dual task, but the prediction unit 30 may also extract movement characteristics of the subject SJ from only the movement of the subject SJ performing a dual task.

[0149] 1 to 12, the characteristics of the responses of the subject SJ are used to predict future changes in the cognitive function of the subject SJ. However, the characteristics of the responses of the subject SJ may be omitted. In other words, the prediction unit 30 may predict future changes in the cognitive function of the subject SJ from only the characteristics of the behavior of the subject SJ. In this case, the response detection unit 22 may be omitted.

[0150] 1 to 12, the characteristics of the movements of the subject SJ are used to predict future changes in the cognitive function of the subject SJ, but the characteristics of the movements of the subject SJ may be omitted. In other words, the prediction unit 30 may predict future changes in the cognitive function of the subject SJ based only on the characteristics of the responses of the subject SJ. In this case, the movement detection unit 21 may be omitted.

[0151] Furthermore, in the embodiment described with reference to Figures 1 to 12, the subject SJ was asked to perform a specified task once per measurement, but the subject SJ may also be asked to perform a specified task multiple times in succession per measurement.

[0152] Furthermore, in the embodiment described with reference to Figures 1 to 12, the subject SJ was asked to perform the cognitive task, motor task, and dual task in this order, but the order in which the subject SJ is asked to perform the cognitive task, motor task, and dual task can be reversed.

[0153] Furthermore, in the embodiment described with reference to FIGS. 1 to 12, the subject SJ was asked to perform a cognitive task, a motor task, and a dual task. However, the tasks that the subject SJ is asked to perform may include at least a dual task. For example, the subject SJ may be asked to perform only a dual task. Alternatively, the subject SJ may be asked to perform a motor task and a dual task, or a cognitive task and a dual task. Furthermore, the tasks that the subject SJ is asked to perform may be tasks that require the subject SJ to perform at least one of the cognitive task, the motor task, and the dual task two or more times. For example, the tasks that the subject SJ is asked to perform may include a dual task two or more times.

[0154] Furthermore, in the embodiment described with reference to Figures 1 to 12, the skeleton SK is composed of 20 joints J, but the number of joints J constituting the skeleton SK is not particularly limited as long as it is possible to represent the entire body of the person being evaluated SJ; for example, the skeleton SK may be composed of 17 or 25 joints J.

[0155] 1 to 12, the skeleton SK represents the entire body of the person being evaluated SJ, but the movement detection unit 21 may generate a skeleton SK that represents a part of the entire body of the person being evaluated SJ. For example, the movement detection unit 21 may generate a skeleton SK that represents the lower limbs of the person being evaluated SJ. Alternatively, the movement detection unit 21 may generate a skeleton SK that represents the right or left half of the body of the person being evaluated SJ, or may generate a skeleton SK that represents the knees, hips, etc. of the person being evaluated SJ.

[0156] 1 to 12, the motion detection unit 21 generates the skeleton sequence SD, but the prediction unit 30 may generate the skeleton sequence SD based on the imaging signal output from the imaging unit 211. In this case, the motion capture unit 212 is omitted.

[0157] 1 to 12, the task presenter 10 has a display, but the configuration of the task presenter 10 is not particularly limited as long as it can present tasks to be performed by the subject SJ. For example, the task presenter 10 may have an audio output device.

[0158] 1 to 12, the answer detection unit 22 has an answer switch for the left hand and an answer switch for the right hand, but the configuration of the answer detection unit 22 is not particularly limited as long as it can detect answers to the cognitive tasks. For example, the answer detection unit 22 may have a gaze direction detection device or a sound collector.

[0159] When using a gaze direction detection device, for example, the answer of the subject SJ can be obtained based on the direction in which the subject SJ looks while the answer candidate presentation screen 12b shown in Figures 3(b) and 4 is displayed. A known gaze direction detection technology can be used for the gaze direction detection device. For example, the gaze direction detection device includes a near-infrared LED and an imaging device. The near-infrared LED irradiates the eyes of the subject SJ with near-infrared light. The imaging device captures an image of the eyes of the subject SJ. The processing unit 101 analyzes the image captured by the imaging device to detect the position of the pupils (gaze direction) of the subject SJ.

[0160] When a sound collector is used, for example, the answer of the evaluation subject SJ can be acquired based on the voice uttered by the evaluation subject SJ in response to the answer candidate presentation screen 12b shown in Figure 3(b) and Figure 4. For example, the processing unit 101 can acquire the answer of the evaluation subject SJ by converting the voice uttered by the evaluation subject SJ into text data by speech recognition processing.

[0161] When a sound collector is used, the cognitive task is not limited to a question that requires the subject J to select one of two possible answers. For example, the subject SJ may be asked to answer a calculation problem. When a sound collector is used, the cognitive task may also be a question that requires the subject SJ to answer a word. A question that requires the subject SJ to answer a word may be, for example, a "shiritori question," a question that requires the subject to list words (e.g., words) that begin with a sound (letter) arbitrarily selected from the Japanese alphabet, or a question that requires the subject SJ to list words (e.g., words) that begin with a letter arbitrarily selected from the alphabet.

[0162] The present invention is useful for a technique for predicting future changes in cognitive function using dual tasks.

[0163] 20: Detection unit 21: Action detection unit 22: Answer detection unit 30: Prediction unit 100: Cognitive function prediction system 101: Processing unit 102: Memory unit BD: Basic feature data FD: Answer feature data ML1: First machine learning model ML2: Second machine learning model SD: Skeleton sequence SJ: Evaluation subject

Claims

1. A cognitive function prediction system comprising: a detection unit that detects at least one of the movements of a person to be evaluated or the person to be evaluated's responses to cognitive tasks from the person to be evaluated while the person is performing a specified task; and a prediction unit that predicts future changes in the cognitive function of the person to be evaluated based on detection data indicating the results of the detection by the detection unit, wherein the specified task includes a dual task in which a first motor task and a first cognitive task are simultaneously assigned, the movements of the person to be evaluated include the movements of the person to be evaluated while performing the dual task, and the responses of the person to be evaluated include the response of the person to the first cognitive task, and the prediction unit includes a machine learning model that outputs predicted results of changes in the future cognitive function of the person to be evaluated, and the machine learning model is constructed by machine learning of training data including label data indicating changes in the evaluation values ​​of the subject's cognitive function collected within a first specified period.

2. The cognitive function prediction system of claim 1, wherein the prediction unit predicts whether the future cognitive function of the person being evaluated will decline as a change in the person's future cognitive function.

3. The cognitive function prediction system of claim 1, wherein the prediction unit predicts whether the subject will develop mild cognitive impairment or dementia in the future as a change in the subject's cognitive function.

4. A cognitive function prediction system as described in any one of claims 1 to 3, wherein the subject performs the specified task multiple times within a second specified period, and the prediction unit predicts future changes in the cognitive function of the subject based on the detection data for each of the multiple times.

5. A cognitive function prediction system described in any one of claims 1 to 3, wherein the specified task further includes an exercise task that imposes a second exercise task, and the movements of the subject further include movements of the subject performing the exercise task.

6. A cognitive function prediction system as described in any one of claims 1 to 3, wherein the predetermined task further includes a cognitive task that imposes a second cognitive task, and the responses of the subject further include the responses of the subject to the second cognitive task.

Images