Information processing device, information processing method, and program

Abstract

To provide an information processing device capable of appropriately evaluating the level of motor skill proficiency. [Solution] The control unit 12 (information processing device) according to this disclosure includes: a subject brainwave acquisition unit 121 that acquires subject brainwaves showing the brainwaves of a subject when the subject performs a predetermined movement; a subject motion acquisition unit 122 that acquires subject motions showing the actions of a subject when the subject performs a predetermined movement; and a proficiency evaluation unit 123 that, using a reference motion as the reference motion when performing a predetermined movement, evaluates the proficiency of the subject motion higher the smaller the amplitude of the subject brainwaves when the absolute value of the difference between the amount of movement of the reference motion and the amount of movement of the subject motion is less than a first threshold.

Description

Technical Field

[0005]

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Techniques for measuring the proficiency of driving a vehicle or performing an operation in sports are known. As a related technique, Patent Document 1 discloses a proficiency evaluation method for evaluating the proficiency with respect to a target operation from the movements of the trainer himself / herself.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the present disclosure, an ability indicating the degree of coordination between the neural activity of the brain and the movement of the body is defined as the motor ability. The motor ability is an ability indicating whether the movement intended by the mover is realized. It can be said that the higher the motor ability of the mover, the more the movement intended by the mover is realized. Also, the higher the motor ability, the higher the proficiency of the movement. ​​​​​​​ [Means for solving the problem]

[0007] The information processing device relating to this disclosure is A subject electroencephalogram (EEG) acquisition unit that acquires subject electroencephalograms showing the brainwaves of a subject when they perform a prescribed exercise, A subject motion acquisition unit that acquires subject motions that indicate the actions taken by the subject when the subject performs the predetermined exercise, The system includes a proficiency evaluation unit that, using the standard for performing the predetermined movement as the standard movement, evaluates the proficiency of the subject's movement higher the smaller the amplitude of the subject's electroencephalogram is, if the absolute value of the difference between the amount of movement of the standard movement and the amount of movement of the subject's movement is less than a first threshold.

[0008] The information processing method relating to this disclosure is: A subject brainwave acquisition step involves acquiring subject brainwaves that show the brainwaves of the subject when they perform a prescribed exercise, and A subject motion acquisition step, which acquires subject motions that indicate the actions performed by the subject when the subject performs the predetermined exercise, The method includes a proficiency evaluation step in which, using the standard for performing the predetermined movement as the standard movement, if the absolute value of the difference between the amount of movement of the standard movement and the amount of movement of the subject movement is less than a first threshold, the smaller the amplitude of the subject's electroencephalogram, the higher the proficiency of the subject movement is evaluated.

[0009] The program related to this disclosure is A subject brainwave acquisition step involves acquiring subject brainwaves that show the brainwaves of the subject when they perform a prescribed exercise, and A subject motion acquisition step, which acquires subject motions that indicate the actions performed by the subject when the subject performs the predetermined exercise, The computer is instructed to perform a proficiency evaluation step in which, using the standard for performing the predetermined movement as the standard movement, if the absolute value of the difference between the amount of movement of the standard movement and the amount of movement of the subject movement is less than a first threshold, the smaller the amplitude of the subject's brainwave, the higher the proficiency of the subject movement is evaluated. [Effects of the Invention]

[0010] The information processing device, information processing method, and program described herein enable appropriate evaluation of motor skill proficiency. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a block diagram showing the configuration of the proficiency visualization system described herein. [Figure 2] Figure 2 shows an example of the appearance of the proficiency visualization system described herein. [Figure 3] Figure 3 shows an example of a proficiency table for evaluating proficiency related to this disclosure. [Figure 4] Figure 4 is a flowchart showing the processing flow performed by the control unit according to this disclosure. [Figure 5] Figure 5 is a flowchart showing the processing flow performed by the control unit according to this disclosure. [Figure 6] Figure 6 is a flowchart showing the processing flow performed by the control unit according to this disclosure. [Modes for carrying out the invention]

[0012] Embodiments of the present disclosure will be described in detail below with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals. For clarity of explanation, redundant explanations will be omitted where necessary.

[0013] When evaluating motor proficiency using the related technologies described above, in addition to the issues mentioned above, there are also the following challenges. For example, the related technologies described above attempt to quantify proficiency by performing the target motor movement multiple times and calculating the variance of the movement. However, such technologies require multiple measurements. Therefore, in addition to the issues mentioned earlier, there is the challenge of the extra effort required for measurement.

[0014] In addition, as a known technique, there is a method that utilizes the movement preparation potential, which is a type of event-related potential. However, this method reflects higher-order movement activities and does not serve as an index for directly evaluating the proficiency of movement corresponding to a specific part. This is because event-related potentials are activity indices of higher-order brain responses calculated by averaging the processing of each channel, and do not perform measurements of brain nerve activities that reflect the functional localization regarding body parts (measurements of brain nerve activities corresponding to specific body parts). The present disclosure addresses these issues.

[0015] (Proficiency Visualization System 1) Referring to FIGS. 1 and 2, the proficiency visualization system 1 according to the present disclosure will be described. FIG. 1 is a block diagram showing the configuration of the proficiency visualization system 1. FIG. 2 is a diagram showing an example of the appearance of the proficiency visualization system 1. Note that the one-way arrows and two-way arrows shown in FIG. 1 simply indicate the flow of information (such as data or signals). The flow of information indicated by the one-way arrows in FIG. 1 does not exclude the bi-directionality of the information.

[0016] The proficiency visualization system 1 includes an input unit 11, a control unit 12, a storage unit 13, and an output unit 14. As an example of the input unit 11, in FIG. 2, an electroencephalogram detection unit 111 attached to the head of the subject and motion detection units 112a to 112d attached to the limbs are shown. Also, in FIG. 2, an example is shown in which the control unit 12, the storage unit 13, and the output unit 14 are provided in the terminal device 5 used by the subject.

[0017] The proficiency visualization system 1 is a system capable of visualizing the proficiency of the actions of a subject performing a predetermined movement. The input unit 11 detects the electroencephalogram and actions of the subject and transmits the detection results to the terminal device 5. The control unit 12 evaluates the proficiency of the actions of the subject using the detection results received from the input unit 11 and the data stored in the storage unit 13. Further, the control unit 12 causes the evaluation result to be output from the output unit 14. Thereby, the proficiency visualization system 1 visualizes the proficiency of the actions of the subject.

[0018] The target individuals are those whose proficiency in a movement is evaluated using the proficiency visualization system 1. These individuals may include, but are not limited to, athletes, rehabilitation users, drivers, or fitness enthusiasts. Any individual whose movement proficiency is to be visualized is acceptable. The type of exercise or movement is arbitrary. In the following explanation, an example of using the proficiency visualization system 1 to visualize proficiency in golf movements may be used.

[0019] As shown in Figure 2, the electroencephalogram detection unit 111, the motion detection units 112a to 112d, and the terminal device 5 are connected via a network N. Network N is a wired or wireless communication line. Network N may be a network used for short-range communication, such as Bluetooth®, or a network used for long-range communication, such as a mobile phone line.

[0020] Terminal device 5 is an information processing device that performs the processing related to this disclosure. The user of terminal device 5 may be the subject, or a person who manages the subject's proficiency in actions. Such a person may be, for example, a sports trainer or a medical professional.

[0021] The terminal device 5 may be, but is not limited to, a smartphone, mobile phone, tablet, or PC (Personal Computer). The terminal device 5 may be any information processing device capable of performing the processing described herein. The terminal device 5 may also be a wearable device such as a smartwatch or smart glasses. Alternatively, the terminal device 5 may be a predetermined server capable of performing the processing described herein. This server may be a server located in a dedicated data center or a virtual server in a cloud computing environment.

[0022] Terminal device 5 includes a processor and memory, although these are not shown in the diagram. A storage unit 13, which is a storage device, stores a computer program on which the processing described herein is implemented. The processor can load the computer program from the storage device into memory and execute the computer program. In this way, the processor realizes the functions of the control unit 12, including the subject electroencephalogram acquisition unit 121, the subject motion acquisition unit 122, the proficiency evaluation unit 123, the display control unit 124, and the warning unit 125.

[0023] Alternatively, the subject brainwave acquisition unit 121, subject motion acquisition unit 122, proficiency evaluation unit 123, display control unit 124, and warning unit 125 of the control unit 12 may each be implemented with dedicated hardware. Furthermore, some or all of each component may be implemented by general-purpose or dedicated circuits, processors, etc., or combinations thereof. These may be configured by a single chip or by multiple chips connected via a bus. Some or all of each component may be implemented by a combination of the above-mentioned circuits, etc., and programs. In addition, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), quantum processor (quantum computer control chip), etc. can be used as the processor.

[0024] Furthermore, if some or all of the components of the proficiency visualization system 1 are implemented by multiple information processing devices or circuits, these multiple information processing devices or circuits may be centrally located or distributed. For example, the information processing devices or circuits may be implemented in a form in which each is connected via a communication network, such as a client-server system or a cloud computing system.

[0025] For example, as shown in Figure 2, the proficiency visualization system 1 may have a terminal device 5 equipped with a control unit 12, a storage unit 13, and an output unit 14, while the electroencephalogram detection unit 111 and motion detection units 112a to 112d that constitute the input unit 11 may be provided separately from the terminal device 5. Note that Figure 2 shows an example in which the terminal device 5 is equipped with a control unit 12, a storage unit 13, and an output unit 14, but it is not limited to this. For example, the storage unit 13 and the output unit 14 may be provided in a device other than the terminal device 5. The following describes each component of the proficiency visualization system 1.

[0026] As shown in Figure 1, the input unit 11 includes an electroencephalogram (EEG) detection unit 111 and a motion detection unit 112. The EEG detection unit 111 detects the subject's electroencephalogram (EEG), which shows the brainwaves of the subject when they perform a predetermined movement. The EEG detected by the EEG detection unit 111 is, for example, a spontaneous EEG. A spontaneous EEG reflects the neuronal activity of the subject's motor-related areas, including the premotor cortex. By analyzing the spontaneous EEG, it is possible to understand the subject's readiness for movement and the brain activity trends associated with movement. The EEG detection unit 111 is, for example, an EEG sensor. The EEG detection unit 111 may also detect EEGs other than spontaneous EEGs.

[0027] The brain regions targeted for detection of spontaneous electroencephalograms (EEGs) are, for example, the areas corresponding to limb movements from the frontal lobe to the parietal lobe. The detection method may be invasive or non-invasive. An invasive method might involve directly implanting electrodes into the brain. A non-invasive method might involve placing electrodes on the scalp. Figure 2 shows an example of a non-invasive method involving the placement of electrodes on the scalp.

[0028] The motion detection unit 112 detects the subject's movements, which indicate the actions the subject takes when performing a predetermined movement. Figure 2 shows four motion detection units 112a to 112d as examples of the motion detection unit 112. Here, each of the motion detection units 112a to 112d will be described as having the same configuration. Unless there is a particular need to distinguish between them, each of the motion detection units 112a to 112d will simply be referred to as the motion detection unit 112.

[0029] The motion detection unit 112 is an acceleration sensor capable of detecting acceleration in the X-axis, Y-axis, and Z-axis directions, for example. As shown in Figure 2, for example, motion detection units 112a to 112d are attached to the subject's right wrist, left wrist, right knee, and left knee, respectively. Each of the motion detection units 112a to 112d detects the acceleration of the part to which it is attached. As a result, the motion detection units 112a to 112d detect the movement of the subject's limbs, and thus the subject's motor state can be measured.

[0030] The motion detection units 112a to 112d transmit the detection results to the terminal device 5. Each of the motion detection units 112a to 112d may transmit the detection results at predetermined time intervals, or it may transmit them when it detects motion in each body part. Note that the attachment site is not limited to the sites shown in Figure 2. The motion detection unit 112 may be attached to other body parts, such as the upper arm or ankle, depending on the purpose of using the proficiency visualization system 1.

[0031] Furthermore, while an acceleration sensor was used as an example of the motion detection unit 112 here, it is not limited to this. Other sensors may be used as the motion detection unit 112. For example, a gyroscope, smartwatch, mobile phone, image analysis device, motion capture system, or a combination thereof may be used as the motion detection unit 112. Also, if necessary, an electromyograph sensor or camera may be used as the motion detection unit 112.

[0032] The control unit 12 comprises a subject electroencephalogram acquisition unit 121, a subject motion acquisition unit 122, a proficiency evaluation unit 123, a display control unit 124, and a warning unit 125. The control unit 12 is an information processing device that controls various functions of the proficiency visualization system 1.

[0033] The subject electroencephalogram (EEG) acquisition unit 121 acquires subject electroencephalograms (EEGs) that show the brain activity of a subject when they perform a predetermined movement. For example, the subject electroencephalogram acquisition unit 121 acquires spontaneous brain waves from the EEG detection unit 111 when the subject performs a predetermined movement, as subject electroencephalograms. Subject electroencephalograms are signal data that reflects the potential fluctuations associated with the subject's brain activity. The subject electroencephalogram acquisition unit 121 acquires subject electroencephalograms in a time-continuous format while the subject is performing a predetermined movement.

[0034] The subject motion acquisition unit 122 acquires subject motion, which indicates the actions the subject takes when performing a predetermined exercise. For example, the subject motion acquisition unit 122 acquires subject motion from the motion detection unit 112. If the motion detection unit 112 is an acceleration sensor, the subject motion is data that reflects the acceleration associated with the subject's body movements. The subject motion acquisition unit 122 acquires subject motion in a temporally continuous format while the subject is performing a predetermined exercise.

[0035] The proficiency evaluation unit 123 evaluates the proficiency of the subject's movements. The proficiency evaluation unit 123 evaluates the proficiency of the subject's movements based on the subject's brainwaves acquired by the subject's brainwave acquisition unit 121, the subject's movements acquired by the subject's movement acquisition unit 122, and a pre-set reference movement. For example, the reference movement is data that shows the ideal movement pattern in a given movement. The subject can improve their proficiency in the movement by performing movements that are close to the reference movement.

[0036] The standard movements may be pre-set by, for example, an administrator managing the proficiency visualization system 1. Here, we will explain an example in which an administrator sets standard movements using an administrator terminal. For example, the administrator sets standard movements by analyzing motion data when an expert or specialist performs a predetermined movement using the administrator terminal. First, the administrator terminal acquires acceleration data from an acceleration sensor that changes in accordance with the movements of the expert, etc. The administrator terminal then analyzes the acquired acceleration data.

[0037] For example, the administrator terminal uses acceleration data from skilled personnel to represent the movement of each body part as a vector and calculates the amount of movement for each body part. The amount of movement is a value that indicates the magnitude of the motion of each body part obtained from the acceleration data. The administrator terminal sets a reference operation based on this amount of movement. For example, the administrator terminal extracts the maximum value and change in the amount of movement of each body part based on the amount of movement. The administrator terminal sets a reference operation based on the extraction results and records the data of the set reference operation in the reference information recording unit 134 of the storage unit 13.

[0038] The reference movements may be set for each stage of the movement. A stage of movement is a division of the process from the start to the end of a given movement into multiple stages. The stages of movement may be set sequentially, for example, as "Stage 1," "Stage 2," "Stage 3," etc., or they may be set according to the progress of the movement, such as "Starting Stage," "Intermediate Stage," "Ending Stage." For example, each stage may be divided chronologically based on the elapsed time from the start of the movement, or it may be divided into units based on characteristic movements, such as acceleration data. If reference movements are set for each stage of the movement, the proficiency evaluation unit 123 may evaluate the proficiency for each stage of the movement.

[0039] For example, suppose the given motion is a golf swing, and each stage of the motion is divided into characteristic movements. For example, there are three stages of motion: "backswing" (the motion of swinging the club up), "downswing" (the motion of swinging the club down), and "follow-through" (the motion of swinging the club through after hitting the ball). A reference motion is set for each stage of the backswing, downswing, and follow-through, and is recorded in the reference information recording unit 134.

[0040] For example, the proficiency evaluation unit 123 uses the standard for performing a predetermined movement as the reference movement, and if the absolute value of the difference between the amount of movement of the reference movement and the amount of movement of the subject movement is less than the first threshold, it evaluates the proficiency of the subject movement higher the smaller the amplitude of the subject's electroencephalogram.

[0041] In evaluating proficiency, the proficiency evaluation unit 123 first compares the amount of movement of the standard movement with the amount of movement of the subject's movement. The proficiency evaluation unit 123 determines whether the absolute value of the difference between the amount of movement of the standard movement and the amount of movement of the subject's movement is less than the first threshold. If the absolute value of the difference between the amount of movement of the standard movement and the amount of movement of the subject's movement is less than the first threshold, the standard movement and the subject's movement can be said to be relatively similar movements. If the proficiency evaluation unit 123 determines that the absolute value of the difference is less than the first threshold, it evaluates proficiency higher the smaller the amplitude of the subject's electroencephalogram (EEG).

[0042] For example, the first threshold can be pre-set by an administrator or other person using the difference that allows for the assessment that the reference movement and the subject's movement are similar. In particular, in the field of sports, the appropriate way to move the body differs depending on the sport, so it is desirable that the first threshold be set flexibly.

[0043] Furthermore, the proficiency evaluation unit 123 may evaluate the proficiency of the subject's movements based on a baseline electroencephalogram (EEG) set in advance as a standard for performing a predetermined movement. The baseline EEG may be, for example, data showing an EEG pattern detected when the proficiency of a predetermined movement is moderate. Alternatively, the baseline EEG may be data showing an EEG pattern detected when the subject is not performing any movement. For example, the baseline EEG can be set by machine learning the EEG patterns of multiple people, or by using statistical values ​​of the EEG patterns of multiple people.

[0044] Generally, if a subject is unfamiliar with a given movement, the amplitude of their brainwaves will be larger than that of the reference brainwaves due to excessive force or insufficient motor skills. As the subject becomes more accustomed to the movement, the amplitude of their brainwaves will approach that of the reference brainwaves. Furthermore, as the subject becomes even more accustomed to the movement and their proficiency increases, the amplitude of their brainwaves will become smaller than that of the reference brainwaves. The proficiency evaluation unit 123 utilizes these properties of the subject's brainwaves to evaluate proficiency. The proficiency evaluation unit 123 can evaluate the proficiency of the subject's movement by comparing the reference movement with the subject's movement and comparing the reference brainwaves with the subject's brainwaves, using the results of these comparisons.

[0045] The proficiency evaluation unit 123 may evaluate proficiency using predetermined information for evaluating proficiency. Figure 3 shows an example of a proficiency table T1 for evaluating proficiency. The proficiency table T1 may be stored in the memory unit 13 in advance. The proficiency evaluation unit 123 evaluates proficiency by referring to the proficiency table T1 based on the comparison result between the standard action and the subject's action, and the comparison result between the standard electroencephalogram and the subject's electroencephalogram.

[0046] For example, first, the proficiency evaluation unit 123 compares the amount of movement of the standard movement with the amount of movement of the subject movement and determines whether the amount of movement of the subject movement is "greater than the standard movement," "within the range of the standard movement," or "smaller than the standard movement." The proficiency evaluation unit 123 also compares the amplitude of the standard electroencephalogram (EEG) with the amplitude of the subject's EEG and determines whether the amplitude of the subject's EEG is "greater than the standard EEG," "within the range of the standard EEG," or "smaller than the standard EEG."

[0047] Furthermore, "within the range of the reference movement" means that the amount of movement of the subject falls within the range from the reference movement amount to less than the first threshold. In the following, the range from the reference movement amount to less than the first threshold may be referred to as the "reference movement range."

[0048] The reference range of motion is defined as the range from a value obtained by subtracting the first threshold from the reference movement amount to a value obtained by adding the first threshold, with respect to the movement amount of the reference movement. For example, suppose that at a certain point in time, the movement amount of the reference movement is 15 cm / second and the first threshold is 5 cm / second. In this case, the reference range of motion is in the range of 10 cm / second to 20 cm / second. As shown in the proficiency table T1, if the subject's movement is within the reference range of motion, the proficiency evaluation unit 123 evaluates the proficiency higher the smaller the amplitude of the subject's electroencephalogram (EEG).

[0049] Furthermore, in Figure 3, "within the range of the reference EEG" means that the amplitude of the subject's EEG falls within the range from the reference EEG amplitude to less than the fifth threshold. Hereafter, the range from the reference EEG amplitude to less than the fifth threshold may be referred to as the "reference EEG range." The fifth threshold may be set in advance by, for example, an administrator.

[0050] The reference EEG range is defined as the range from the value obtained by subtracting the fifth threshold from the reference EEG amplitude to the value obtained by adding the fifth threshold, with the reference EEG amplitude as the reference. For example, suppose that at a certain point in time, the reference EEG amplitude is 15 μV and the fifth threshold is 3 μV. In this case, the reference EEG range is the amplitude range of 12 to 18 μV. As shown in the proficiency table T1, if the amount of movement of the subject's action is within the reference action range and the subject's EEG is within the reference EEG range, the proficiency evaluation unit 123 evaluates the proficiency as "medium proficiency". This represents the subject's tendency to become accustomed to the movement.

[0051] Furthermore, if the amount of movement of the subject's actions falls within the standard range of motion and the subject's electroencephalogram (EEG) is smaller than the standard EEG, the proficiency evaluation unit 123 evaluates the proficiency as "high proficiency." This indicates a good tendency for the subject to become accustomed to the movements. In such cases, it is assumed that neurons in the subject's premotor cortex are activated, and the movements of the limbs are performed appropriately. The subject's brain and limbs work in coordination, and the subject is able to achieve the desired movements.

[0052] Furthermore, if the subject's movement volume is greater than the standard movement, and the subject's electroencephalogram (EEG) is smaller than the standard EEG, the proficiency evaluation unit 123 evaluates the proficiency as "unstable proficiency." This indicates a poor tendency to adapt to movement, in contrast to the good adaptation tendency described above. In such cases, it is assumed that the neurons in the subject's premotor cortex are not activated, and that the limb movements are excessive. Because the subject's brain is not functioning properly, the subject is unable to achieve the desired movement. For example, it is assumed that the subject is in a state where their limbs move excessively regardless of their will (e.g., their hands tremble). When such a poor tendency to adapt to movement is observed, the warning unit 125 issues a warning.

[0053] Note that the proficiency level table T1 shown in Figure 3 is just an example and can be modified as needed. For example, in the example shown in the figure, there are three patterns of items each for the amount of movement of the subject's actions and the amplitude of the subject's electroencephalogram, resulting in a total of nine possible combinations. However, this is not the only option, and the number of items and combinations in the proficiency level table T1 can be set arbitrarily. Also, while the example in Figure 3 shows qualitative evaluations such as "low proficiency" and "high proficiency," each of the nine combination patterns may also be represented by a quantitative evaluation using numerical values.

[0054] The proficiency evaluation unit 123 may calculate the proficiency level as a first predetermined value if the amount of movement of the subject's actions is within the standard range of motion and the subject's brainwaves are smaller than the standard brainwaves. The proficiency evaluation unit 123 may also calculate the proficiency level as a second predetermined value, which is smaller than the first predetermined value, if the amount of movement of the subject's actions is not within the standard range of motion or if the subject's brainwaves are not smaller than the standard brainwaves.

[0055] Returning to Figure 1, the display control unit 124 displays the reference operation on the display unit 141 of the output unit 14. For example, the display control unit 124 acquires the reference operation data recorded in the reference information recording unit 134 and transmits this data to the terminal device 5, thereby displaying the reference operation on the display unit 141. In this way, the display control unit 124 assists the subject in performing the appropriate movements in a given exercise. The subject can train to perform the movements with the correct form and timing by performing the movements while looking at the displayed reference operation.

[0056] The display control unit 124 may display the reference motion by moving a 3D model of the human body based on the reference motion. The display control unit 124 may display the reference motion using one of the following as the 3D model of the human body: a standard physique, the physique of the subject, the physique of an expert or specialist used to set the reference motion, or a predetermined physique selected by the administrator.

[0057] The display control unit 124 may display a reference movement for each stage of the exercise. For example, the display control unit 124 may display a list of reference movements for each stage so that the subject or administrator can select the reference movement for the desired stage. The display control unit 124 may also display the video of the reference movement, divided into stages. For example, the display control unit 124 may display the reference movement using chapters set in the video of each stage of the reference movement.

[0058] Furthermore, the display control unit 124 may display the standard movement for the next stage if the proficiency level corresponding to the subject's current stage of movement is equal to or greater than the second threshold, and may divide the standard movement for the current stage into two or more stages and display them if the proficiency level is less than the second threshold. The second threshold may be set in advance by, for example, an administrator. The display control unit 124 may also select any of the stages included in the exercise as the current stage.

[0059] Let's explain this using a golf swing as an example. For instance, the first stage is the motion from when you start swinging the club up until the club head hits the ball, and the second stage is the motion from when the club head hits the ball until the end of the swing.

[0060] Here, the subject performs the first stage action, and the proficiency evaluation unit 123 evaluates the proficiency of the first stage action. The display control unit 124 determines whether the proficiency evaluation corresponding to the first stage action is above the second threshold. If it determines that the evaluation is above the second threshold, the display control unit 124 displays the standard action for the second stage. This allows the subject to perform the second stage action while confirming the standard action for the second stage.

[0061] If the display control unit 124 determines that the proficiency level corresponding to the first stage movement is below the second threshold, it divides the first stage reference movement into two or more stages and displays them. For example, the display control unit 124 divides the first stage reference movement into a backswing reference movement (raising the club) and a downswing reference movement (swinging the club down). The display control unit 124 then displays the backswing reference movement first, followed by the downswing reference movement. By dividing and displaying movements with low proficiency in this way, the user can easily practice those movements. The display control unit 124 may also display the first stage reference movement again before dividing it.

[0062] The display control unit 124 may determine whether the evaluation result of the proficiency corresponding to the subject's backswing motion is above the second threshold, and if it determines that it is above the second threshold, it may display the standard motion for the downswing. In this example, one motion is divided into two, but it is not limited to this. The display control unit 124 may also divide the standard motion for each stage into three or more parts and display them.

[0063] The display control unit 124 may divide the reference operation into multiple stages, even if the reference operation does not include multiple stages. The display control unit 124 may divide the reference operation by dividing the time from the start to the end of the reference operation into predetermined proportions. The display control unit 124 may divide the reference operation based on the sum of the magnitudes of the movement amounts of the reference operation. The display control unit 124 may calculate the sum of the movement amounts from the start to the end of the reference operation at predetermined time intervals and divide the reference operation at times when the sum exceeds a predetermined proportion.

[0064] Furthermore, when the display control unit 124 displays the second stage of standard operations, it may display two or more stages of standard operations together, depending on the level of proficiency corresponding to the first stage of standard operations. For example, if the evaluation of the level of proficiency is higher than the second threshold, such as the sixth threshold, the display control unit 124 may display two or more stages of standard operations together. For example, the display control unit 124 may display the second and third stages of standard operations together. The sixth threshold may be set in advance by, for example, an administrator.

[0065] Furthermore, the display control unit 124 may adjust the amount of movement of the standard movement according to the subject's motor ability. For example, if the subject has a physical disability, it is expected that the subject will have difficulty performing movements close to the standard movement. In such cases, the display control unit 124 will reduce the amount of movement of the standard movement and display it. This allows the display control unit 124 to display a standard movement that has been adjusted to correspond to the subject's motor ability.

[0066] For example, the display control unit 124 will change the current reference movement amount to a smaller value and display it if the subject's movement is smaller than the reference movement, the difference between the amount of movement of the reference movement and the amount of movement of the subject's movement is greater than or equal to the third threshold, and the amplitude of the subject's electroencephalogram is greater than or equal to the fourth threshold.

[0067] The display control unit 124 reduces the amount of movement of the reference movement according to, for example, the amount of movement of the subject's movement. For example, the display control unit 124 may reduce the amount of movement of the reference movement by the amount of the third threshold. Alternatively, the display control unit 124 may reduce the amount of movement of the reference movement to such an extent that the amount of movement of the subject's movement falls within the range of the reference movement. In this way, the subject can train while referring to the reference movement according to their own motor ability. The third and fourth thresholds may be set in advance by, for example, an administrator.

[0068] Such processing is effective, for example, when using the proficiency visualization system 1 as a rehabilitation device for providing rehabilitation to individuals whose motor skills have deteriorated due to injury or illness. In this way, the proficiency visualization system 1 can evaluate the progress of the individual's rehabilitation based on the individual's electroencephalogram (EEG) and movements. The display control unit 124 may also change the content of the reference movement and display it according to the proficiency level corresponding to the movement.

[0069] Furthermore, the display control unit 124 may take into account the difference between the physique of the person assumed in the standard movement and the physique of the subject, and correct the amount of movement in the standard movement. This allows the display control unit 124 to display the standard movement according to the subject's physique, even when the physique of the person assumed in the standard movement differs significantly from that of the subject.

[0070] The display control unit 124 may correct the standard movement based on the ratio of the physique of the skilled or expert used to set the standard movement to the physique of the subject. The display control unit 124 may correct the amount of movement of the standard movement by multiplying the amount of movement of the standard movement by the ratio of the height of the skilled or expert used to set the standard movement to the height of the subject. If the amount of movement of a predetermined part of the body is set as the part-specific standard movement within the standard movement, the display control unit 124 may correct the amount of movement of the part-specific standard movement by multiplying the amount of movement of the part-specific standard movement by the ratio of the length of the predetermined part of the skilled or expert used to set the part-specific standard movement to the length of the predetermined part of the subject.

[0071] The warning unit 125 issues a warning via the output unit 14 according to the proficiency evaluation result in the proficiency evaluation unit 123. For example, the warning unit 125 generates a warning message that warns that the subject's motor proficiency is low or unstable. For example, the warning unit 125 generates a warning message such as, "There is a tendency for the subject to become accustomed to the movement in a negative way. Please check the standard movement and try to perform the movement carefully." The warning unit 125 may also generate a message to encourage the subject's movement to move closer to the standard movement and improve proficiency. For example, the warning unit 125 may generate a message such as, "The movement of your right foot seems small. Let's move it a little bigger."

[0072] The format of the warning message is arbitrary. The warning message may be displayed or audible. The warning unit 125 transmits the generated warning message to the terminal device 5. As a result, the warning message is output from the output unit 14 of the terminal device 5. The warning unit 125 may also output the warning using a device other than the output unit 14. For example, the warning unit 125 may send a warning message to an administrator terminal, causing the administrator terminal's display unit or audio output unit to output the warning message.

[0073] The memory unit 13 stores various data and programs. At least a portion of the memory unit 13 is composed of non-volatile memory so that data is retained even when the power to the terminal device 5 is turned off. The memory unit 13 also includes an electroencephalogram recording unit 131, an action recording unit 132, a proficiency recording unit 133, and a reference information recording unit 134. The memory unit 13 may also consist only of the reference information recording unit 134.

[0074] The electroencephalogram (EEG) recording unit 131 records the subject's electroencephalogram data detected by the EEG detection unit 111. For example, the EEG recording unit 131 records the subject ID, which identifies the subject, in association with the subject's electroencephalogram data.

[0075] The motion recording unit 132 records data on the subject's movements detected by the motion detection unit 112. For example, if the motion detection unit 112 is an acceleration sensor, the motion recording unit 132 records data on the acceleration that changes in accordance with the subject's movements. The motion recording unit 132 may record physical quantities corresponding to the type of sensor used as the motion detection unit 112, not limited to acceleration. For example, the motion recording unit 132 records the subject ID in association with the subject's movement data.

[0076] The proficiency recording unit 133 records the proficiency data evaluated by the proficiency evaluation unit 123. For example, the proficiency recording unit 133 records the subject ID, the subject action ID that identifies the subject action, and the proficiency level in association with each other. The proficiency recording unit 133 may also record the proficiency level corresponding to the subject action at each stage of the movement. The proficiency level may be represented by words such as "high proficiency" or by numerical values.

[0077] The reference information recording unit 134 records reference electroencephalogram (EEG) data and reference motion data. For example, the reference information recording unit 134 records a reference EEG ID that identifies the reference EEG and the reference EEG data in association. The reference EEG data may be time-series data including the amplitude of the reference EEG. The reference EEG data may also include frequency, phase, etc. The reference information recording unit 134 also records a reference motion ID that identifies the reference motion and the reference motion data in association. The reference motion data may be time-series data including the amount of motion of the reference motion. The reference motion data may be, for example, acceleration data.

[0078] The display control unit 124 may display the reference operation and reference operation ID recorded in the reference information recording unit 134 on the display unit 141. The input unit 11 includes a reference operation selection unit (not shown), and the selection of the reference operation ID by the administrator or subject may be accepted via a touch panel or the like. The display control unit 124 may select a reference operation recorded in the reference information recording unit 134 using the reference operation ID selected by the reference operation selection unit. The display control unit 124 may also select reference operation data recorded in the reference information recording unit 134 based on the subject operation acquired by the subject operation acquisition unit 122.

[0079] The output unit 14 includes a display unit 141 and an audio output unit 142. The display unit 141 is a display device such as a display that shows images. The images displayed on the display unit 141 may be still images or moving images. The audio output unit 142 is an audio output device such as a speaker that outputs sound. For example, the display unit 141 and the audio output unit 142 output the proficiency evaluation results from the proficiency evaluation unit 123. The display unit 141 and the audio output unit 142 may also output warning messages generated by the warning unit 125. The output unit 14 may also have functions other than the display unit 141 and the audio output unit 142. For example, the output unit 14 may include a vibration device that outputs vibration when issuing a warning.

[0080] (Processing by control unit 12) Referring to Figures 4 to 6, the processing performed by the control unit 12 of the proficiency visualization system 1 will be explained. Figures 4 to 6 are flowcharts showing the flow of processing performed by the control unit 12. It is assumed that the electroencephalogram detection unit 111 and the motion detection unit 112 of the input unit 11 have started detecting the subject's electroencephalogram and motion, respectively. The electroencephalogram detection unit 111 transmits the subject's electroencephalogram to the terminal device 5. The motion detection unit 112 also transmits the subject's motion detected at the attachment site to the terminal device 5.

[0081] First, referring to Figure 4, the process for evaluating proficiency will be explained. The subject electroencephalogram (EEG) acquisition unit 121 acquires the subject's EEG from the EEG detection unit 111 (S101). The subject motion acquisition unit 122 acquires the subject's motion from the motion detection unit 112 (S102).

[0082] Next, the proficiency evaluation unit 123 compares the amount of movement of the standard movement with the amount of movement of the subject's movement (S103). The proficiency evaluation unit 123 also determines whether the absolute value of the difference between the amount of movement of the standard movement and the amount of movement of the subject's movement is less than the first threshold (S104). If it is determined that the absolute value of the difference is less than the first threshold (YES in S104), the proficiency evaluation unit 123 evaluates the proficiency higher the smaller the amplitude of the subject's electroencephalogram (S105). If it is determined that the absolute value of the difference is greater than or equal to the first threshold (NO in S104), the proficiency evaluation unit 123 evaluates the proficiency as low or unstable according to the proficiency table T1 (S106).

[0083] Next, referring to Figure 5, the process of displaying the reference operation will be explained. First, the display control unit 124 determines whether the proficiency level corresponding to the current operation is above the second threshold (S201). If it is determined that the proficiency level is above the second threshold (YES in S201), the display control unit 124 displays the reference operation for the next stage (S202).

[0084] If the proficiency level is determined to be below the second threshold (NO in S201), the display control unit 124 divides the current standard operation into two or more stages and displays them (S203). The display control unit 124 then determines whether to terminate the display of the standard operation (S204). If it determines to terminate the display of the standard operation (YES in S204), the process ends. If it determines not to terminate the display of the standard operation (NO in S204), the process returns to step S201 and repeats the subsequent processing.

[0085] Next, referring to Figure 6, the process of changing and displaying the amount of movement of the reference motion will be explained. First, the display control unit 124 determines whether the subject's movement is smaller than the reference motion (S301). If it is determined that the subject's movement is not smaller than the reference motion (NO in S301), the process proceeds to step S305.

[0086] If the system determines that the subject's movement is smaller than the reference movement (YES in S301), the display control unit 124 determines whether the difference between the amount of movement of the reference movement and the amount of movement of the subject is greater than or equal to the third threshold (S302). If the system determines that the difference between the amount of movement of the reference movement and the amount of movement of the subject is less than the third threshold (NO in S302), the system proceeds to step S305.

[0087] If the difference between the amount of movement of the reference movement and the amount of movement of the subject movement is determined to be greater than or equal to the third threshold (YES in S302), the display control unit 124 determines whether or not the amplitude of the subject's electroencephalogram is greater than or equal to the fourth threshold (S303). If the amplitude of the subject's electroencephalogram is determined to be less than the fourth threshold (NO in S303), the process proceeds to step S305.

[0088] If the amplitude of the subject's electroencephalogram is determined to be above the fourth threshold (YES in S303), the display control unit 124 reduces the current reference movement amount and displays it (S304). If the answer in any of steps S301, S302, or S303 is NO, the display control unit 124 displays the reference movement amount without changing it (S305).

[0089] The display control unit 124 determines whether or not to terminate the display of the reference operation (S306). If it determines to terminate the display of the reference operation (YES in S306), the process ends. If it determines not to terminate the display of the reference operation (NO in S306), the process returns to step S301 and repeats the subsequent process.

[0090] As described above, in the proficiency visualization system 1 according to this disclosure, the control unit 12 acquires the subject's brainwaves and subject's movements when the subject performs a predetermined movement, and evaluates the proficiency of the subject's movements based on the subject's brainwaves, subject's movements, and a preset reference movement. The control unit 12 compares the amount of movement of the reference movement with the amount of movement of the subject's movements, and if the absolute value of the difference between them is less than the first threshold, the control unit 12 evaluates the proficiency higher the smaller the amplitude of the subject's brainwaves.

[0091] In this way, the control unit 12 can quantitatively measure the gap between the subject's intention and actual movement by sensing brain responses that reflect movement and limb movements. As a result, the proficiency visualization system 1 can appropriately evaluate the subject's level of motor proficiency.

[0092] Furthermore, the proficiency visualization system 1 can display standard movements for each stage of movement. If the proficiency level corresponding to the current movement is above the second threshold, the proficiency visualization system 1 displays the standard movement for the next stage. If the proficiency level is below the second threshold, it divides the standard movement for the current stage into two or more stages and displays them. This allows the subject to train by viewing standard movements adjusted according to their own proficiency level.

[0093] Furthermore, if the subject's movement is smaller than the standard movement, the difference between the movement volume of the standard movement and the movement volume of the subject's movement is greater than or equal to the third threshold, and the amplitude of the subject's electroencephalogram is greater than or equal to the fourth threshold, the proficiency visualization system 1 will reduce the movement volume of the current standard movement and display it. In this way, the proficiency visualization system 1 can adjust the standard movement to match the subject's motor ability. As a result, if the subject's motor ability is reduced due to injury or illness, they can see the appropriately adjusted standard movement and carry out rehabilitation.

[0094] Each functional component of the proficiency visualization system 1 described above may be implemented by hardware that realizes each functional component (e.g., hardwired electronic circuits), or by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it). For example, this disclosure can also realize any processing by having a CPU execute a computer program.

[0095] The program, when loaded into a computer, includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored in various types of non-transitory computer-readable medium or tangible storage medium. Examples, but not limited to, include RAM (Random-Access Memory), ROM (Read-Only Memory), flash memory, SSD (Solid-State Drive), or other memory technologies, CD-ROM, DVD (Digital Versatile Disc), Blu-ray® disc, or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage devices. The program may also be transmitted over various types of transient computer-readable medium or communication medium. Examples, but not limited to, include transient computer-readable medium or communication medium, including electrically, optically, acoustically, or otherwise propagating signals.

[0096] This disclosure is not limited to the embodiments described above, and may be modified as appropriate without departing from its intent. For example, the first to sixth threshold values ​​described above may be adjusted as appropriate depending on the sensor used, individual differences among subjects, or use case. [Explanation of symbols]

[0097] 1. Proficiency Visualization System 5 Terminal devices 11 Input section 12 Control Unit 13 Storage section 14 Output section 111 EEG detection unit 112, 112a~112d Motion detection unit 121 Subject EEG Acquisition Unit 122 Target Person Motion Acquisition Unit 123 Proficiency Assessment Department 124 Display Control Unit 125 Warning section 131 Electroencephalogram Recording Unit 132 Operation Recording Unit 133 Proficiency Record Department 134 Standard Information Recording Unit 141 Display section 142 Audio output section N Network T1 Proficiency Chart

Claims

1. A subject electroencephalogram (EEG) acquisition unit that acquires subject electroencephalograms showing the brainwaves of a subject when they perform a prescribed exercise, A subject motion acquisition unit that acquires subject motions that indicate the actions taken by the subject when the subject performs the predetermined exercise, The system includes a proficiency evaluation unit that, using the standard for performing the predetermined movement as the standard movement, evaluates the proficiency of the subject's movement higher the smaller the amplitude of the subject's electroencephalogram (EEG) is, if the absolute value of the difference between the amount of movement of the standard movement and the amount of movement of the subject's movement is less than a first threshold. Information processing device.

2. The system further includes a display control unit that displays the reference operation for each stage of the aforementioned movement, The aforementioned reference operation is set for each of the aforementioned stages, The proficiency evaluation unit evaluates the proficiency level for each stage, The display control unit displays the standard operation for the next stage if the proficiency level corresponding to the current operation is equal to or greater than the second threshold, and divides the standard operation for the current operation into two or more stages and displays them if the proficiency level is less than the second threshold. The information processing apparatus according to claim 1.

3. The display control unit, if the subject's movement is smaller than the reference movement, the difference between the amount of movement of the reference movement and the amount of movement of the subject's movement is greater than or equal to the third threshold, and the amplitude of the subject's electroencephalogram is greater than or equal to the fourth threshold, will reduce the amount of movement of the reference movement at the current stage and display it. The information processing apparatus according to claim 2.

4. A subject brainwave acquisition step involves acquiring subject brainwaves that show the brainwaves of the subject when they perform a prescribed exercise, and A subject motion acquisition step, which acquires subject motions that indicate the actions performed by the subject when the subject performs the predetermined exercise, The proficiency evaluation step includes, using the standard for performing the predetermined movement as the standard movement, and if the absolute value of the difference between the amount of movement of the standard movement and the amount of movement of the subject movement is less than a first threshold, the smaller the amplitude of the subject's electroencephalogram, the higher the proficiency of the subject movement is evaluated. Information processing methods.

5. A subject brainwave acquisition step involves acquiring subject brainwaves that show the brainwaves of the subject when they perform a prescribed exercise, and A subject motion acquisition step, which acquires subject motions that indicate the actions performed by the subject when the subject performs the predetermined exercise, The computer is instructed to perform a proficiency evaluation step in which, using the standard for performing the predetermined movement as the standard movement, if the absolute value of the difference between the amount of movement of the standard movement and the amount of movement of the subject movement is less than a first threshold, the smaller the amplitude of the subject's brainwave, the higher the proficiency of the subject movement is evaluated. program.

Images