Information processing program, information processing device, and information processing method

Abstract

This invention provides an information processing program, an information processing device, and an information processing method that can support exercise in order to stabilize the user's exercise load. [Solution] The information processing program causes the processor to perform the following actions: acquire a user video of user 70 performing an exercise; input the user video into an evaluation model that outputs evaluation results regarding the exercise captured in the input video; and acquire a user exercise evaluation from the evaluation model regarding the exercise load of user 70 captured in the user video. The user exercise evaluation includes an evaluation of whether the fluctuation of the estimated load, which is an estimated value of the exercise load of user 70 captured in the user video, was within the fluctuation setting range.

Description

Technical Field

[0001] The present disclosure relates to an information processing program, an information processing apparatus, and an information processing method.

Background Art

[0002] As described in Patent Document 1, a cardiac rehabilitation support apparatus that adjusts the exercise load of a subject's pedaling motion based on the subject's biological signal is known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In physical therapy, the effectiveness of physical therapy decreases due to insufficient exercise load of the user. Also, the safety of physical therapy decreases due to excessive exercise load of the user. It is required to support the user so that the exercise load is stabilized.

[0005] The present disclosure has been made in view of the above points, and an object thereof is to provide an information processing program, an information processing apparatus, and an information processing method capable of supporting exercise so that the exercise load of a user is stabilized.

Means for Solving the Problems

[0006] An information processing program (1) according to one embodiment of the present disclosure causes a processor to perform the following actions: acquire a user video of a user performing an exercise; input the user video into an evaluation model that outputs evaluation results regarding the exercises shown in the input video; and obtain a user exercise evaluation from the evaluation model as the evaluation result, which relates to the exercise load of the user shown in the user video. The user exercise evaluation includes an evaluation of whether the fluctuation of the estimated load, which is an estimated value of the exercise load of the user shown in the user video, was within a fluctuation setting range.

[0007] (2) In the information processing program described in (1) above, the exercise type may be defined such that the exercise load is constant when the exercise is performed according to the exercise type.

[0008] (3) In the information processing program described in (1) or (2) above, the user exercise evaluation may include an evaluation of whether the difference between the standard exercise load, which is the standard exercise load when the user performs the exercise, and the estimated load was within the load setting range.

[0009] (4) In the information processing program described in any one of (1) to (3) above, the exercise type may include multiple unit exercises. The exercise type may be defined such that the unit standard load, which is the standard exercise load when the user performs each of the multiple unit exercises, is equal.

[0010] (5) In the information processing program described in (4) above, the user movement evaluation may include an evaluation of whether the fluctuation of the estimated unit load, which is an estimated value of the user's movement load for each of the plurality of unit movements, was within the unit fluctuation setting range.

[0011] (6) The information processing program described in (5) above may include the number of unit motions in which the difference between the unit standard load and the unit estimated load was within the unit load setting range.

[0012] (7) In the information processing program described in any one of (1) to (6) above, the user movement evaluation may include a score calculated with a higher value the closer the fluctuation of the estimated load is to zero.

[0013] (8) In the information processing program described in any one of (4) to (6) above, the user movement evaluation may include a unit movement score, which is calculated as a higher value the closer the variation in the estimated unit load, which is an estimated value of the user's movement load for each of the plurality of unit movements, is to zero.

[0014] (9) In the information processing program described in any one of (1) to (8) above, the standard load of the exercise performed by the user may be an exercise load corresponding to the user's anaerobic metabolic threshold.

[0015] (10) In the information processing program described in any one of (1) to (9) above, the evaluation model may be configured to further input the user's state during at least a portion of the entire duration of the user video or at least a portion of the time points in time, and to output as the evaluation result that the level of the user's exercise load is constant at an estimated level based on the user's state throughout the entire duration of the user video.

[0016] (11) In the information processing program described in (10) above, the evaluation model may be configured to output as the evaluation result that the level of the user's exercise load is constant at the reference level when the estimated level is the reference level.

[0017] (12) In the information processing program described in any one of (1) to (11) above, the exercise type may be defined so that the user can perform the exercise while standing.

[0018] An information processing apparatus (13) according to an embodiment of the present disclosure includes a processor. The processor acquires a user video that captures a user performing a sports event, inputs the user video into an evaluation model that outputs an evaluation result regarding the motion shown in the input video, and obtains, as the evaluation result from the evaluation model, a user motion evaluation regarding the amount of exercise load of the user shown in the user video. The user motion evaluation includes an evaluation that the variation of the estimated load amount, which is an estimated value of the amount of exercise load of the user shown in the user video, is within a variation setting range.

[0019] An information processing method (14) according to an embodiment of the present disclosure includes: a processor acquiring a user video that captures a user performing a sports event; inputting the user video into an evaluation model that outputs an evaluation result regarding the motion shown in the input video; and obtaining, as the evaluation result from the evaluation model, a user motion evaluation regarding the amount of exercise load of the user shown in the user video. The user motion evaluation includes an evaluation that the variation of the estimated load amount, which is an estimated value of the amount of exercise load of the user shown in the user video, is within a variation setting range.

Advantages of the Invention

[0020] According to the information processing program, information processing apparatus, and information processing method according to the present disclosure, exercise is supported so that the amount of exercise load of the user is stabilized.

Brief Description of the Drawings

[0021] [Figure 1] It is a block diagram showing a configuration example of an information processing system according to the present disclosure. [Figure 2] It is a schematic diagram for explaining an example of a state of supporting a user's exercise using an information processing system. [Figure 3] It is a diagram showing an example of a screen of an information processing apparatus for supporting a user's exercise. [Figure 4] It is a flowchart showing an example of a procedure for outputting an evaluation result of a user's exercise. [Figure 5]It is a flowchart showing an example of a procedure for evaluating a user's exercise with a score. [Figure 6] It is a flowchart showing an example of a procedure for evaluating a user's exercise by dividing it into unit exercises.

Embodiments for Carrying Out the Invention

[0022] There is known a physical therapy that utilizes exercise for the treatment or prevention of diseases and the like. It is preferable for the target user of physical therapy to perform exercise with an appropriate exercise load. Here, the exercise load is an index representing the burden on the body of the target user when the target user performs exercise. The exercise load may be defined based on, for example, the oxygen uptake or energy consumption when the target user is performing exercise. The exercise load may also be defined based on the amount of physical work when the target user is performing exercise. The exercise load is not limited to these examples and may be defined based on various information.

[0023] In physical therapy, the higher the exercise load at which the target user of physical therapy performs exercise, the higher the effectiveness of physical therapy. On the other hand, if the exercise load is too high, an excessive burden is placed on the body of the target user and safety decreases. Therefore, in physical therapy, an exercise load that can enhance both the effectiveness and safety of physical therapy is defined as an appropriate exercise load.

[0024] Also, when the exercise load when performing exercise fluctuates from an appropriate exercise load, the exercise load becomes too high or too low. When the exercise load becomes too high or too low, the effectiveness or safety of physical therapy decreases. Therefore, in physical therapy, it is required to exercise with a stable exercise load.

[0025] It is considered that the exercise load stabilizes by repeating the same exercise. In fact, it has been confirmed that by defining the range of motion and pace of exercise and performing them continuously, the exercise load reaches a plateau, that is, the oxygen uptake and heart rate can be adjusted within a certain range.

[0026] Furthermore, in exercise therapy, an appropriate exercise load is, for example, an exercise load corresponding to the anaerobic threshold (AT) of the user undergoing exercise therapy. The effects of exercise therapy are more likely to appear when the exercise load for the user is around the AT level. Therefore, it is preferable to define the exercise so that the exercise load for the user can be adjusted to be around the AT level.

[0027] However, even if a user undergoing exercise therapy mimics exercises defined to maximize the effectiveness of the therapy, differences in the user's movements compared to the defined exercises can result in an exercise load that is either insufficient or excessive, failing to reach the intended value. Insufficient load reduces the effectiveness of the exercise therapy. Insufficient load reduces the safety of the exercise therapy. Therefore, it is necessary to confirm whether the user is performing the exercises at a stable exercise load.

[0028] To stabilize the exercise load for users undergoing exercise therapy, it is conceivable to use equipment that allows for adjustment of the exercise load, such as ergometers. However, users would need to travel to a facility where such equipment is installed to use it. Furthermore, installing such equipment in a user's home is not easy. In other words, the convenience for users when performing exercise therapy is reduced. Due to the low convenience for users when performing exercise therapy, users may not be able to easily perform exercise therapy and may not be able to continue performing it.

[0029] To perform exercises in a way that the exercise load reaches a plateau without using equipment such as an ergometer, one could, for example, perform exercises that are aligned with a sample exercise defined to reach a plateau in exercise load.

[0030] One method to confirm that exercise is being performed in a way that the exercise load reaches a plateau is to verify that the user's movements match those of a model exercise. However, the exercise load may still reach a plateau even if the user's movements do not match those of the model exercise. Therefore, it is possible to miss the fact that the exercise load has reached a plateau.

[0031] Furthermore, as a method to confirm that users undergoing exercise therapy are able to perform exercises in such a way that their exercise load reaches a plateau, it is conceivable to actually measure the user's exercise load and confirm that they have reached a plateau. Measuring the user's oxygen consumption during exercise as information to define the user's exercise load is not easy, as it would require analyzing the user's exhaled gases. Similarly, measuring the user's energy expenditure during exercise as information to define the user's exercise load is also not easy.

[0032] As mentioned above, it is difficult to confirm that users undergoing exercise therapy are performing exercises in a way that their exercise load reaches a plateau. There is a need for a simple way to confirm that users are performing exercises in a way that their exercise load reaches a plateau.

[0033] The following explains that, according to the Information Processing System 1 related to this disclosure (see Figure 1), it is possible to easily confirm that the exercise load reaches a plateau, that is, that the user of exercise therapy is performing exercise at a stable exercise load. By easily confirming that the user is performing exercise at a stable exercise load, the user can easily adjust and perform exercise to achieve a stable exercise load. In other words, the Information Processing System 1 related to this disclosure can support the user of exercise therapy in performing exercise at a stable exercise load.

[0034] (Example configuration of Information Processing System 1) As shown in Figure 1, the information processing system 1 according to this disclosure comprises an information processing device 10. The information processing system 1 further comprises a server 20, although this is not required. The information processing device 10 and the server 20 are communicated with each other via a wired or wireless network 40. The information processing device 10 and the server 20 may be communicated with each other directly without using the network 40. The information processing system 1 further comprises a wearable device 30, although this is not required. The information processing device 10 and the wearable device 30 are communicated with each other either without using the network 40 or via the network 40.

[0035] <Information Processing Device 10> The information processing device 10 comprises a processor 11, a storage unit 12, a communication unit 13, an input unit 14, an output unit 15, and a camera 16.

[0036] The processor 11 controls the operation of the information processing device 10 and realizes the functions of the information processing device 10. The processor 11 may be composed of one or more general-purpose processors or dedicated circuits. The general-purpose processor may include a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), etc. The dedicated circuit may include an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), etc.

[0037] The memory unit 12 stores any data, information, or programs used in the operation of the information processing device 10. The memory unit 12 may store, for example, a system program such as an OS (Operating System) or an application program. The application program may include, for example, a web browser, a therapeutic application, a rehabilitation application, or a fitness application. The diseases targeted by the therapeutic application or rehabilitation application may include, for example, lifestyle-related diseases such as heart disease, hypertension, diabetes, dyslipidemia, or hyperlipidemia, or diseases such as obesity where exercise may contribute to the improvement of symptoms. The data may include, for example, a database referenced in information processing, or data obtained by performing information processing, i.e., the results of performing information processing.

[0038] The storage unit 12 may include, but is not limited to, semiconductor memory, magnetic memory, or optical memory. The storage unit 12 may function as, for example, main memory, auxiliary memory, or cache memory. The storage unit 12 may include an electromagnetic storage medium such as a magnetic disk. The storage unit 12 may include a non-temporary computer-readable medium. The storage unit 12 may be included in the processor 11.

[0039] The communication unit 13 may include a communication module configured to communicate with other devices such as the server 20. The communication module may support mobile communication standards such as 4G (4th Generation) or 5G (5th Generation). The communication module may also support communication standards such as LAN (Local Area Network). The communication module may also support wired or wireless communication standards. The communication module is not limited to these and may support various communication standards.

[0040] The input unit 14 may include an input device that receives information or data from a user of the information processing system 1. The input device may include, for example, a touch panel or touch sensor, or a pointing device such as a mouse. The input device may also include physical keys. The input device may also include an audio input device such as a microphone. The input unit 14 may include an input interface that can be connected to an external input device. The input unit 14 may be configured to acquire information or data input to an external input device via the input interface.

[0041] The output unit 15 may include an output device that outputs information or data to the user of the information processing system 1. The output device may include, for example, a display device that outputs visual information such as images, characters, or graphics. The display device may include, for example, an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, an inorganic EL display, or a PDP (Plasma Display Panel). The display device is not limited to these displays and may include various other types of displays. The display device may include a light-emitting device such as an LED (Light Emitting Diode) or an LD (Laser Diode). The display device may include various other devices. The output device may include, for example, an audio output device such as a speaker that outputs auditory information such as sound. The output device is not limited to these examples and may include various other devices. The output unit 15 may include an output interface that is configured to be connectable to an external output device. The output unit 15 may be configured to output information or data to an external output device via the output interface.

[0042] Camera 16 captures the subject and outputs a still image or video of the subject. In this disclosure, Camera 16 captures the subject of exercise therapy as a user performing exercise and outputs the captured video.

[0043] Camera 16 may include a stereo camera capable of measuring distance data to the object being photographed. Camera 16 may be configured to acquire point cloud data of the object being photographed. The information processing device 10 may include a distance sensor or a point cloud data generation device separately from camera 16.

[0044] Camera 16 may be configured as an external camera separate from the information processing device 10. If camera 16 is configured as an external camera, the information processing device 10 may acquire user video from the external camera, for example, via the input interface of the input unit 14.

[0045] The information processing device 10 may include a mobile device such as a smartphone or tablet, or a PC such as a notebook PC (Personal Computer) or tablet PC. The information processing device 10 is not limited to these examples and may include various other devices.

[0046] <Server 20> Server 20 may include a server control unit. The server control unit controls the operation of Server 20 and implements the functions of Server 20. The server control unit may be configured to include one or more general-purpose processors or dedicated circuits. The general-purpose processor may include a CPU or GPU, etc. The dedicated circuit may include an FPGA or ASIC, etc. The server control unit may be configured similarly to the processor 11 of the information processing device 10.

[0047] Server 20 may include a server storage unit. The server storage unit stores any data, information, or programs used in the operation of Server 20. The server storage unit may be configured similarly to the storage unit 12 of the information processing device 10.

[0048] Server 20 may include a server communication unit. The server communication unit may include a communication module configured to communicate with other devices such as the information processing device 10. The server communication unit may be configured similarly to the communication unit 13 of the information processing device 10.

[0049] Server 20 may include a server input unit. The server input unit may be configured similarly to the input unit 14 of the information processing device 10. Server 20 may also include a server output unit. The server output unit may be configured similarly to the output unit 15 of the information processing device 10.

[0050] Server 20 may consist of one or multiple server devices that can communicate with each other. Server 20 may be implemented as a cloud server.

[0051] <Wearable Devices 30> The wearable device 30 is configured to be worn on the body of a user undergoing exercise therapy. The wearable device 30 may be equipped with sensors.

[0052] The sensor may include, for example, a heart rate sensor. The heart rate sensor measures the heart rate of a user when the wearable device 30 is attached to the user's body during exercise therapy and outputs the measurement result of the user's heart rate. The heart rate measurement result may include the heart rate. The heart rate measurement result may include the heart rate waveform. The heart rate sensor may be configured to measure the heart rate using an optical measurement technique.

[0053] The sensor may include, for example, an accelerometer. The accelerometer is configured to detect the acceleration of a part of the body of the target user wearing the wearable device 30.

[0054] The wearable device 30 may include a device control unit. The device control unit controls the operation of the wearable device 30 and realizes the functions of the wearable device 30. The device control unit may be configured similarly to the processor 11 of the information processing device 10.

[0055] The wearable device 30 may include a device storage unit. The device storage unit stores any data, information, or programs used for the operation of the wearable device 30. The device storage unit may be configured similarly to the storage unit 12 of the information processing device 10.

[0056] The wearable device 30 may include a device communication unit. The device communication unit may include a communication module configured to communicate with other devices such as the information processing device 10. The device communication unit may be configured similarly to the communication unit 13 of the information processing device 10.

[0057] The wearable device 30 may include a device input unit. The device input unit may be configured similarly to the input unit 14 of the information processing device 10. The wearable device 30 may also include a device output unit. The device output unit may be configured similarly to the output unit 15 of the information processing device 10.

[0058] (Example of operation of Information Processing System 1) As illustrated in Figure 2, the information processing system 1 captures images of the user performing the exercise, i.e., the exercise user 70, using a camera 16 of the information processing device 10 mounted on a tripod. The camera 16 may be positioned to capture the entire body of the exercise user 70 from the front or diagonally in front, as the exercise user 70 performs the exercise. The camera 16 may be positioned, for example, about 2 meters away from the exercise user 70. The camera 16 may be positioned at an appropriate height relative to the exercise user 70 using height adjustment means such as a tripod. The video of the exercise user 70 captured by the camera 16 is also called a user video. The user video may include distance data or point cloud data of the exercise user 70.

[0059] In this example, the exercise user 70 is assumed to be a user targeted for exercise therapy. Exercise therapy may include cardiac rehabilitation or exercise instruction. The exercise user 70 may also be a user who exercises independently of exercise therapy. The exercise user 70 may perform exercises defined to match the exercise load based on the exercise therapy plan or instruction. The exercise user 70 may perform exercises while watching videos demonstrating the exercises, as described later. The exercise user 70 may perform exercises based on verbal or written instructions.

[0060] The processor 11 of the information processing device 10 acquires user video from the camera 16. The processor 11 may store the user video in the storage unit 12. Based on the user video, the processor 11 evaluates whether the exercise user 70 has reached a plateau in exercise load, that is, whether they are able to perform the exercise at a constant exercise load. The processor 11 may output a pass / fail evaluation if the exercise user 70 is able to perform the exercise at a constant exercise load.

[0061] The exercise load can fluctuate depending on the timing of movements of at least a part of the body by the person performing the exercise. Therefore, when the temporal change in the exercise load is captured over short time intervals, the temporal change in the exercise load may be large, and the exercise load may not remain constant. However, the stability of the exercise load required in exercise therapy is sufficient if the exercise load remains constant when the temporal change in the exercise load is captured over a long period. Therefore, the processor 11 may, for example, calculate a moving average of the exercise load over a period corresponding to the cycle in which at least a part of the body is repeatedly moved during exercise, and output a pass / fail evaluation if the exercise load after the moving average is constant. The period for calculating the moving average is not limited to the example above and may be set appropriately depending on the content of the exercise.

[0062] The processor 11 may evaluate whether the exercise user 70 is able to perform the exercise defined to result in an exercise load based on the exercise therapy plan or instruction.

[0063] The following describes a specific example of the operation of Information Processing System 1.

[0064] <User Profile> If the exercise user 70 is a target user for exercise therapy, they may perform exercises defined to provide an appropriate exercise load for the exercise user 70, based on the exercise therapy plan or instruction. The processor 11 of the information processing device 10 may manage the user profile of the exercise user 70 in order to suggest exercises defined to provide an appropriate exercise load for the exercise user 70. The user profile may include information that associates an ID or username that identifies the exercise user 70 with the target load for the exercise user 70, as shown in Table 1 below. In Table 1, the target load for the exercise user 70 whose username is XYZ is 3.5 METs.

[0065] [Table 1]

[0066] The target exercise load is information regarding the target value of the exercise load when the exercise user 70 performs exercise. The exercise load may be defined by items such as oxygen consumption, energy expenditure, or heart rate, or a combination thereof.

[0067] For example, the target load may be set by a physician or other person in charge of planning or guiding exercise therapy, based on the results of measuring the exercise tolerance of the exercise user 70 by cardiopulmonary exercise testing, i.e., CPX testing, when using an ergometer, or based on the results of measuring the heart rate by a heart rate sensor. CPX testing can determine the anaerobic threshold (AT) of the person being tested based on the measurement results of oxygen consumption, carbon dioxide output, tidal volume, respiratory rate, or minute ventilation, or a combination thereof, by exhaled gas analysis. CPX testing is not limited to AT, but can also determine the maximum oxygen consumption corresponding to the exercise intensity near the maximum exercise tolerance. The person in charge of planning or guiding exercise therapy may set the target load based on the oxygen consumption or heart rate when the exercise user 70 performs exercise at an exercise load equivalent to the AT.

[0068] The target load may not be set based on measurement results, but rather at the discretion of the physician. Alternatively, the target load may be determined by an algorithm based on the results of measuring the exercise tolerance of the exercise user 70, for example, by a CPX test.

[0069] In order for exercise user 70 to perform aerobic exercise safely and effectively, it is preferable that exercise user 70 perform exercise at an exercise load equivalent to their AT (anaerobic threshold). Therefore, the target load for exercise user 70 may be set to an exercise load equivalent to their AT. Alternatively, the target load for exercise user 70 may be set to an exercise load calculated using other criteria.

[0070] The appropriate exercise load for exercise user 70's exercise therapy, or the exercise types corresponding to that exercise load, may be determined by the person responsible for planning or supervising exercise user 70's exercise therapy. This person may include, for example, a healthcare professional, a nutritionist, or a trainer. Healthcare professionals may include physicians, nurses, pharmacists, physical therapists, occupational therapists, or clinical laboratory technicians. The appropriate exercise load for exercise user 70's exercise therapy, or the exercise types corresponding to that exercise load, may be associated with an ID or other identifier for exercise user 70 in the user profile.

[0071] As an example of exercise prescription, a physician may set an upper limit on the exercise load for exercise user 70. This upper limit may be associated with an ID or other identifier for exercise user 70 in the user profile. If an upper limit on exercise load is set, exercise user 70 is not permitted to select exercises that exceed this limit.

[0072] A user interface (UI) screen for exercise prescription may be displayed on the display of the terminal used by the physician for exercise prescription. The UI screen may include, for example, the display of the exercise user 70's CPX data, or the display of sample videos for multiple exercise types that the exercise user 70 can select. The sample videos for each exercise type may be displayed in an order based on the exercise load corresponding to that exercise type. For example, if 3.6 METs is recommended as the upper limit of the exercise load for a person subject to exercise prescription based on the measurement results of a CPX test, sample videos for exercise types corresponding to 3.4 METs, 3.6 METs, and 3.8 METs may be displayed on the UI screen. When the physician selects a sample video from any of the exercise types, the exercise load associated with the exercise type corresponding to the selected sample video is set as the upper limit for the exercise user 70.

[0073] The upper limit of exercise load specified by a physician's exercise prescription may be changed by a healthcare professional under the supervision of a physician, for example, during regular medical guidance sessions or physician rounds every two weeks.

[0074] The user profile may include information that associates the exercise user 70's physical information with an ID or username that identifies the exercise user 70. Physical information refers to information about the exercise user 70's body or physical function. For example, physical information may include information such as the exercise user 70's age, gender, weight, or height, or information about the exercise user 70's illnesses, etc.

[0075] The user profile may include information identifying the person responsible for planning or supervising the exercise therapy of the exercise user 70, or information identifying the physician in charge of the exercise user 70.

[0076] The user profile may be stored in the storage unit 12 of the information processing device 10, the server storage unit of the server 20, or in other storage devices such as databases.

[0077] <Sports Category> If the exercise user 70 is a target user for exercise therapy, they perform the target exercises for exercise therapy. The target exercises for exercise therapy are defined to have an exercise load based on the exercise therapy plan or instruction. In this disclosure, the target exercises are defined so that the exercise load is constant when the exercises are performed as described. The standard load for the target exercises may be an exercise load corresponding to the target user's anaerobic metabolic threshold.

[0078] Exercise programs may be defined in a way that allows them to be performed without the use of special equipment such as ergometers. Exercise programs may also be defined in a way that allows users of exercise therapy to perform the exercises while standing. Doing so will reduce obstacles to the implementation of exercise therapy.

[0079] Information regarding the types of exercises covered by the exercise therapy may be stored in the storage unit 12 of the information processing device 10, the server storage unit of the server 20, or other storage devices such as databases. The information regarding the types of exercises covered by the exercise therapy may include explanations necessary for the exercise user 70 to perform the exercises. Explanations of the exercises may be provided as sample videos. Sample videos are videos in which a trainer or the like demonstrates the movements of the exercises. Explanations of the exercises may also be provided as audio data with verbal explanations, or as text data with written explanations. In this example, the explanations of the exercises are provided as sample videos.

[0080] Information regarding the types of exercises targeted for exercise therapy may be associated with an ID or name that identifies the exercise type and a standard load, as shown in Table 2 below. In Table 2, the standard load for the exercise called "leg raises" is 4.5 METs. The standard load for the exercise called "A exercise" is 2.5 METs.

[0081] [Table 2]

[0082] The standard load for an exercise is information about the exercise load when a person with standard physical function performs the exercise in question. In other words, an exercise is defined such that the standard load is the standard exercise load when the exercise is performed as specified. The standard load for an exercise may be derived, for example, by measuring the average oxygen consumption of multiple people performing the exercise in question using exhaled gas analysis, and then performing statistical processing such as averaging on the measurement results of the average oxygen consumption. The measurement items used to derive the standard load are not limited to average oxygen consumption, but may also include other items such as energy consumption. The standard load may also be obtained as an exercise load set by a third-party organization.

[0083] Standard loads may be associated with exercises that are more subdivided than commonly recognized exercises. For example, an exercise may include multiple variations that change at least one element, such as form, pace, number of repetitions, or rest time or number of repetitions. Standard loads may be associated with each of the exercises subdivided by variations of commonly recognized exercises. In other words, an exercise that is commonly recognized as a single exercise can be defined as multiple exercises with slightly different exercise loads by specifying details such as form, pace, number of repetitions, or rest time. For example, high knees are an exercise that is commonly recognized as a single exercise. For exercises subdivided by variations that change at least one element of high knees, multiple exercises can be defined within high knees that differ in exercise load by 0.2 METs each.

[0084] The exercise user 70 can perform aerobic exercise by selecting and playing a sample video of an exercise with a standard load that is equal to or less than the exercise load equivalent to the exercise user 70's AT, and then performing the exercise by imitating the sample video. If the exercise therapy is cardiac rehabilitation, the person in charge of planning or supervising the exercise therapy may determine that exercises with a standard load that is equal to or less than the exercise load equivalent to the exercise user 70's AT are exercises with an exercise load appropriate for the exercise user 70's cardiac rehabilitation.

[0085] The exercise user 70 may perform anaerobic exercise by selecting and playing a sample video of an exercise type in which the standard load is greater than the exercise load corresponding to the exercise user 70's AT, and then performing the exercise by imitating the sample video.

[0086] The exercise user 70 may perform aerobic or anaerobic exercise by selecting and playing a sample video of an exercise type whose standard load is around the exercise load corresponding to the exercise user 70's AT, and then imitating the sample video. The exercise load around the exercise user 70's AT may be an exercise load within the AT setting range. The lower limit of the AT setting range is an exercise load that is one predetermined value lower than the exercise load corresponding to the exercise user 70's AT. The upper limit of the AT setting range is an exercise load that is one predetermined value higher than the exercise load corresponding to the exercise user 70's AT. The first predetermined value and the second predetermined value may be set as appropriate. The first predetermined value and the second predetermined value may be the same value or may be different values.

[0087] Exercise types may be associated with index reference values. Index reference values ​​are information that quantify the amount of exercise load when a person with standard physical function performs the exercise type corresponding to the index reference value.

[0088] For example, an exercise can be subdivided into multiple unit movements. A unit movement is a constituent unit of an exercise, and if an exercise includes multiple patterns of movement, each pattern of movement can correspond to a unit movement. For example, if the exercise is dancing, the individual choreographies included in the dance can correspond to unit movements. Also, if the exercise is squats, the transition from a standing position to a squatting position, and the transition from a squatting position to a standing position, can each correspond to unit movements.

[0089] The unit motion may be a more subdivided movement than the example described above. For example, in a leg-raising exercise, the unit motion may be a movement subdivided into raising one leg and lowering one leg once.

[0090] A unit movement may be a set pattern of movements. For example, in leg raises, a unit movement may be a set of movements where the left and right thighs are raised and lowered multiple times. Also, for example in radio calisthenics, a unit movement may be a series of movements performed while counting from 1 to 8.

[0091] The index reference value may be derived by having multiple individuals perform the exercise corresponding to the index reference value, measuring the exercise load for each individual included in the exercise, applying the measurement results of the exercise load for each unit exercise to a predetermined formula to calculate each individual's personal index, and then averaging each individual's personal index. The index reference value may also be used as a substitute index for the standard load described above.

[0092] As described above, the exercise load can be measured for each unit exercise. Therefore, a standard load can be associated with each unit exercise. The standard load associated with a unit exercise is also called the unit standard load to distinguish it from the standard load for the entire exercise. Information regarding the types of exercises targeted in exercise therapy may include information that associates the unit standard load with each unit exercise, which is a subdivision of the target exercises. The target exercises may be defined such that the unit standard loads for each unit exercise are equal.

[0093] <Exercise performed by 70 exercise users> The exercise user 70 may perform exercises selected from a list of target exercises defined to provide an appropriate exercise load for the exercise therapy of the exercise user 70. The exercise user 70 may also perform exercises prescribed by a physician. In this example, the exercise user 70 performs the exercises while watching a sample video of the exercise they are performing. The following describes the specific manner in which the exercise user 70 selects and plays a sample video and performs the exercises while watching the sample video.

[0094] The processor 11 of the information processing device 10 receives input from the input unit 14 for the exercise user 70 to select a sample video. The processor 11 may present all pre-prepared sample videos to the exercise user 70 and allow the exercise user 70 to select a sample video. If an upper limit on the exercise load for the exercise user 70 is set, the processor 11 may not present the exercise user 70 with sample videos of exercises defined to result in an exercise load exceeding the set upper limit, but may only present the exercise user 70 with sample videos of exercises defined to result in an exercise load below the set upper limit. If the exercises to be performed by the exercise user 70 have been determined by the person in charge of planning or instructing the exercise therapy for the exercise user 70, the processor 11 may only present the exercises of the determined exercises to the exercise user 70.

[0095] As illustrated in Figure 3, the processor 11 plays a sample video selected by the exercise user 70 and displays it in the sample video frame 72 of the display device, which is the output unit 15 of the information processing device 10. The processor 11 may play a video that has been pre-recorded and stored in the server 20 or the storage unit 12 of the information processing device 10 as a sample video and display it on the display device. The processor 11 may also display a sample video that is delivered in real time from an external device such as the server 20 on the display device.

[0096] The exercise user 70 performs the exercise while watching the sample video displayed in the sample video frame 72.

[0097] The processor 11 may display a remaining time display frame 73 for the video on the display device when playing a sample video. The processor 11 may display operation buttons 74 on the display device for inputting operations such as playing, pausing, stopping, fast forwarding, or rewinding the sample video. The processor 11 may also accept operation inputs related to controlling the playback of the sample video as voice instructions from the motor user 70 at the input unit 14.

[0098] The processor 11 may display a user video in a user video frame 71 located next to the sample video frame 72 that displays the sample video. By performing the exercise while comparing the sample video and the user video, the exercise user 70 can easily recognize the difference between the movements shown in the sample video and the exercise user 70's own movements.

[0099] As illustrated in Figure 2, the exercise user 70 may wear the wearable device 30. In the example in Figure 2, the wearable device 30 is worn on the exercise user 70's wrist, but it may be worn on various other parts of the body.

[0100] In this example, the wearable device 30 measures the heart rate of the exercise user 70 and outputs the heart rate to the information processing device 10. The processor 11 of the information processing device 10 acquires the measurement result of the exercise user 70's heart rate from the wearable device 30 via the communication unit 13 or the input unit 14. The processor 11 may display the measurement result of the exercise user 70's heart rate as a heart rate display frame 75 on the display device, which is the output unit 15 of the information processing device 10. The processor 11 may display the exercise user 70's current heart rate in the heart rate display frame 75, or it may display a graph of the time change of past heart rates.

[0101] <Evaluation of exercise performed by 70 exercise users> The processor 11 of the information processing device 10 evaluates the exercise performed by the exercise user 70 based on the user video. The processor 11 may evaluate whether the estimated load, which is an estimated value of the exercise load when the exercise user 70 performs the exercise, is constant. The processor 11 may evaluate the difference between the estimated load and the standard load of the exercise type corresponding to the sample video. The following describes specific embodiments of evaluating the exercise performed by the exercise user 70.

[0102] The processor 11 may acquire user video from the camera 16 after the exercise user 70 has performed the exercise, and evaluate whether the estimated load, which is an estimated value of the exercise load when the exercise user 70 performed the exercise, was constant based on the user video. The processor 11 may also evaluate the difference between the estimated load and the standard load for the exercise type corresponding to the sample video based on the user video.

[0103] The processor 11 may evaluate the exercise performed by the exercise user 70 using an evaluation model. The evaluation model comprises a video input unit into which a video is input, an evaluation unit that evaluates the exercise performed by the person shown in the video, and an evaluation result output unit that outputs the evaluation results.

[0104] The evaluation unit may evaluate the exercise load of exercise user 70 shown in the user video. The evaluation result regarding the exercise load of exercise user 70 is also called the user exercise evaluation. The evaluation unit may evaluate whether the estimated load, which is an estimated value of the exercise load of exercise user 70 shown in the user video, was constant. If the evaluation unit evaluates that the estimated load was constant, the evaluation result output unit outputs an evaluation result indicating that the estimated load was constant. If the processor 11 obtains an evaluation result from the evaluation model indicating that the estimated load was constant, it may determine that the exercise performed by exercise user 70 shown in the user video is acceptable.

[0105] The evaluation unit may evaluate the estimated load as constant if the fluctuation of the estimated load during the period shown in the user video falls within the fluctuation setting range. The fluctuation setting range is a range that can be set as appropriate. The fluctuation of the estimated load may be calculated as the difference between the maximum and minimum values ​​of the estimated load during the period shown in the user video. If the fluctuation of the estimated load is the difference between the maximum and minimum values ​​of the estimated load, the fluctuation setting range may be specified as a range that is less than or equal to a value that can be set as an appropriate upper limit for the difference between the maximum and minimum values ​​of the estimated load. The upper limit for the difference between the maximum and minimum values ​​of the estimated load may be set to the maximum value that is allowed as fluctuation of the estimated load. The fluctuation setting range may be incorporated into the evaluation model. The evaluation model may be configured so that the fluctuation setting range is input to the video input unit along with the user video. The evaluation unit may calculate the moving average of instantaneous estimates of the exercise load as the estimated load.

[0106] The evaluation unit may evaluate the difference between the estimated exercise load, which is an estimated value of the exercise load of the exercise user 70 shown in the user video, and the standard exercise load for the exercise type corresponding to the sample video that the exercise user 70 imitated in the user video. The standard exercise load for the exercise type corresponding to the sample video may be incorporated into the evaluation model. If the standard exercise load is incorporated into the evaluation model, an evaluation model may be prepared for each sample video. The evaluation model may be configured so that the standard exercise load for the exercise type corresponding to the sample video is input to the video input unit along with the user video.

[0107] The evaluation unit may evaluate whether the difference between the estimated load and the standard load is within the load fluctuation setting range. In this case, the evaluation result output unit outputs an evaluation result indicating that the difference between the estimated load and the standard load was within the load setting range. When the processor 11 obtains an evaluation result indicating that the difference between the estimated load and the standard load was within the load setting range, it may determine that the exercise performed by the exercise user 70 shown in the user video is acceptable. The load setting range is a range that can be set as appropriate. The load setting range may be incorporated into the evaluation model. The evaluation model may be configured so that the load setting range is input to the video input unit along with the user video.

[0108] The evaluation unit may calculate the difference between the estimated load and the standard load. In this case, the evaluation result output unit outputs the difference between the estimated load and the standard load as the evaluation result. The processor 11 may determine that the exercise performed by the exercise user 70 shown in the user video is acceptable if the difference between the estimated load and the standard load is within the load setting range.

[0109] The evaluation unit may calculate a score as an evaluation result of the exercise performed by the exercise user 70 shown in the user video. In this case, the evaluation result output unit outputs the score as the evaluation result. The evaluation unit may calculate a higher score the closer the fluctuation of the estimated load is to zero. The evaluation unit may calculate a higher score the closer the difference between the estimated load and the standard load is to zero. The processor 11 may determine that the exercise performed by the exercise user 70 shown in the user video is acceptable if the score is equal to or greater than the score threshold.

[0110] The evaluation model may be a trained model generated by performing training using training data that associates user videos with evaluation results of exercises performed by the exercise user 70 shown in the user videos. The evaluation results associated with the user videos in the training data may be generated by evaluating the exercise based on the results of actually measuring the exercise load of the exercise user 70 performing the exercise while the exercise is being performed as shown in the user video. Measurement of exercise load may include, for example, a CPX test. Measurement of exercise load may include, for example, heart rate measurement. Measurement of exercise load is not limited to these examples and may be performed by various other means.

[0111] The evaluation model is not limited to a pre-trained model; it may also be a rule-based model or other various other types of models.

[0112] <<Example of a procedure for evaluating exercise>> The processor 11 may execute an information processing method, including the steps of the flowchart illustrated in Figure 4, to evaluate the movements performed by the exercise user 70 shown in the user video based on the user video. The information processing method may be implemented as an information processing program to be executed by the processor 11. The information processing program may be stored on a non-temporary computer-readable medium.

[0113] Processor 11 acquires user video from camera 16 showing exercise user 70 performing exercises (step S1). Processor 11 inputs the user video into the evaluation model (step S2). The evaluation model outputs evaluation results regarding the exercises performed by exercise user 70 as seen in the input user video. Processor 11 acquires and outputs the evaluation results regarding the exercises performed by exercise user 70 from the evaluation model (step S3). Processor 11 may output a pass / fail judgment for the exercises performed by exercise user 70 based on the evaluation results. Processor 11 may also output the evaluation results themselves acquired from the evaluation model. After executing the procedure in step S3, Processor 11 terminates the execution of the procedure in the flowchart in Figure 4.

[0114] If the evaluation model outputs a score as an evaluation result, the processor 11 may execute an information processing method including the steps of the flowchart illustrated in Figure 5.

[0115] The processor 11 obtains a score from the evaluation model (step S11). The processor 11 determines whether the score is equal to or greater than the score threshold (step S12).

[0116] If the score is above the score threshold (step S12: YES), the processor 11 outputs a pass judgment indicating that the exercise performed by the exercise user 70 shown in the user video input to the evaluation model is acceptable (step S13). After executing the procedure in step S13, the processor 11 terminates the execution of the procedure in the flowchart in Figure 5.

[0117] If the score is not equal to or greater than the score threshold (step S12: NO), i.e., if the score is less than the score threshold, the processor 11 terminates the execution of the steps in the flowchart of Figure 5 without outputting a pass judgment. If the score is less than the score threshold, the processor 11 may output a fail judgment indicating that the exercise performed by the exercise user 70 shown in the user video input to the evaluation model was unsuccessful.

[0118] <<Summary of Exercise Evaluation>> As described above, the information processing device 10 relating to this disclosure can easily evaluate the exercise performed by the exercise user 70 based on user video footage of the exercise user 70 performing the exercise. The exercise user 70 can improve their own exercise according to the evaluation results. As a result, the exercise of the exercise user 70 is appropriately supported. In addition, the effectiveness and safety of exercise therapy are enhanced when the exercise user 70 performs exercise therapy.

[0119] <Evaluation of Unit Motion> As described above, exercise types can be subdivided into individual movements. The processor 11 of the information processing device 10 may evaluate whether the exercise load is constant throughout the entire movement shown in the user video, or it may evaluate the exercise load for each individual movement included in the movement shown in the user video. The processor 11 may evaluate the exercise load for each individual movement for all individual movements. The processor 11 may evaluate the exercise load for each individual movement for at least some of the individual movements. Conversely, the processor 11 does not have to evaluate the exercise load for some of the individual movements.

[0120] The evaluation model may be configured to subdivide the movement into unit movements and output the results of evaluating the exercise load for each unit movement. For example, the evaluation model may include a subdivision unit that subdivides the movement shown in the user video input to the input unit 14 into unit movements and outputs them to the evaluation unit. Furthermore, the evaluation unit of the evaluation model may be configured to include a function for subdividing the movement into unit movements.

[0121] The evaluation unit evaluates the exercise load for each subdivided unit movement. The evaluation unit may evaluate whether the estimated unit load, which is an estimated value of the exercise load of a unit movement included in the exercise performed by exercise user 70 shown in the user video, was constant for each unit movement. Specifically, the evaluation unit may evaluate that the estimated unit load was constant if the fluctuation of the estimated unit load was within the unit fluctuation setting range. The unit fluctuation setting range may be set to the same range as the fluctuation setting range, or to a narrower or wider range than the fluctuation setting range. The evaluation unit may calculate the moving average of the instantaneous estimates of the exercise load of the unit movement as the estimated unit load. If the length of the unit movement is not long enough to calculate a moving average, the evaluation unit may calculate the average of the instantaneous estimates of the exercise load of the unit movement as the estimated unit load.

[0122] If the evaluation unit evaluates that there are unit movements with a constant estimated unit load, the evaluation result output unit may output information identifying the unit movements that were evaluated as having a constant estimated unit load as part of the evaluation result. The processor 11 may determine that the unit movements identified in the evaluation result are acceptable. The processor 11 may also determine that the movements performed by the exercise user 70 shown in the user video are acceptable if the number of unit movements determined to be acceptable is greater than or equal to a predetermined number. The predetermined number may be set appropriately according to the number of unit movements included in the movements shown in the user video. For example, the predetermined number may be set to a value obtained by multiplying the number of unit movements included in the movements shown in the user video by a predetermined ratio. The predetermined ratio may be set appropriately.

[0123] The evaluation unit may evaluate whether the difference between the unit standard load and the unit estimated load for each unit movement falls within the unit load setting range. The unit load setting range may be set to the same range as the load setting range, or to a narrower or wider range than the load setting range. The evaluation unit may calculate the moving average of the instantaneous estimated values ​​of the exercise load of the unit movement as the unit estimated load, and calculate the difference between the unit standard load and the unit estimated load. If the length of the unit movement is shorter than the period for calculating the moving average, the evaluation unit may calculate the average value of the estimated values ​​of the exercise load of the unit movement as the unit estimated load.

[0124] If the evaluation unit evaluates that there are unit movements in which the difference between the unit standard load and the unit estimated load is within the unit load setting range, the evaluation result output unit may output information as an evaluation result that identifies the unit movements in which the difference between the unit standard load and the unit estimated load was evaluated to be within the unit load setting range. The processor 11 may determine that the unit movements identified in the evaluation result are acceptable. Furthermore, if the number of unit movements determined to be acceptable is greater than or equal to a predetermined number, the processor 11 may determine that the entire set of movements, including the unit movements whose exercise load was evaluated, is acceptable.

[0125] The evaluation unit may calculate a unit motion score for each unit motion as an evaluation result of the exercise load of the unit motion. In this case, the evaluation result output unit outputs the unit motion score for each unit motion as the evaluation result. The evaluation unit may calculate a higher unit motion score the closer the variation in the estimated unit load is to zero. The evaluation unit may calculate a higher unit motion score the closer the difference between the estimated unit load and the standard unit load is to zero.

[0126] The processor 11 may determine that a unit motion corresponding to a unit motion score is acceptable if the unit motion score is equal to or greater than the unit motion score threshold. Furthermore, the processor 11 may determine that the entire exercise, including the unit motions for which the exercise load was evaluated, is acceptable if the number of unit motions with unit motion scores equal to or greater than the unit motion score threshold is greater than or equal to a predetermined number.

[0127] The processor 11 may calculate the average unit motion score for each unit motion, and if the average value is equal to or greater than the unit motion score threshold, it may determine that the entire exercise, including the unit motions for which the exercise load was evaluated, is satisfactory.

[0128] The processor 11 may determine that the entire exercise, including the unit movements for which the exercise load was evaluated, is satisfactory if the number of unit movements for which the unit movement score is equal to or greater than the unit movement score threshold is a predetermined number or greater, and the average value of the unit movement scores of each unit movement is equal to or greater than the unit movement score threshold.

[0129] If the evaluation model is configured to evaluate the exercise load of a unit movement, it may be a trained model generated by performing training using training data that includes data associating user videos with information that subdivides the movements shown in the user videos into unit movements. The training data may include data associating user videos with the movements shown in the user videos and the evaluation results of the movements performed by user 70, similar to the training data used to generate an evaluation model that does not consider unit movements.

[0130] Even when the evaluation model is configured to evaluate the exercise load of a unit motion, it is not limited to a pre-trained model, but may be a rule-based or other type of model.

[0131] Each of the subdivided unit movements may be the same movement. For example, a squat is an exercise in which a person repeatedly moves from a standing position to a squatting position and then back up. When a single movement is repeated, each repeated movement may be considered a unit movement. For example, an exercise that is repeated 20 times may be subdivided into 20 unit movements.

[0132] Multiple subdivided unit movements may include at least one different movement. In other words, at least one unit movement among multiple subdivided unit movements may be a different movement. For example, dance is a type of exercise that includes movements with different choreography. Therefore, movements with different choreography may be considered as different unit movements from one another.

[0133] Among the subdivided unit motions, the unit motions that are different from each other may be motions defined to occur over the same length of time, or they may be motions defined to occur over different length of time.

[0134] <<Example of a procedure for evaluating unit motion>> The processor 11 may execute an information processing method including the steps of the flowchart exemplified in Figure 6 when the evaluation model outputs a unit motion score in order to subdivide and evaluate the movements performed by the exercise user 70 shown in the user video based on the user video.

[0135] Processor 11 acquires user video footage of the exercise user 70 performing an exercise, inputs the user video into the evaluation model, and obtains a unit exercise score from the evaluation model as an evaluation result of the exercise load of the unit exercises included in the user video (step S21). Processor 11 calculates the average value of the unit exercise score corresponding to each of the multiple unit exercises and determines whether the average value of the unit exercise scores is equal to or greater than the score threshold (step S22). If the average value of the unit exercise scores is equal to or greater than the score threshold (step S22: YES), Processor 11 proceeds to step S24.

[0136] If the average unit motion score is not equal to or greater than the score threshold (step S22: NO), that is, if the average unit motion score is less than the score threshold, the processor 11 determines whether each unit motion is successful. Specifically, the processor 11 compares the unit motion score with the score threshold for each unit motion and determines that a unit motion in which the unit motion score is equal to or greater than the score threshold is successful. The processor 11 then determines whether the number of unit motions determined to be successful is equal to or greater than a predetermined number (step S23). If the number of unit motions determined to be successful is not equal to or greater than a predetermined number (step S23: NO), the processor 11 terminates the execution of the procedure in the flowchart of Figure 6.

[0137] If the average unit movement score is equal to or greater than the score threshold (step S22: YES), or if the number of unit movements judged as passing is equal to or greater than a predetermined number (step S23: YES), the processor 11 outputs a pass judgment indicating that the movement performed by the movement user 70 shown in the user video is passable (step S24). After executing step S24, the processor 11 terminates the execution of the steps in the flowchart of Figure 6. If the conditions for outputting a pass judgment are not met, the processor 11 may output a fail judgment indicating that the movement performed by the movement user 70 shown in the user video input to the evaluation model is failing. In the steps of the flowchart of Figure 6, the processor 11 may execute only one of steps S22 or S23 instead of executing both steps S22 and S23.

[0138] <<Summary of the evaluation of unit motion>> As described above, the information processing device 10 according to this disclosure can subdivide the exercise performed by the exercise user 70 into unit exercises based on user video footage of the exercise user 70 performing exercises, and evaluate the exercise load for each unit exercise in a simple and detailed manner. The exercise user 70 can then make detailed improvements to their own exercise according to the evaluation results. As a result, the exercise of the exercise user 70 is appropriately supported. Furthermore, the effectiveness and safety of exercise therapy are enhanced when the exercise user 70 performs exercise therapy.

[0139] The processor 11 may perform an evaluation of the exercise load in a manner that does not consider individual movements, in addition to evaluating the exercise load for each individual movement. For example, the evaluation model may be configured to output the results of evaluating the exercise load for each individual movement, as well as the results of evaluating the exercise load for the entire exercise.

[0140] <Evaluation of exercise load levels for 70 exercise users> The processor 11 of the information processing device 10 may perceive the exercise load of the exercise user 70 as a level and evaluate at what level the exercise load of the exercise user 70 remains constant throughout the entire duration of the user video. The level of the exercise load of the exercise user 70 may be set as a reference level when the exercise load of the exercise user 70 is within the AT setting range. In addition, if the exercise load of the exercise user 70 is greater than the upper limit of the AT setting range, the level of the exercise load of the exercise user 70 may be set as at least one excess level, divided into ranges of the same width as the reference level. The excess level may be expressed to include the degree of the excess of the exercise load from the reference level, for example, level +1 or level +2. Level +1 and level +2 represent levels one and two steps above the reference level, respectively. In addition, if the exercise load of the exercise user 70 is less than the lower limit of the AT setting range, the level of the exercise load of the exercise user 70 may be set as at least one deficiency level, divided into ranges of the same width as the reference level. The deficit level may be expressed in a way that includes the degree of insufficiency in exercise load compared to the baseline level, for example, Level-1 or Level-2. Level-1 and Level-2 represent levels one and two steps below the baseline level, respectively.

[0141] If the processor 11 evaluates that the exercise load level of exercise user 70 remains constant throughout the entire user video, it can evaluate at what level the exercise load level of exercise user 70 remains constant throughout the entire user video by estimating the exercise load level of exercise user 70 at least a portion of the user video or at a specific point in time.

[0142] The processor 11 may estimate the exercise load level of the exercise user 70 based on the exercise user 70's state during exercise. The processor 11 may estimate the exercise load level of the exercise user 70 for at least a portion of the user video's duration or time period based on the exercise user 70's state during exercise as shown in the user video. The exercise user 70's state during exercise may or may not include indicators such as oxygen consumption that define the exercise load.

[0143] The state of exercise user 70 during exercise may be measured as perceived exercise intensity. Perceived exercise intensity is an index that numerically represents the subjective burden when performing exercise. For example, the Borg index is a well-known index of perceived exercise intensity. The Borg index represents the subjective burden using integers from 6 to 20. If the Borg index felt by exercise user 70 when performing exercise is 13, it is considered that the level of exercise load for exercise user 70 is at the level of exercise load corresponding to exercise user 70's AT, i.e., the baseline level. The processor 11 may obtain the measurement result of the subjective exercise intensity of exercise user 70 as the state of exercise user 70 during exercise, for example, by receiving input of the Borg index from exercise user 70 at the input unit 14 after exercise user 70 has performed exercise.

[0144] The state of the exercise user 70 during exercise may be measured by a talk test. The talk test is performed by a specialist such as a physician or physical therapist. The talk test determines whether the exercise load of the exercise user 70 is around or outside the exercise load equivalent to the exercise user 70's athletic threshold (AT). In other words, the talk test determines whether the exercise load of the exercise user 70 is within or outside the AT setting range. The talk test may be performed using a trained model that mimics the talk test. The processor 11 may obtain the results of the talk test after the exercise user 70 has performed the exercise as the state of the exercise user 70 during exercise.

[0145] The exercise user 70's condition during exercise may be measured based on the analysis results of the sweat or lactic acid contained in the exercise user 70's blood, or the analysis results of the exercise user 70's exhaled gas. The processor 11 may obtain, for example, the analysis results of the sweat or lactic acid contained in the exercise user 70's blood, or the analysis results of the exercise user 70's exhaled gas, as the exercise user 70's condition during exercise, or after the exercise user 70 has performed the exercise.

[0146] The state of exercise user 70 during exercise may be obtained as the average state of exercise user 70 over the entire duration of the user video. The state of exercise user 70 during exercise may be obtained as the average state of exercise user 70 within one period of the entire duration of the user video. The state of exercise user 70 during exercise may be measured as a state that is an average of the average states of exercise user 70 over multiple periods of the entire duration of the user video. The state of exercise user 70 during exercise may be measured as the instantaneous state of exercise user 70 at one point in time within the entire duration of the user video. The state of exercise user 70 during exercise may be measured as a state that is an average of the instantaneous states of exercise user 70 at multiple points in time within the entire duration of the user video.

[0147] The input unit of the evaluation model may be configured to accept input of the state of the exercise user 70 during at least a portion of the period or time point in time while performing the exercise shown in the user video. The evaluation unit of the evaluation model may estimate the level of exercise load of the exercise user 70 during the period or time point corresponding to that state, based on the state of the exercise user 70 input to the input unit. The estimated result of the exercise load level of the exercise user 70 is also referred to as the estimated level.

[0148] The evaluation unit may evaluate, based on the user video, that the exercise load level of exercise user 70 remains constant throughout the entire duration of the user video, and if it is possible to estimate the exercise load level of exercise user 70, it may evaluate that the exercise load level of exercise user 70 remains constant at the estimated level. Furthermore, if the estimated exercise load level of exercise user 70 is the reference level for exercise user 70, the evaluation unit may evaluate that the exercise load level of exercise user 70 remains constant at the reference level throughout the entire duration of the user video.

[0149] In the examples described above, the range of each level used to determine the exercise load level of the exercise user 70 was set to the range of the AT setting range. In this case, the range of the variation setting range may be set to be the same as or less than the range of the AT setting range. By setting the range of the variation setting range based on the range of the AT setting range, if the exercise load level of the exercise user 70 at any point in time or period within the user video is at the reference level, then even if the exercise load of the exercise user 70 fluctuates within the variation setting range, the exercise load level of the exercise user 70 will remain at least within level ±1 for the entire duration of the user video. In other words, if the exercise load level of the exercise user 70 at any point in time or period within the user video is at the reference level, the evaluation unit can evaluate that the exercise load level of the exercise user 70 is constant within the range of the reference level or level ±1.

[0150] The range of the baseline level may be set to less than the range of the AT setting range. In this way, even if the exercise load of exercise user 70 fluctuates within the variable setting range, the level of exercise load of exercise user 70 is more likely to stay within the baseline level throughout the entire duration of the user video.

[0151] Furthermore, the range of the reference level and the range of the fluctuation setting may be set to less than 1 / 3 of the range of the AT setting. In this way, if the level of exercise load of exercise user 70 is at the reference level during at least a portion of the user video period or time, even if the exercise load of exercise user 70 fluctuates within the fluctuation setting range, the level of exercise load of exercise user 70 will remain at the reference level throughout the entire user video period. In other words, if the level of exercise load of exercise user 70 is at the reference level during at least a portion of the user video period or time, the evaluation unit can evaluate that the level of exercise load of exercise user 70 is constant within the range of the reference level or level ±1.

[0152] The processor 11 inputs the user video along with the state of the exercise user 70 at at least a portion of the time or point in time while performing the exercise shown in the user video into the evaluation model. This allows the processor 11 to obtain evaluation results indicating that the exercise load of the exercise user 70 remains constant throughout the entire user video, as well as evaluation results indicating that the level of the exercise load of the exercise user 70 remains constant within the estimated level, reference level, or level ±1 throughout the entire user video. In this case, the processor 11 does not need to measure values ​​such as oxygen consumption that define the exercise load of the exercise user 70. As a result, exercise can be evaluated in a simple manner.

[0153] If we evaluate whether the exercise load level of exercise user 70 remains constant within the standard level or within ±1 of the standard level throughout the entire duration of the user video, we can conclude that exercise user 70 is able to safely perform exercise at or around the exercise load equivalent to the athletic anaerobic threshold (AT). As a result, exercise user 70's exercise is appropriately supported. Furthermore, the effectiveness and safety of exercise therapy are enhanced when exercise user 70 undergoes exercise therapy.

[0154] <Real-time evaluation of exercise> The processor 11 may evaluate in real time whether the exercise load of the exercise user 70 remains constant based on the user video while the exercise user 70 is performing the exercise. The processor 11 may also evaluate in real time the difference between the exercise load of the exercise user 70 and the standard load of the exercise corresponding to the sample video, based on the user video while the exercise user 70 is performing the exercise. Furthermore, the processor 11 may acquire the state of the exercise user 70 at least a portion of the exercise during the exercise and evaluate in real time whether the level of the exercise load of the exercise user 70 remains constant within the estimated level, the reference level, or within the range of level ±1, based on the user video and the state of the exercise user 70.

[0155] In this disclosure, real-time evaluation means outputting an evaluation of the content performed up to an intermediate evaluation point in the exercise being evaluated, with a delay of a predetermined time from the evaluation point. By evaluating the exercise of the exercise user 70 in real time, the exercise user 70 can improve their movements while performing the exercise. As a result, the exercise of the exercise user 70 is appropriately supported. In addition, both the effectiveness and safety of the exercise therapy are enhanced while the exercise is being performed.

[0156] While embodiments relating to this disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications or alterations based on this disclosure. Therefore, it should be noted that these modifications or alterations are within the scope of this disclosure. For example, the functions included in each component can be rearranged in a logically consistent manner, and multiple components can be combined into one or separated. [Explanation of Symbols]

[0157] 1. Information Processing System 10. Information Processing Device (11: Processor, 12: Memory Unit, 13: Communication Unit, 14: Input Unit, 15: Output Unit, 16: Camera) 20 servers 30 Wearable Devices 40 Networks 70 Exercise Users 71 User video slots 72 Sample video frame 73 Remaining time display frame 74 Operation Buttons 75 Heart rate display frame

Claims

1. This involves obtaining user videos of users performing various exercises, The user video is input into an evaluation model that outputs evaluation results regarding the movements captured in the input video. From the evaluation model, the evaluation result is obtained as a user exercise evaluation regarding the amount of exercise load of the user shown in the user video. Make the processor execute it, The user exercise evaluation includes an evaluation of whether the fluctuation of the estimated exercise load, which is an estimated value of the exercise load of the user as seen in the user video, was within the set fluctuation range. Information processing program.

2. The information processing program according to claim 1, wherein the exercise type is defined such that the exercise load is constant when the exercise is performed according to the exercise type.

3. The information processing program according to claim 1 or 2, wherein the user exercise evaluation includes an evaluation of whether the difference between the standard load, which is the standard exercise load when the user performs the exercise, and the estimated load was within the load setting range.

4. The information processing program according to claim 1, wherein the exercise type includes a plurality of unit exercises, and the unit standard loads, which are the standard exercise loads when the user performs each of the plurality of unit exercises, are defined to be equal.

5. The information processing program according to claim 4, wherein the user exercise evaluation includes an evaluation of whether the fluctuation of the estimated unit load, which is an estimated value of the exercise load of the user for each of the plurality of unit exercises, was within the unit fluctuation setting range.

6. The information processing program according to claim 5, wherein the user motion evaluation includes the number of unit motions in which the difference between the unit standard load and the unit estimated load was within the unit load setting range.

7. The information processing program according to claim 1, wherein the user exercise evaluation includes a score calculated with a higher value the closer the fluctuation of the estimated load is to zero.

8. The information processing program according to claim 4, wherein the user exercise evaluation includes a unit exercise score calculated with a higher value the closer the variation in the estimated unit exercise load, which is an estimated value of the exercise load of the user for each of the plurality of unit exercises, is to zero.

9. The information processing program according to claim 1, wherein the standard load of the exercise performed by the user is an exercise load corresponding to the user's anaerobic metabolic threshold.

10. The aforementioned evaluation model, The system is configured to allow further input of the user's state during at least a portion of the entire duration of the user video or at least a portion of the time period. The system is configured to output as the evaluation result that the level of exercise load of the user remains constant at an estimated level based on the user's state throughout the entire duration of the user video. The information processing program according to claim 1.

11. The information processing program according to claim 10, wherein the evaluation model is configured to output as an evaluation result that the level of the user's exercise load is constant at the reference level when the estimated level is the reference level.

12. The information processing program according to claim 1, wherein the exercise type is defined so that the user can perform the exercise while standing.

13. An information processing device equipped with a processor, The aforementioned processor, We obtain user videos of users performing exercises. The user video is input to an evaluation model that outputs evaluation results regarding the movements captured in the input video. From the evaluation model, the user exercise evaluation regarding the amount of exercise load of the user as seen in the user video is obtained as the evaluation result. The user exercise evaluation includes an evaluation of whether the fluctuation of the estimated exercise load, which is an estimated value of the exercise load of the user as seen in the user video, was within the set fluctuation range. Information processing device.

14. The processor acquires user videos of users performing exercises, The processor inputs the user video to an evaluation model that outputs evaluation results regarding the motion captured in the input video. The processor obtains, from the evaluation model, a user exercise evaluation regarding the amount of exercise load of the user as seen in the user video, as the evaluation result. Includes, The user exercise evaluation includes an evaluation of whether the fluctuation of the estimated exercise load, which is an estimated value of the exercise load of the user as seen in the user video, was within the set fluctuation range. Information processing methods.

Images