Information processing device, method, program, and system

Abstract

To provide a judgment material for planning or guiding an exercise therapy, or a signal showing a physical function of a subject.SOLUTION: A program allows a computer to function as means for acquiring first exercise intensity when a user is performing a first exercise type on the basis of first data based on a sensing result when the user is performing the first exercise type, means for acquiring second exercise intensity when a user is performing a second exercise type different from the first exercise type, when the user is performing the second exercise type on the basis of second data based on sensing during resting, or during resting, means for determining a user's individual index by performing a predetermined calculation to multiple pieces of exercise intensity that include the first exercise intensity and the second exercise intensity and are acquired for the user, and means for calculating a parameter expressing a user's features regarding exercise tolerability on the basis of the user's individual index and reference value.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a method, a program, and a system. [Background technology]

[0002] The importance of exercise therapy, which utilizes exercise for the treatment and prevention of disorders and diseases, is increasing. For example, cardiac rehabilitation aims to help heart disease patients regain their physical strength and confidence, return to a comfortable home and social life, and prevent recurrence of heart disease or re-hospitalization through a comprehensive activity program that includes exercise therapy. The core of exercise therapy is aerobic exercise, such as walking, jogging, cycling, and aerobics. To perform aerobic exercise more safely and effectively, it is preferable for the subject to exercise at an intensity near their anaerobic threshold (AT).

[0003] The anaerobic metabolic threshold is an example of an evaluation index of exercise tolerance and corresponds to a change point in cardiopulmonary function, i.e., an exercise intensity near the boundary between aerobic exercise and anaerobic exercise. The anaerobic metabolic threshold is generally determined by a cardiopulmonary exercise test (CPX test), in which a test subject is subjected to a gradually increasing exercise load while exhaled gas is collected and analyzed (see Non-Patent Document 1). In a CPX test, the anaerobic metabolic threshold is determined based on the results measured by exhaled gas analysis (e.g., oxygen intake, carbon dioxide output, tidal volume, respiratory rate, minute ventilation, or a combination thereof). In addition to the anaerobic metabolic threshold, a CPX test can also determine the maximum oxygen intake, which corresponds to an exercise intensity near the maximum exercise tolerance.

[0004] To ensure the effectiveness and safety of exercise therapy, it is important to properly manage the exercise intensity of the subject.

[0005] Patent Document 1 describes that whether or not the ventilatory threshold (VT) has been reached is determined based on the subject's pulse information, and the exercise load of the exercise providing device is adjusted according to the determination result. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2022-059494 [Non-patent literature]

[0007] [Non-Patent Document 1] Muneyasu Saito, Cardiac Rehabilitation, Physical Therapy, 1997, Vol. 24, No. 8, pp. 414-418 Summary of the Invention [Problem to be solved by the invention]

[0008] The technical concept described in Patent Document 1, in general, performs feedback control so that exercise intensity approaches the ventilatory threshold (VT). However, the exercise intensity for the exerciser depends not only on the type of exercise but also on the individual's physical function and daily physical condition. Therefore, even if this technical concept is applied to exercise therapy, it is not possible to obtain information on the level of exercise intensity that a specified type of exercise will require for the subject, or on what type of exercise should be recommended for the subject to achieve the specified exercise intensity. Furthermore, the technical concept described in Patent Document 1 cannot be simply applied, for example, to cases where a subject is required to exercise at an exercise intensity lower than the ventilatory threshold (VT).

[0009] An object of the present disclosure is to provide information for planning or guiding an exercise regimen, or a signal indicating a subject's physical function. [Means for solving the problem]

[0010] A program according to one embodiment of the present disclosure causes a computer to function as: means for acquiring first data based on sensing results obtained when a user is performing a first type of exercise; means for acquiring a first exercise intensity obtained when the user is performing the first type of exercise based on the first data; means for acquiring second data based on sensing obtained when the user is performing a second type of exercise different from the first type of exercise or at rest; means for acquiring a second exercise intensity obtained when the user is performing the second type of exercise or at rest based on the second data; means for determining a first personal index of the user by performing a predetermined calculation on multiple exercise intensities obtained for the user, including the first exercise intensity and the second exercise intensity; and means for calculating a parameter representing the user's characteristics related to exercise tolerance based on the user's first personal index and a first reference value. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a block diagram showing the configuration of a client device according to the present embodiment. [Figure 3] FIG. 2 is a block diagram showing the configuration of a server according to the present embodiment. [Figure 4] FIG. 1 is a block diagram showing the configuration of a wearable device according to an embodiment of the present invention. [Figure 5] FIG. 1 is an explanatory diagram of one aspect of the present embodiment. [Figure 6] FIG. 2 is a diagram showing the data structure of an exercise event database according to the present embodiment. [Figure 7] FIG. 2 is a diagram showing the data structure of a user profile database according to the present embodiment. [Figure 8] FIG. 2 is a diagram illustrating a data structure of a parameter log database according to the present embodiment. [Figure 9] 10 is a flowchart of an exercise event recommendation process according to the present embodiment. [Figure 10] 10A and 10B are diagrams illustrating an example of a screen displayed in the exercise type recommendation process of the present embodiment. [Figure 11]10 is a flowchart of a parameter monitoring process according to the present embodiment. [Figure 12] FIG. 10 is a diagram showing the data structure of a training data set that can be used in this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the drawings for explaining the embodiment, the same components are generally designated by the same reference numerals, and repeated description thereof will be omitted.

[0013] (1) Information processing system configuration The configuration of the information processing system will now be described with reference to Fig. 1, which is a block diagram showing the configuration of the information processing system according to this embodiment.

[0014] As shown in FIG. 1, the information processing system 1 includes a client device 10, a server 30, and a wearable device 50.

[0015] Here, the number of client devices 10 and wearable devices 50 varies depending on, for example, the number of users. Therefore, the number of client devices 10 and wearable devices 50 may each be two or more. Furthermore, a terminal of a person who plans or instructs exercise therapy may also be included in the information processing system 1. The person who plans or instructs exercise therapy may include, for example, a medical professional (e.g., a doctor, nurse, pharmacist, physical therapist, occupational therapist, or clinical laboratory technician), a nutritionist, or a trainer.

[0016] The client device 10 and the server 30 are connected via a network (for example, the Internet or an intranet) NW. The client device 10 and the wearable device 50 are connected via a wireless channel using, for example, Bluetooth (registered trademark) technology.

[0017] The client device 10 is an example of an information processing device that transmits a request to the server 30. The client device 10 is, for example, a smartphone, a tablet terminal, or a personal computer.

[0018] The server 30 is an example of an information processing device that provides the client device 10 with a response in response to a request transmitted from the client device 10. The server 30 is, for example, a server computer.

[0019] The wearable device 50 is an example of an information processing device that can be worn on the user's body (for example, on the arm, hand, or head).

[0020] (1-1) Client device configuration The configuration of the client device will now be described with reference to Fig. 2, which is a block diagram showing the configuration of the client device of this embodiment.

[0021] 2, the client device 10 includes a storage device 11, a processor 12, an input / output interface 13, and a communication interface 14. The client device 10 is connected to a display 15, a camera 16, a depth sensor 17, a microphone 18, and an acceleration sensor 19.

[0022] The storage device 11 is configured to store programs and data, and is, for example, a combination of a read-only memory (ROM), a random access memory (RAM), and a storage (for example, a flash memory or a hard disk).

[0023] The programs include, for example, the following programs: OS (Operating System) programs · Programs for applications that process information (e.g., web browsers, therapy apps, rehabilitation apps, or fitness apps) Here, the diseases that are the target of therapeutic or rehabilitation apps are, for example, heart disease, lifestyle-related diseases (hypertension, diabetes, dyslipidemia, hyperlipidemia), obesity, and other diseases in which exercise may contribute to improving symptoms.

[0024] The data includes, for example, the following data: Databases referenced in information processing Data obtained by performing information processing (i.e., the results of performing information processing)

[0025] The processor 12 is a computer that implements the functions of the client device 10 by running a program stored in the storage device 11. The processor 12 is, for example, at least one of the following: ·CPU(Central Processing Unit) ·GPU(Graphic Processing Unit) ·ASIC(Application Specific Integrated Circuit) ·FPGA(Field Programmable Gate Array)

[0026] The input / output interface 13 is configured to acquire information (e.g., user instructions, images, sounds) from an input device connected to the client device 10, and to output information (e.g., images, commands) to an output device connected to the client device 10.

[0027] The input device is, for example, a camera 16, a depth sensor 17, a microphone 18, an acceleration sensor 19, a keyboard, a pointing device, a touch panel, a sensor, or a combination thereof. The output device is, for example, a display 15, a speaker, or a combination thereof.

[0028] The communication interface 14 is configured to control communication between the client device 10 and external devices (eg, another client device 10, a server 30, and a wearable device 50). Specifically, the communication interface 14 may include a module (e.g., a WiFi module, a mobile communication module, or a combination thereof) for communication with the server 30. The communication interface 14 may include a module (e.g., a Bluetooth module) for communication with the wearable device 50.

[0029] The display 15 is configured to display an image (a still image or a moving image). The display 15 is, for example, a liquid crystal display or an organic EL display.

[0030] The camera 16 is configured to take pictures and generate image signals.

[0031] The depth sensor 17 is, for example, a light detection and ranging (LIDAR) sensor. The depth sensor 17 is configured to measure the distance (depth) from the depth sensor 17 to a surrounding object (for example, a user).

[0032] The microphone 18 is configured to receive sound waves and generate sound signals, and is preferably placed near the user's body (particularly the respiratory tract) as an earphone microphone, for example.

[0033] The acceleration sensor 19 is configured to detect acceleration.

[0034] (1-2) Server configuration The configuration of the server will now be described with reference to Fig. 3, which is a block diagram showing the configuration of the server according to this embodiment.

[0035] As shown in FIG. 3, the server 30 includes a storage device 31, a processor 32, an input / output interface 33, and a communication interface .

[0036] The storage device 31 is configured to store programs and data, and is, for example, a combination of ROM, RAM, and storage.

[0037] The programs include, for example, the following programs: OS programs Application programs that perform information processing

[0038] The data includes, for example, the following data: Databases referenced in information processing - Results of information processing

[0039] The processor 32 is a computer that implements the functions of the server 30 by running a program stored in the storage device 31. The processor 32 is, for example, at least one of the following: ·CPU GPU ASIC FPGA

[0040] The input / output interface 33 is configured to obtain information (for example, a user's instruction) from an input device connected to the server 30 and to output information to an output device connected to the server 30. The input device is, for example, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, a display.

[0041] The communication interface 34 is configured to control communications between the server 30 and an external device (eg, the client device 10).

[0042] (1-3) Wearable device configuration The configuration of the wearable device will now be described with reference to Figure 4. Figure 4 is a block diagram showing the configuration of the wearable device of this embodiment.

[0043] 4, the wearable device 50 includes a storage device 51, a processor 52, an input / output interface 53, and a communication interface 54. The wearable device 50 is connected to a display 55, a heart rate sensor 56, and an acceleration sensor 57.

[0044] The storage device 51 is configured to store programs and data, and is, for example, a combination of ROM, RAM, and storage.

[0045] The programs include, for example, the following programs: OS programs · Programs for applications that process information (e.g., therapeutic, rehabilitation, or fitness apps)

[0046] The data includes, for example, the following data: Databases referenced in information processing - Results of information processing

[0047] The processor 52 is a computer that executes the programs stored in the storage device 51 to realize the functions of the wearable device 50. The processor 52 is, for example, at least one of the following: ·CPU GPU ASIC FPGA

[0048] The input / output interface 53 is configured to acquire information (e.g., user instructions, sensing results) from an input device connected to the wearable device 50, and to output information (e.g., images, commands) to an output device connected to the wearable device 50.

[0049] The input device is, for example, a heart rate sensor 56, an acceleration sensor 57, a microphone, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, a display 55, a speaker, or a combination thereof.

[0050] The communication interface 54 is configured to control communication between the wearable device 50 and an external device (eg, the client device 10). Specifically, the communication interface 54 may include a module for communication with the client device 10 (eg, a Bluetooth module).

[0051] The display 55 is configured to display an image (still image or moving image). The display 55 is, for example, a liquid crystal display or an organic EL display.

[0052] The heart rate sensor 56 is configured to measure the heart rate and generate a sensing signal. As an example, the heart rate sensor 56 measures the heart rate using an optical measurement technique.

[0053] The acceleration sensor 57 is configured to detect acceleration.

[0054] (2) One aspect of the embodiment An example of this embodiment will now be described with reference to Fig. 5, which is an explanatory diagram of this example.

[0055] As shown in FIG. 5, the client device 10 and the wearable device 50 sense the user US1 while he or she is exercising. The user US sequentially performs multiple exercises. The combination of multiple exercises (hereinafter referred to as a "target exercise set") used to calculate these individual difference parameters (described later) may be fixed or arbitrarily selectable. In the latter case, the exercises constituting the target exercise set may be determined by the user, the person planning or instructing the user's exercise therapy, or an algorithm. Furthermore, the multiple exercises constituting the target exercise set may be selected from exercises with different standard exercise intensities. In either case, information on the standard exercise intensity corresponding to each of the exercises constituting the target exercise set is stored in an exercise database (described later). The user US1 is typically a person undergoing exercise therapy, such as a participant in a (cardiac) rehabilitation program or an exercise instruction program. In the example of FIG. 5, the exercise type is gymnastics, but the exercise type may include any exercise (aerobic exercise or anaerobic exercise).

[0056] As an example, the camera 16 captures an image of the appearance (e.g., the entire body) of the user US1 while exercising from a front or oblique front view at a distance of, for example, about 2 m. The camera 16 may be installed at an appropriate height using a tripod or other height adjustment means. The depth sensor 17 measures the distance (depth) from the depth sensor 17 to each part of the user US1. Note that it is also possible to generate three-dimensional video data by combining, for example, video data (two-dimensional) generated by the camera 16 with depth data generated by, for example, the depth sensor 17. The microphone 18 receives sounds (e.g., sounds generated by breathing or speaking) emitted by the user US1 while exercising and generates a sound signal.

[0057] The heart rate sensor 56 of the wearable device 50 measures the heart rate of the user US1 while exercising and transmits the measurement results to the client device 10. The acceleration sensor 57 measures the acceleration of the user US1 while exercising and transmits the measurement results to the client device 10.

[0058] The client device 10 acquires various types of sensing data and analyzes it as necessary. As an example, the client device 10 may analyze the physical state of the user US1 while exercising by referring to video data acquired from the camera 16. The client device 10 may further refer to depth data acquired from the depth sensor 17 to analyze the physical state of the user US1 while exercising. The client device 10 transmits user data including at least one of the sensing data or the analysis results of the sensing data to the server 30.

[0059] Based on the user data acquired from the client device 10, the server 30 estimates the exercise intensity of the user when performing each of the multiple exercise events constituting the target exercise set. The server 30 then performs a predetermined calculation on the multiple exercise intensities corresponding to each of the multiple exercise events to determine the individual index (described below) of the user US1 for the target exercise set. Furthermore, the server 30 calculates an individual difference parameter that represents the characteristics of the user US1 regarding exercise tolerance (e.g., whether the user US1 tends to perform exercise at a higher or lower intensity than a typical person) based on the determined individual index and a reference value (described below) of the individual index for the target exercise set. For example, because oxygen consumption during exercise depends on muscle mass, individual differences in exercise intensity will occur even for the same exercise event depending on the user US1's muscle development and the body parts subjected to load.

[0060] The server 30 stores the calculated individual difference parameters in the storage device 31. Using the individual difference parameters, the server 30 can predict the exercise intensity of the user US1 when performing an exercise not included in the target exercise set, or estimate what exercise the user US1 should perform to achieve a desired exercise intensity. Furthermore, by continuously monitoring the individual difference parameters, it is possible to grasp the trend of change in the user US1's physical function (e.g., improvement, maintenance, or decline) or the rate of change (e.g., whether it is rapid or not). For example, if the user US1's physical function is rapidly declining, it is possible to output an alert indicating a suspected abnormality, such as an exacerbation of a heart disease such as heart failure, a physical breakdown, or a temporary deterioration in physical condition.

[0061] In this way, the information processing system 1 calculates individual difference parameters that represent the characteristics of the user US1 related to exercise tolerance, based on data based on the sensing results of the user US1 while performing each exercise event that constitutes the target event set. Therefore, according to this information processing system 1, the individual difference parameters can be used as information for planning or instructing an exercise therapy to be provided to the user US1, or as a signal indicating the physical function or physical condition of the user US1.

[0062] (3) Database The databases of this embodiment will be described. The following databases are stored in the storage device 31.

[0063] (3-1) Sports event database The exercise event database of this embodiment will be described below with reference to Fig. 6, which is a diagram showing the data structure of the exercise event database of this embodiment.

[0064] The exercise type database stores exercise type information. The exercise type information is information about exercise types that are candidates for recommended exercise types or target exercise types. In this embodiment, the exercise types include exercise types that can be performed without using equipment with adjustable exercise load, such as gymnastics, bodyweight training, dancing, walking, running, and treadmills. These exercise types include types performed in a standing position, providing a wide variety of options. Furthermore, in this embodiment, the exercise intensity of these exercise types can be adjusted through form (e.g., the positional relationship or angle of parts (which may include parts that are not moved during exercise), the range of motion of parts that are moved during exercise, etc.), pace, number of repetitions, or the duration or number of rest periods. However, the exercise types in this embodiment may also include exercise types performed using equipment with adjustable exercise load, such as an ergometer or strength training using training equipment.

[0065] 6, the exercise event database includes an "ID" field, a "name" field, an "exercise intensity" field, and an "exercise definition" field. Each field is associated with the others.

[0066] The "ID" field stores an exercise type ID. The exercise type ID is information that identifies the exercise type corresponding to the record.

[0067] The "Name" field stores information about the name of the exercise type. The exercise type name information is information about the name of the exercise type corresponding to the record.

[0068] The "exercise intensity" field stores exercise intensity information. The exercise intensity information is information related to the standard exercise intensity of the exercise type corresponding to the record. Standard exercise intensity may refer to the exercise intensity when, for example, a person with standard physical function performs the corresponding exercise type as defined. As an example, such exercise intensity may be derived by actually measuring the exercise intensity (e.g., average oxygen consumption) when one or more people perform the corresponding exercise type as defined, for example, by performing exhaled gas analysis (or estimating it by analyzing video footage of the user exercising), and then statistically processing the results (e.g., averaging).

[0069] Note that exercise intensity may be derived by, for example, measuring (or estimating) the exercise intensity of one or more people performing a corresponding exercise type as defined for each section of the exercise type, applying the measurement results for each section to a predetermined calculation formula for each person, and then statistically processing (e.g., averaging) the values obtained by this calculation formula across people. Here, a section is a component unit of an exercise type, and if an exercise type consists of multiple movement patterns, each movement pattern may correspond to a section. For example, if the exercise type is dance, each choreography may correspond to a section. Also, if the exercise type is squats, the transition from a standing position to a crouching position and the transition from a crouching position to a standing position may each correspond to a section.

[0070] The "exercise definition" field stores exercise definition information. The exercise definition information is information about the definition of the exercise event corresponding to the record. The exercise definition information can include information about at least one of the following elements: Exercise pace (e.g., the time it takes to complete one cycle of movement) - Positional relationship of parts (which may be one or multiple parts) (for example, foot width, etc.) The angle of the part (which may be one or more) (for example, the direction of the knee, the angle between the upper arm and the forearm) The range of motion of the body parts (which may be one or more) that are moved during exercise (for example, the range of movement of each body part during one cycle of movement) Number of reps Break time Exercise load (for example, the magnitude of the external load set when exercising using an ergometer or other device that allows for adjustable exercise load)

[0071] Even for an exercise that is generally recognized as a single event (such as squats), multiple exercise events with slightly different exercise intensities can be defined by specifying details of pace, form (especially range of motion), number of repetitions, rest time, and exercise load. For example, for an exercise that is generally recognized as a single event, multiple exercise events with differences of 0.2 METs can be defined.

[0072] (3-2) User profile database The user profile database of this embodiment will now be described with reference to Fig. 7, which shows the data structure of the user profile database of this embodiment.

[0073] The user profile database stores user profile information, which is information relating to the profile of a user of the information processing system 1 (i.e., a person who exercises).

[0074] 7, the user profile database includes an "ID" field, a "name" field, a "target strength" field, and a "body" field. Each field is associated with the others.

[0075] The "ID" field stores a user ID. The user ID is information that identifies the user corresponding to the corresponding record.

[0076] The "Name" field stores user name information. The user name information is information about the name of the user corresponding to the record (for example, name, account name, etc.).

[0077] The "target intensity" field stores target intensity information (an example of a "predetermined exercise intensity"). The target intensity information is information about a target value of exercise intensity (e.g., oxygen consumption, energy consumption, heart rate, or a combination thereof) set for the user corresponding to the record. As an example, the target intensity information is specified by a person (e.g., a doctor) who plans or instructs exercise therapy based on the results of measuring the user's exercise tolerance, for example, by CPX while using an ergometer (e.g., oxygen consumption and heart rate at the anaerobic threshold (AT)). However, CPX is not required, and the target value may be specified at the doctor's discretion. As another example, the target intensity information is determined by an algorithm based on the results of measuring the user's exercise tolerance, for example, by CPX. In order to perform aerobic exercise more safely and effectively, it is preferable to exercise at an intensity near the anaerobic threshold. Therefore, the target value of exercise intensity is, for example, but not limited to, an exercise intensity corresponding to the anaerobic threshold. While an ergometer is typically used as the exercise type for CPX measurement, the oxygen consumption at the anaerobic threshold when using a treadmill, for example, is approximately 1.2 to 1.3 times that when using an ergometer. This is thought to be because the total muscle mass used on the treadmill exceeds the total muscle mass used on the ergometer. Therefore, as a first example, the target intensity information may be a value obtained by correcting the target value based on the results measured by CPX when using an ergometer to approximately 1.2 to 1.3 times the original value, or by further subtracting a predetermined value (e.g., 1 MET). As a second example, the target intensity information may be a value obtained by correcting the target value based on the results measured by CPX when using an ergometer without correction. This prevents the target value from becoming excessively high when selecting an exercise type that uses a relatively small amount of muscle mass, such as an ergometer. As a third example, the target value for each exercise type may be corrected, for example, using a coefficient corresponding to the total muscle mass used.

[0078] The doctor may also set an upper limit on the exercise intensity for the user (an example of exercise prescription). In this case, the user is not permitted to select an exercise type that exceeds the prescribed upper limit of exercise intensity. For exercise prescription, a UI (User Interface) screen for exercise prescription may be displayed on the display of the terminal used by the doctor. Such a UI screen may include, for example, the following information: User CPX data -Display area for sample videos of multiple exercises that can be selected Here, the display areas of the sample videos for each exercise type may be arranged according to the exercise intensity information corresponding to the exercise type. For example, if 3.6 METs is recommended as the upper limit based on the CPX data, a group of display areas of sample videos for exercise types corresponding to 3.4 METs, a group of display areas of sample videos for exercise types corresponding to 3.6 METs, and a group of display areas of sample videos for exercise types corresponding to 3.8 METs may be arranged on the UI screen. Note that the information on exercise types (e.g., sample videos) may be arranged in units smaller than or larger than 0.2 METs. When the doctor selects one of the display areas, the exercise intensity associated with the corresponding exercise type is set as the upper limit.

[0079] In addition, the upper limit specified by a doctor through exercise prescription may be raised or lowered by a medical professional under the supervision of a doctor during regular (for example, every two weeks) medical guidance or doctor rounds.

[0080] The "Physical" field stores physical information. The physical information is information about the body (functions) of the user corresponding to the record. As an example, the physical information may include information about the user's age, sex, weight, height, illnesses, etc.

[0081] Additionally, the user profile database may store the following information: Information indicating the person who planned or instructed the user's exercise regimen Information indicating the user's doctor

[0082] (3-3) Parameter log database The parameter log database of this embodiment will now be described with reference to Fig. 8, which is a diagram showing the data structure of the parameter log database of this embodiment.

[0083] The parameter log database may be constructed, for example, for each user (i.e., person who exercises) of the information processing system 1. Alternatively, the parameter log database may be configured to store records including information that can identify a user (e.g., a user ID).

[0084] The parameter log database stores parameter log information, which is information relating to a log of individual difference parameters calculated for a user.

[0085] 8, the parameter log database includes a "date" field and a "parameter" field, each of which is associated with the other.

[0086] The "Date" field stores date information. The date information is information about the date (or date and time) when the individual difference parameters of the corresponding record were calculated.

[0087] The "parameter" field stores parameter information, which is information about the value of the individual difference parameter of the corresponding record.

[0088] (4) Information processing The information processing of this embodiment will be described.

[0089] (4-1) Exercise recommendation processing The exercise type recommendation process of this embodiment will be described below. Fig. 9 is a flowchart of the exercise type recommendation process of this embodiment. Fig. 10 is a diagram showing an example of a screen displayed in the exercise type recommendation process of this embodiment.

[0090] The exercise type recommendation process starts when, for example, any of the following start conditions is met. The exercise recommendation process was called by another process. The user or the person who plans or instructs the user's exercise therapy performs an operation to call the exercise type recommendation process. The client device 10 enters a predetermined state (for example, a predetermined application is started). The appointed date and time has arrived. A certain amount of time has passed since a certain event.

[0091] As shown in FIG. 9, the client device 10 acquires sensing data (S110). Specifically, the client device 10 may enable the operation of the camera 16 to start capturing video of the user exercising (hereinafter referred to as "user video"). The client device 10 may also enable the operation of the depth sensor 17 to start measuring the distance from the depth sensor 17 to each part of the user exercising (hereinafter referred to as "user depth"). The client device 10 may enable the operation of the microphone 18 to start collecting sound (e.g., sound generated by the user's breathing or vocalization (hereinafter referred to as "user sound")).

[0092] Furthermore, the client device 10 may cause the wearable device 50 to start measuring the heart rate (hereinafter referred to as the "user's heart rate") using the heart rate sensor 56. Furthermore, the client device 10 may enable any sensor of the client device 10 or the wearable device 50 (for example, the acceleration sensor 19 or the acceleration sensor 57).

[0093] The client device 10 then acquires sensing data from each sensor while the user is performing an exercise event (an exercise event constituting the target event set). Specifically, the client device 10 acquires sensing results generated by the various sensors enabled in step S110. For example, the client device 10 may acquire user video data from the camera 16, user depth data from the depth sensor 17, user sound data from the microphone 18, user heart rate data from the wearable device 50, and user acceleration data related to the user's acceleration (hereinafter, "user acceleration") from the acceleration sensor 19 or at least one of the wearable device 50. The client device 10 may also acquire user oxygen intake data related to the user's oxygen intake from a device that performs a test related to exhaled gas (e.g., a CPX test).

[0094] The client device 10 can repeatedly acquire sensing data (S110) while performing one exercise event.

[0095] After step S110, the client device 10 generates user data (S111). Specifically, the client device 10 generates user data based on the sensing data acquired in step S110. The user data may include at least one of the following: Data acquired in step S111 (for example, user video data, user depth data, user sound data, user heart rate data, user acceleration data, or user oxygen intake data) Data obtained by processing the data acquired in step S111 Data acquired by analyzing the user video data (or the user video data and the user depth data) acquired in step S111 (for example, skeletal data, facial expression data, skin color data, breathing data, or a combination thereof, which will be described later) Information that can identify the exercise type (exercise types that make up the target exercise type set) that the user was performing in step S110 Information indicating the user's subjective assessment of the exercise intensity of the completed exercise type (hereinafter referred to as "Perceived Exertion Intensity").

[0096] The client device 10 may generate user data (S111) repeatedly while performing one exercise event, or may generate user data each time one exercise event is completed.

[0097] After step S111, the client device 10 transmits the user data (S112). Specifically, the client device 10 transmits the user data generated in step S111 to the server 30.

[0098] The client device 10 may send user data (S112) repeatedly during the execution of one exercise event, may send the data each time an exercise event is completed, or may send the data only once after all the exercise events that make up the target exercise set have been completed.

[0099] After step S112, the server 30 executes estimation of exercise intensity (S130). Specifically, the server 30 receives the user data transmitted by the client device 10 in step S112. Based on the user data acquired from the client device 10, the server 30 estimates the exercise intensity of the user when performing each exercise event constituting the target event set. The exercise intensity may be calculated as, for example, energy consumption (e.g., METs), oxygen consumption, exercise intensity based on heart rate (e.g., exercise intensity calculated using the Karvonen method), or a combination thereof. The server 30 may refer to user profile information stored in a user profile database (FIG. 7) to estimate the exercise intensity.

[0100] As a first example of estimating exercise intensity (S130), the server 30 estimates the exercise intensity when the user performs one of the exercise types that make up the target exercise type set by performing a Karvonen calculation based on the user's heart rate measured over multiple points in time while performing the exercise type and the user's age.

[0101] As a second example of estimating exercise intensity (S130), the server 30 estimates the exercise intensity when the user performs each exercise event constituting the target event set, using an estimation model described below. Specifically, the server 30 estimates the exercise intensity by applying the estimation model to input data based on user data (e.g., skeletal data, facial expression data, skin color data, respiration data, heart rate data, or a combination thereof).

[0102] As a third example of estimating exercise intensity (S130), the server 30 may calculate the exercise intensity when the user performs each exercise type that constitutes the target exercise type set based on the user's oxygen intake data when performing the exercise type.

[0103] As a fourth example of estimating exercise intensity (S130), the server 30 may conduct a talk test using user audio data by an expert, such as a physical therapist, while the user is performing each exercise type that constitutes the target exercise set, and calculate the exercise intensity when the user performs that exercise type based on the results of the talk test.

[0104] As a fifth example of estimating exercise intensity (S130), the server 30 may acquire information on the perceived intensity of exertion (e.g., the Borg index) while performing each exercise event that constitutes the target exercise set, and calculate the exercise intensity when the user performs that exercise event based on the perceived intensity of exertion.

[0105] The sixth example of exercise intensity estimation (S130) is a combination of multiple examples from the first to fifth examples.

[0106] The server 30 may estimate the exercise intensity for each section constituting an exercise type, and treat a representative value of the estimated results (e.g., mean, median, mode, maximum, minimum, first quartile, or third quartile) as the exercise intensity for that exercise type. The client device 10 or the server 30 may identify which section each sensing result is associated with, for example, by analyzing user video data (and user depth data as needed). Alternatively, in the case where a fixed time is assigned to each section, such as in gymnastics or dance, the client device 10 or the server 30 may identify which section each sensing result is associated with based on the time assignment. Furthermore, the server 30 may be a combination of two or more of the first to third examples described above.

[0107] Since the estimated exercise intensity is not stable immediately after starting exercise (for example, within 1 to 2 minutes), estimation of exercise intensity may be omitted for a predetermined period after starting exercise, or the estimated exercise intensity may be discarded. In other words, the server 30 may estimate exercise intensity only when it determines that the user is in a plateau state.

[0108] After step S130, the server 30 determines the personal index (S131). Specifically, the server 30 determines the user's individual index of exercise intensity for the target event set by performing a predetermined calculation using the exercise intensity estimated in step S130. As an example, the server 30 treats the exercise intensities estimated for the multiple event sets constituting the target event set, or the product or ratio of the exercise intensities, as elements (elements may include values (bias) not based on exercise intensity). The server 30 determines the individual index by performing a predetermined calculation, such as multiplying the element by a predetermined coefficient or adding a bias, and then finding the sum, product, or ratio of these. For example, the individual index may be a value obtained by subtracting the exercise intensity of event B from the exercise intensity of event A. The calculation formula for the individual index may be defined for each target event set, or may be defined commonly across multiple target event sets.

[0109] After step S131, the server 30 calculates the individual difference parameters (S132). Specifically, the server 30 calculates individual difference parameters that represent the user's characteristics related to exercise tolerance based on the personal indices determined in step S131 and the reference values of exercise intensity for the target event set. The server 30 creates a record based on the calculated individual difference parameters and the calculation date (or calculation date and time), and adds the record to the user's parameter log database (FIG. 8).

[0110] The reference value of exercise intensity for the target event set can be determined based on information on the exercise intensity when one or more reference individuals (e.g., individuals with standard exercise tolerance) perform each of the exercise events constituting the target event set, and on the individual index calculated for each of the individuals based on the exercise intensity. Specifically, the exercise intensity when each individual performs each exercise event is acquired (actually measured or estimated), an individual index is calculated using the same calculation formula as in step S131, and the reference value can be determined by performing statistical processing on the individual index (e.g., calculating a representative value such as averaging). The reference value may be calculated in advance for each available target event set and stored in a database. The individual referenced to calculate the reference value to be applied to a user may be determined to have at least one attribute (e.g., age group, gender, or a combination thereof) that is the same as that of the user.

[0111] As a first example, the server 30 calculates the individual difference parameter by dividing the individual index by the reference value. In this case, the individual difference parameter corresponds to a prediction result of the ratio of the exercise intensity to be higher or lower than that of a standard person when the user performs the exercise type constituting the target exercise set or another exercise type.

[0112] As a second example, the server 30 calculates the individual difference parameter by subtracting the reference value from the individual index. In this case, the individual difference parameter corresponds to a prediction result of how much the exercise intensity will be higher or lower than that of a standard person when the user performs the exercise events that make up the target event set or other exercise events.

[0113] After step S132, the server 30 selects a recommended exercise type (S133). Specifically, the server 30 refers to the user profile database (FIG. 7) and acquires the user's target intensity information (which is determined based on the results of measuring the user's exercise tolerance as described above). The server 30 selects an exercise type (hereinafter referred to as a "recommended exercise type") that is suitable for the characteristics of the user's exercise tolerance based on the acquired target intensity information, the individual difference parameters calculated in step S132, and the exercise intensity information for each exercise type stored in the exercise type database (FIG. 6). The number of recommended exercise types may be one or more.

[0114] As a first example, the server 30 corrects the target value of the user's exercise intensity using an individual difference parameter. For example, the server 30 obtains a corrected target value by dividing the target value by the individual difference parameter or by subtracting the individual difference parameter from the target value. The server 30 then selects recommended exercise types from exercise types whose exercise intensity indicated by the corresponding exercise intensity information (hereinafter referred to as "standard intensity") does not exceed the corrected target value. For example, the server 30 may include, in the recommended exercise types, exercise types whose standard intensity is the maximum within a range equal to or less than the corrected target value. Note that, when selecting recommended exercise types, the server 30 may use, instead of the corrected target value, a value obtained by adding or subtracting a margin to or from the corrected target value, or a value obtained by multiplying the corrected target value by a positive coefficient other than 1. Alternatively, when selecting recommended exercise types, the server 30 may use, instead of the standard intensity, a value obtained by adding or subtracting a margin to or from the standard intensity, or a value obtained by multiplying the standard intensity by a positive coefficient other than 1.

[0115] As a second example, the server 30 corrects the standard intensity for each exercise type using an individual difference parameter. For example, the server 30 obtains a corrected intensity by multiplying or adding the standard intensity by the individual difference parameter. The server 30 then selects recommended exercise types from exercise types whose corrected intensity does not exceed the user's target exercise intensity. For example, the server 30 may include, in the recommended exercise types, exercise types whose corrected intensity is the maximum within a range equal to or less than the target value. Note that, when selecting recommended exercise types, the server 30 may use, instead of the target value, a value obtained by adding or subtracting a margin from the target value, or a value obtained by multiplying the target value by a positive coefficient other than 1. Alternatively, when selecting recommended exercise types, the server 30 may use, instead of the corrected intensity, a value obtained by adding or subtracting a margin from the corrected intensity, or a value obtained by multiplying the corrected intensity by a positive coefficient other than 1.

[0116] After step S133, the server 30 transmits information on recommended exercise events (S134). Specifically, the server 30 transmits information about the recommended exercise selected in step S133 (hereinafter referred to as "recommended exercise information") to the client device 10. The recommended exercise information may be, for example, information that identifies the recommended exercise, or screen information for displaying the recommended exercise.

[0117] After step S134, the client device 10 executes screen display (S113). Specifically, the client device 10 receives the recommended exercise information transmitted by the server in step S134. The client device 10 displays on the display 15 a screen based on the recommended exercise information.

[0118] For example, the client device 10 displays the screen of Fig. 10 on the display 15. The screen of Fig. 10 includes objects J20 to J23.

[0119] The object J20 displays recommended exercises. The object J20 also accepts user instructions to start a recommended exercise or to play a model video of the recommended exercise. When an object J20 is selected, the client device 10 may re-execute the exercise recommendation process of this embodiment, with the exercise corresponding to the object J20 as an exercise that constitutes a new target exercise set. The server 30 may then calculate the user's individual index for the new target exercise set and calculate the user's individual difference parameters based on the individual index and a reference value corresponding to the new target exercise set. Alternatively, when an object J20 is selected, the client device 10 may play a model video of the exercise corresponding to the object J20.

[0120] The object J21 accepts a user instruction to select an exercise type other than the recommended exercise types. When the object J21 is selected, the client device 10 may, for example, display a list of exercise types other than the recommended exercise types and accept a user instruction to select an exercise type. In response to such a user instruction, the client device 10 may re-execute the exercise type recommendation process of this embodiment, with the selected exercise type as an exercise type constituting a new target exercise type set. Here, the server 30 may calculate the user's individual index for the new target exercise type set and calculate the user's individual difference parameters based on the individual index and a reference value corresponding to the new target exercise type set. Alternatively, when an exercise type is selected, the client device 10 may play a model video of the selected exercise type.

[0121] The object J22 receives a user instruction to end the exercise. When the object J22 is selected, the client device 10 ends the exercise type recommendation process of this embodiment.

[0122] After step S113, the client device 10 may end the exercise event recommendation process (FIG. 9).

[0123] (4-2) Parameter monitoring process The parameter monitoring process of this embodiment will be described below with reference to the flowchart of FIG.

[0124] The parameter monitoring process of this embodiment may be started, for example, each time the exercise type recommendation process (Figure 9) of this embodiment (particularly, the calculation of individual difference parameters (S133)) is performed, or may be repeatedly performed at a predetermined interval for each user.

[0125] As shown in FIG. 11, the server 30 acquires parameter log information (S230). Specifically, the server 30 refers to the parameter log database of the user to be processed and acquires the parameter log information.

[0126] After step S230, the server 30 executes a determination of a predetermined condition (S231). Specifically, the server 30 determines whether or not a predetermined condition is met for the parameter log information acquired in step S230. For example, the predetermined condition may include at least one of the following. The representative value of a certain number of recent individual difference parameters (one or more) exceeds the threshold (i.e., the user's exercise intensity is predicted to be excessively high compared to a typical person). The representative value of the individual difference parameter over the recent specified period exceeds the threshold (i.e., the user is predicted to exercise at an excessively high intensity compared to the average person). The rate of increase in the individual difference parameter for the most recent specified number of times exceeds the threshold (i.e., it is predicted that the user's exercise intensity for the same exercise type is rapidly increasing). The rate of increase in the individual difference parameter in the most recent specified period exceeds the threshold (i.e., the user's exercise intensity for the same exercise type is predicted to be increasing rapidly).

[0127] If it is determined in step S231 that the predetermined condition is met, the server 30 outputs an alert (S232). Specifically, the server 30 transmits the alert to the user's client device 10 or a predetermined terminal. The predetermined terminal may be a terminal of a person designated by the user (e.g., a family member), a terminal of the user's doctor, or a terminal of a person who plans or instructs the user's exercise therapy.

[0128] The alert may include, for example, at least one of the following information: -The most recent predetermined number of individual difference parameters Individual difference parameters over a recent specified period -Increase rate of individual difference parameters in the most recent specified number -Increase rate of individual difference parameters in the most recent specified period A message (e.g., text, image, audio, or a combination thereof) indicating that the user may be experiencing a heart condition A message indicating that the user is suspected of having a physical injury A message indicating that the user may be experiencing a temporary illness

[0129] Based on the alert received from the server 30, the alert output destination (the user's client device 10 or a predetermined terminal) presents information by means of an image, sound, vibration, or a combination thereof.

[0130] After step S232, the server 30 ends the parameter monitoring process of this embodiment.

[0131] If it is determined in step S231 that the predetermined condition is not met, the server 30 skips outputting the alert (S232) and ends the parameter monitoring process of this embodiment.

[0132] (5) Summary As described above, the server 30 of this embodiment acquires first data based on sensing results obtained when the user is performing a first type of exercise, and acquires a first exercise intensity for the user performing the first type of exercise based on the first data. The server 30 acquires second data based on sensing results obtained when the user is performing a second type of exercise different from the first type of exercise, and acquires a second exercise intensity for the user performing the second type of exercise based on the second data. The server 30 determines a first individual index for the user by performing a predetermined calculation on multiple exercise intensities acquired for the user, including the first exercise intensity and the second exercise intensity, and calculates a parameter representing the user's characteristics related to exercise tolerance based on the user's first individual index and a first reference value. This allows the individual difference parameter to be used as a decision-making tool for planning or instructing an exercise therapy (which may include a prescription) provided to the user, or as a signal indicating the user's physical function or physical condition. Furthermore, by referring to the exercise intensities for multiple types of exercise, a more reliable individual index can be calculated.

[0133] The server 30 may use the parameters, the user's target exercise intensity, and a database that associates each of a plurality of exercise types with a standard exercise intensity to select a recommended exercise type for the user from a plurality of exercise types and output information indicating the recommended exercise type. This allows the server 30 to recommend an exercise type not only by matching the measurement results of the user's exercise tolerance with the standard exercise intensity of each exercise type, but also by taking into consideration the characteristics of the user's exercise tolerance.

[0134] The server 30 may select, as a recommended exercise type, an exercise type associated with an exercise intensity that does not exceed a corrected exercise intensity obtained by correcting the target exercise intensity using a parameter. This makes it possible to recommend an exercise type whose exercise intensity when actually performed by the user is expected to not exceed the target exercise intensity.

[0135] The server 30 may select recommended exercise types so as to include, among the multiple exercise types, an exercise type associated with the maximum exercise intensity that does not exceed the corrected exercise intensity. This makes it possible to recommend exercise types that are estimated to result in exercise intensity that does not exceed the target exercise intensity when actually performed by the user and is close to the target exercise intensity.

[0136] The target exercise intensity may be determined based on the exercise intensity specified by an algorithm or by a person who plans or instructs the user's exercise therapy based on the results of the user's exercise tolerance measurement, thereby making it possible to recommend an exercise type that matches the exercise intensity determined to be suitable for the user from a medical perspective.

[0137] The first reference value may be a representative value of the first individual indexes of multiple people calculated using the same formula as the first individual index of the user, thereby obtaining an individual difference parameter that appropriately represents the characteristics of the user's exercise tolerance.

[0138] The multiple people may be defined to have at least one attribute that is the same as the user, thereby obtaining individual difference parameters that more appropriately represent the characteristics of the user's exercise tolerance.

[0139] The plurality of exercise types may include exercise types that can be performed without using a device that can adjust the exercise load, thereby expanding the range of exercise types available to the user.

[0140] The first exercise type may have a different standard exercise intensity from the second exercise type, which makes it easier to obtain a reasonable individual difference parameter when calculating an individual index using, for example, the difference or ratio between the exercise intensities.

[0141] The plurality of exercise intensities may be expressed using at least one of oxygen consumption and energy consumption, thereby obtaining an individual difference parameter that represents the characteristics of the user's oxygen consumption or energy consumption.

[0142] The server 30 may acquire third data based on the sensing results when the user is performing a third exercise type selected from the recommended exercise types, and may acquire a third exercise intensity when the user is performing the third exercise type based on the third data. The server 30 may determine a second individual index of the user by performing a predetermined calculation on multiple exercise intensities acquired for the user, including the third exercise intensity, and calculate parameters based on the second individual index and a second reference value. This makes it possible to calculate individual difference parameters for various combinations of exercise types.

[0143] (6) Estimation model As described above, the server 30 may estimate exercise intensity using an estimation model. In this case, the estimation model corresponds to a trained model created by supervised learning using a training data set, which will be described below, or a derived model or distilled model of the trained model. The estimation model may be constructed for each type of exercise, or may be constructed commonly across multiple types of exercise.

[0144] (6-1) Training dataset A teacher dataset that can be used in supervised learning for constructing an estimation model will be described below. Fig. 12 is a diagram showing the data structure of the teacher dataset that can be used in this embodiment.

[0145] As shown in Figure 12, the training data set includes multiple training data. The training data is used to train or evaluate a model to be learned (hereinafter referred to as a "target model"). The training data includes a sample ID, input data, and correct answer data.

[0146] The sample ID is information that identifies the training data.

[0147] The input data is data that is input to the target model during training or evaluation. The input data corresponds to examples used during training or evaluation of the target model. As an example, the input data is data regarding the subject's physical state during exercise (i.e., relatively dynamic data) and data regarding the subject's health state (i.e., relatively static data). At least a portion of the data regarding the subject's physical state is obtained by analyzing the subject's physical state with reference to the subject video data (or the subject video data and subject depth data).

[0148] The subject video data is data related to a video of a subject during exercise. The subject video data can be obtained, for example, by capturing an external image (e.g., the whole body) of a subject undergoing an exhaled gas test (e.g., a CPX test) from the front or diagonally in front (e.g., 45 degrees forward) using a camera (e.g., a camera mounted on a smartphone).

[0149] The subject depth data is data on the distance (depth) from the depth sensor to each part of the subject during exercise. The subject depth data can be acquired by operating the depth sensor when shooting the subject video.

[0150] The subject may be the same person as the user whose exercise intensity is estimated based on exercise tolerance during operation of the information processing system 1, or may be a different person. By making the subject and the user the same person, the target model may learn the user's personality and improve estimation accuracy. On the other hand, allowing the subject to be a different person from the user has the advantage of making it easier to enrich the training data set. Furthermore, the subjects may be composed of multiple people, including the user, or multiple people excluding the user.

[0151] In the example of FIG. 12, the input data includes skeletal data, facial expression data, skin color data, respiration data, heart rate data, and health condition data.

[0152] Skeletal data is data (e.g., features) related to the subject's skeleton during exercise. Skeletal data includes, for example, data related to the speed or acceleration of each part of the subject (which may include data related to changes in the parts of the muscles used by the subject or data related to the subject's perceived shaking). Skeletal data can be acquired by analyzing the subject's skeleton during exercise with reference to subject video data (or subject video data and subject depth data). For example, Vision, an SDK for iOS (registered trademark) 14, or other skeletal detection algorithms (e.g., OpenPose, PoseNet, MediaPipe Pose) can be used to analyze the skeleton. Alternatively, skeletal data for a training dataset can be acquired by, for example, having the subject exercise while wearing motion sensors on each part of their body.

[0153] The results of skeletal detection can be used for quantitative evaluation of exercise, qualitative evaluation, or a combination of these. As a first example, the results of skeletal detection can be used to count the number of reps. As a second example, the results of skeletal detection can be used to evaluate the form of an exercise or the appropriateness of the load applied by the exercise. For example, if the exercise is a squat, the results of skeletal detection can be used to evaluate whether the knees are protruding too far forward, resulting in a dangerous form, or whether the hips are lowered firmly and deeply to apply sufficient load.

[0154] Facial expression data is data (e.g., features) about a subject's facial expressions while exercising. Facial expression data can be analyzed by applying an algorithm or trained model to video data of the subject. Alternatively, facial expression data for a training dataset can be obtained by, for example, human labeling of the subject's videos.

[0155] Skin color data is data (e.g., features) about the skin color of a subject during exercise. Skin color data can be analyzed by applying an algorithm or trained model to video data of the subject. Alternatively, skin color data for a training dataset can be obtained by, for example, human labeling of the subject's video.

[0156] The respiratory data is data (e.g., feature quantities) related to the subject's breathing during exercise. The respiratory data relates to, for example, the number of breaths per unit time or the breathing pattern. The breathing pattern may include at least one of the following: Ventilation rate Ventilation volume Ventilation rate (i.e., the amount of ventilation per unit time, or the number of ventilations) Ventilation acceleration (i.e., the time derivative of the ventilation rate) Carbon dioxide emission concentration Carbon dioxide emissions (VCO2) Oxygen intake concentration Oxygen uptake (VO2) The data relating to the breathing pattern may include data that can be calculated based on a combination of the above data, such as the gas exchange ratio R (=VCO2 / VO2).

[0157] The respiration data can be obtained by, for example, analyzing the skeletal data. As an example, the following items can be analyzed from the skeletal data: Movement (expansion) of the shoulders, chest (which may include the lateral chest), abdomen, or a combination thereof Inhalation time Exhalation time - Use of accessory respiratory muscles

[0158] The respiratory data for the training dataset can be obtained, for example, from the results of a test on exhaled gases performed on a subject while exercising. Details of exhaled gas tests that can be performed on a subject while exercising will be described later. Alternatively, the respiratory data for the training dataset, such as the ventilation rate, ventilation volume, ventilation rate, or ventilation acceleration, can be obtained, for example, from the results of a respiratory function test (e.g., a pulmonary function test or a spirometry test) performed on the subject while exercising. In this case, the respiratory function test can be performed using commercially available testing equipment, not limited to medical equipment.

[0159] Heart rate data is data (e.g., feature values) related to the heart rate of a subject during exercise. Heart rate data can be obtained, for example, by analyzing subject video data or the analysis results thereof (e.g., skin color data). Alternatively, heart rate data for the training dataset may be obtained, for example, from the results of a test on exhaled gas along with respiratory data, which will be described later. Subject heart rate data for the training dataset can also be obtained by having the subject perform the above exercise while wearing a heart rate sensor or electrodes for an electrocardiogram monitor.

[0160] Health condition data is data related to the subject's health condition. Health condition data can be obtained in various ways. The subject's health condition data may be obtained before, during, or after the subject's exercise. The subject's health condition data may be obtained based on a report from the subject or their doctor, by extracting information linked to the subject in a medical information system, or via the subject's app (e.g., a healthcare app).

[0161] The health condition includes at least one of the following: ·age ·sex ·height ·body weight ·Body fat percentage Muscle mass ·Bone density History of current illness ·Past history ·Medication history Surgical history Life history (e.g., smoking history, drinking history, activities of daily living (ADL), frailty score, etc.) Family history Respiratory function test results Test results other than respiratory function tests (e.g., blood tests, urine tests, electrocardiograms (including Holter ECGs), cardiac ultrasounds, X-rays, CT scans (including cardiac morphological CT and coronary artery CT), MRI scans, nuclear medicine scans, PET scans, etc.) Data obtained during cardiac rehabilitation (including Borg index)

[0162] The correct answer data is data that corresponds to the correct answer for the corresponding input data (example question). The target model is trained (supervised learning) to output the input data in a way that is as close as possible to the correct answer data. As an example, the correct answer data represents exercise intensity.

[0163] Exercise intensity is an index for quantitatively evaluating the exercise load on a subject. Exercise intensity can be expressed numerically using at least one of the following: Energy (calorie) consumption Oxygen consumption Heart rate

[0164] The correct answer data can be obtained, for example, from the results of a test on exhaled gases conducted on a subject who is exercising. A first example of a test on exhaled gases is a test (typically a CPX test) conducted while a subject wearing an exhaled gas analyzer is performing exercise with gradually increasing load (e.g., on an ergometer). A second example of a test on exhaled gases is a test conducted while a subject wearing an exhaled gas analyzer is performing exercise of a constant or variable intensity (e.g., bodyweight exercise, gymnastics, strength training).

[0165] Alternatively, the correct data can be obtained from the results of tests other than exhaled gas tests conducted on the exercising subject. Specifically, the correct data can be obtained from the results of a cardiopulmonary exercise load prediction test based on the lactate concentration measurement in the sweat or blood of the exercising subject. A wearable lactate sensor can also be used to measure the lactate concentration of the subject.

[0166] It is also possible to construct an estimation model for each of a plurality of health condition categories based on (at least a part of) the subject's health condition. In this case, (at least a part of) the user's health condition may be referenced to select the estimation model. In this further variation, the input data for the estimation model may be data that is not based on the user's health condition, or may be data that is based on the user's health condition and user video.

[0167] (7) Other variations The storage device 11 may be connected to the client device 10 via a network NW. Each input device or output device may be integrated with the client device 10. The storage device 31 may be connected to the server 30 via the network NW. Each input device or output device may be integrated with the wearable device 50.

[0168] In the embodiment, the information processing system 1 is implemented as a client / server system. However, the information processing system 1 can also be implemented as a peer-to-peer system or a standalone computer. As an example, the client device 10 may estimate exercise intensity.

[0169] Each step of the above information processing can be performed by either the client device 10 or the server 30. As an example, the server 30, instead of the client device 10, may acquire at least a portion of the user data or the user skeletal data by analyzing the user video (or the user video and the user depth).

[0170] One or more steps of the above information processing may be performed using a trained model.

[0171] In the above example, a recommended exercise type is selected based on the standard exercise intensity of each of multiple exercise types, the user's individual difference parameters, and the results of measuring the user's exercise tolerance. However, the server 30 may determine instructions for adjusting the exercise intensity of the exercise type selected by the user based on this information (in other words, presenting an exercise type (variation) whose exercise definition information differs only partially from the exercise type selected by the user) and output the instructions to the client device 10. The instructions may include, for example, the form of the exercise type (e.g., the positional relationship or angle of body parts (which may include body parts that are not moved during exercise), or the range of motion of body parts that are moved during exercise), pace, number of reps, or the duration or number of rest periods. This allows, for example, even if the exercise intensity of the exercise type selected by the user is estimated to exceed the aforementioned target value, to prevent the user's exercise intensity from deviating from the target value by specifying a form that results in a lighter exercise intensity than the standard, a slower pace than the standard, a reduced number of reps than the standard, or an increased duration or number of rest periods. Alternatively, even if it is estimated that the exercise intensity of the exercise type selected by the user will be lower than the target value, the user can prevent the exercise intensity from deviating from the target value by specifying a form that makes the exercise intensity heavier than standard, setting a faster pace than standard, increasing the number of repetitions than standard, or reducing the length or number of rest periods.

[0172] In the above description, an example was shown in which a recommended type of exercise is selected based on the standard exercise intensity of each of a plurality of exercise types, the user's individual difference parameters, and the results of measuring the user's exercise tolerance. However, the server 30 may also select a recommended type of exercise based additionally on at least one of the user's pulse rate, respiration rate, or perceived exertion during exercise. Here, the perceived exertion can be acquired, for example, by receiving a Borg index input from the user via the client device 10 during a rest period. If at least one of the pulse rate, respiration rate, or perceived exertion exceeds an upper limit, the server 30 may select a recommended type of exercise with a lower standard exercise intensity than in other cases. On the other hand, if at least one of the pulse rate, respiration rate, or perceived exertion falls below a lower limit, the server 30 may select a recommended type of exercise with a higher standard exercise intensity than in other cases. However, if a doctor has set an upper limit for the exercise intensity of the user, the server 30 will not select an exercise type corresponding to an exercise intensity exceeding that upper limit. Furthermore, the user's pulse rate, respiration rate, or perceived exertion level during exercise may be used to adjust the exercise intensity of the next set when performing an exercise event consisting of multiple sets. For example, when at least one of the pulse rate, respiration rate, or perceived exertion level exceeds an upper limit, the server 30 may determine instructions (e.g., form, pace, number of repetitions, or rest time or number) so that the exercise intensity of the next set is lower than that of the current set, and output information about the instructions to the client device 10. On the other hand, when at least one of the pulse rate, respiration rate, or perceived exertion level falls below a lower limit, the server 30 may determine instructions so that the exercise intensity of the next set is higher than that of the current set, and output information about the instructions to the client device 10. However, if a doctor has set an upper limit for the exercise intensity for the user, instructions corresponding to an exercise intensity exceeding the upper limit will not be selected. For example, the server 30 may determine a recommended type of exercise or instructions to reduce the exercise intensity of the next set by 0.2 METs when the average heart rate is 5 or more higher than the target heart rate or the perceived intensity of exertion is 14 or higher. The server 30 may determine a recommended type of exercise or instructions to reduce the exercise intensity of the next set by 0.4 METs when the average heart rate is 10 or more higher than the target heart rate or the perceived intensity of exertion is 16 or higher. The server 30 may also determine a recommended type of exercise or instructions to increase the exercise intensity of the next set by 0.2 METs when the average heart rate is 5 or more lower than the target heart rate and the perceived intensity of exertion is less than 10.

[0173] The client device 10 may further perform the following processing, for example, when acquiring the sensing data (S110) (in other words, while the user is exercising). Specifically, the client device 10 estimates the user's movements during exercise (e.g., movements of the skeleton or other feature points over multiple points of time, or the state of the skeleton or other feature points at a single point of time) based on the sensing data. Then, if the results of the estimation of the movements do not match at least one of the form or pace defined for the type of exercise the user is performing (e.g., the range of motion of a body part is too narrow or too wide, the angle of a body part deviates from the standard, or the pace is too fast or too slow), the client device 10 provides feedback to the user. The feedback may include at least one of the following: Light, sound or audio output (e.g., audio that tells the user what they did wrong) Displaying images (e.g., images that tell the user what they did wrong) Vibration on wearable devices This allows the user's personal index to be estimated more accurately. Alternatively, if the number or frequency of occurrences of the motion estimation results not conforming to at least one of the form or pace defined for the type of exercise being performed by the user exceeds a threshold, the client device 10 may recommend that the user repeat the same type of exercise or perform a different type of exercise. In this case, the client device 10 may omit the processes subsequent to the generation of user data (S111).

[0174] Instead of recommending an exercise type, the information processing system 1 may perform a process for measuring oxygen consumption at the anaerobic threshold. This process may be automatically selected, for example, during the first three days after the user begins an exercise regimen, or may be selected by the user or the person planning or instructing the exercise regimen. In this mode, the information processing system 1 sequentially selects exercise types so that the exercise intensity increases by 0.2 METs until one of the following conditions is met: the average heart rate is 5 or more times higher than the target heart rate, or the perceived intensity of exertion is 14 or higher. The information processing system 1 treats the oxygen consumption estimated for the exercise type performed immediately before the exercise type for which the above condition is met as the user's oxygen consumption at the anaerobic threshold. The information processing system 1 may also treat the exercise intensity corresponding to the exercise type that yields the average heart rate closest to the target heart rate as the optimal exercise intensity.

[0175] The above description describes an example in which an alert is output when a user's individual difference parameters satisfy a predetermined condition. However, alerts may be output based on various triggers other than individual difference parameters. For example, the server 30 stores data on the types of exercise the user has previously performed and the user's heart rate, perceived intensity of exertion, or respiration rate while performing the exercise. The server 30 may then output an alert when the user's heart rate, perceived intensity of exertion, or respiration rate during exercise increases (worsens) by more than a threshold compared to the user's heart rate, perceived intensity of exertion, or respiration rate when performing the same exercise type in the past. Furthermore, if the user's level of distress (hereinafter referred to as "distress level") is quantified based on facial expression data or skeletal data, for example, this can be used as a trigger similar to the heart rate, perceived intensity of exertion, or respiration rate. In addition, the server 30 may use as a trigger input regarding the onset or worsening of the user's subjective symptoms of weight gain, shortness of breath, edema, fatigue, loss of appetite, or insomnia in daily life, or sensor input suggesting the onset or worsening of these symptoms.

[0176] The above description illustrates an example in which a user's individual difference parameters are calculated during the exercise type recommendation process or parameter monitoring process. However, the user's individual difference parameters may also be calculated based on data obtained during a CPX test. For example, the individual difference parameters may be calculated based on the exercise intensity (e.g., the actual oxygen intake value) when the user is rowing an ergometer at a first load (e.g., 20 W) and the exercise intensity when the user is rowing an ergometer at a second load (e.g., 40 W). When a doctor sets an upper limit on exercise intensity for a user (an example of an exercise prescription), the type of exercise presented to the doctor can be selected based on the CPX data and the individual difference parameters. For example, if 3.6 METs is recommended as the upper limit based on the CPX, a group of display areas for sample videos of exercise types corresponding to the aforementioned corrected intensity of 3.4 METs, a group of display areas for sample videos of exercise types corresponding to a corrected intensity of 3.6 METs, and a group of display areas for sample videos of exercise types corresponding to a corrected intensity of 3.8 METs may be arranged on the UI screen. Note that information about exercise events (for example, sample videos) may be arranged in units smaller than or larger than 0.2 METs.

[0177] In the above embodiment, an example was shown in which a user's individual difference parameter is calculated based on the exercise intensity when the user is performing a plurality of different exercise types. However, instead of one of the exercise types, the user's exercise intensity when at rest may be used. By defining a standard exercise intensity when at rest and acquiring the user's exercise intensity when at rest, the individual difference parameter can be calculated in the same way.

[0178] In the above description, an example has been given in which a user video is captured using the camera 16 of the client device 10. However, the user video may be captured using a camera other than the camera 16. In the above description, an example has been given in which the user depth is measured using the depth sensor 17 of the client device 10. However, the user depth may be measured using a depth sensor other than the depth sensor 17.

[0179] In the above description, an example has been given in which the wearable device 50 measures the user's heart rate. However, the heart rate can also be obtained by analyzing video data or its analysis results (e.g., skin color data) (e.g., rPPG (Remote Photoplethysmography) analysis). The heart rate analysis may be performed using a trained model constructed using machine learning technology. Alternatively, the user may exercise while wearing electrodes for an electrocardiogram monitor, allowing the electrocardiogram monitor to measure the user's heart rate. In these modified examples, the user does not need to wear the wearable device 50 to measure the heart rate.

[0180] Instead of the heart rate sensor 56 and the acceleration sensor 57, or in addition to the heart rate sensor 56 and the acceleration sensor 57, the wearable device 50 can be provided with a sensor for measuring at least one of the following items: Blood sugar levels Oxygen saturation The measurement results from each sensor may be used as appropriate to generate input data, estimate exercise intensity or ventilation index, present information based on the estimation results, or in other situations. For example, blood glucose level measurements may be referenced to evaluate exercise intensity converted into energy consumption or oxygen consumption. For another example, acceleration measurements may be used to determine a user's exercise (e.g., gymnastics) score.

[0181] The acceleration data may be used as part of the input data for the estimation model described in this embodiment or the modified example. Alternatively, the user's skeleton may be analyzed by referring to the acceleration data. The acceleration data may be acquired by the acceleration sensor 19 or the acceleration sensor 57 when capturing a user video.

[0182] Oxygen saturation data can also be used as part of the input data for the estimation model described in this embodiment or the modified example. The oxygen saturation data can be obtained, for example, by having the user wear a wearable device equipped with a sensor (e.g., an optical sensor) capable of measuring blood oxygen concentration or a pulse oximeter when capturing the user's video. The oxygen saturation data can be estimated, for example, by performing rPPG analysis on the user's video data.

[0183] The information processing system 1 of this embodiment and each modification can also be applied to a video game in which the progress of the game is controlled according to the player's physical movements. The video game may be a mini-game that can be played while the aforementioned treatment app, rehabilitation app, or fitness app is running, or may be configured as a standalone app. As an example, the information processing system 1 may estimate the user's exercise intensity during game play and determine one of the following depending on the result of the estimation (e.g., a numerical value indicating the user's exercise intensity). This can enhance the effect that the video game has on improving the user's health. The quality (e.g., difficulty) or quantity of video game challenges (e.g., stages, missions, quests) provided to users The quality (e.g., type) or quantity of video game benefits (e.g., in-game currency, items, bonuses) provided to users Game parameters related to the progression of a video game (e.g., score, damage)

[0184] The information processing system 1 of this embodiment and each modification can also be applied to a video game in which game progress is controlled according to the player's physical movements. The video game may be a mini-game that can be played while the aforementioned treatment app, rehabilitation app, or fitness app is running, or may be configured as a standalone app. As an example, the information processing system 1 estimates the user's skeletal structure based on the user's video during gameplay. The estimation of the user's skeletal structure may be performed based on at least one of the user's depth and user acceleration in addition to the user's video. Based on the result of the estimation of the user's skeletal structure, the information processing system 1 evaluates how well the user's posture during exercise (e.g., gymnastics) matches an ideal posture (model). The information processing system 1 may determine one of the following depending on the result of this evaluation (e.g., a numerical value indicating the degree to which the user's posture matches the ideal posture). This can enhance the effect of the video game on improving the user's health. The quality (e.g., difficulty) or quantity of video game challenges (e.g., stages, missions, quests) provided to users The quality (e.g., type) or quantity of video game benefits (e.g., in-game currency, items, bonuses) provided to users Game parameters related to the progression of a video game (e.g., score, damage)

[0185] In addition to or instead of microphone 18, a microphone of wearable device 50 (a microphone provided in wearable device 50 or connected to wearable device 50) may receive sound waves emitted by the user when capturing the user video and generate sound data. The sound data may constitute input data for the estimation model described in this embodiment or the modified example. The sound emitted by the user is, for example, at least one of the following: Sound waves emitted by the rotation of the user's legs (e.g., sounds coming from the pedals or the drive mechanism connected to the pedals) - Sounds that occur when the user breathes or speaks

[0186] In the above description, a CPX test is exemplified as a test related to exhaled gas. In a CPX test, a gradually increasing exercise load is applied to the test subject. However, it is not necessary to gradually increase the exercise load applied to the user when recording the user video. Specifically, real-time exercise intensity can be estimated even when the user is given a constant or constantly changeable exercise load. For example, the exercise performed by the user may be bodyweight exercise, calisthenics, or strength training.

[0187] Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to the above-described embodiments. Furthermore, the above-described embodiments can be improved or modified in various ways without departing from the spirit of the present invention. Furthermore, the above-described embodiments and modifications can be combined. [Explanation of symbols]

[0188] 1: Information processing system 10: Client device 11:Storage device 12: Processor 13: Input / output interface 14: Communication interface 15: Display 16: Camera 17: Depth sensor 18: Microphone 19: Acceleration sensor 21: Display 30: Server 31:Storage device 32: Processor 33: Input / output interface 34: Communication interface 50: Wearable devices 51:Storage device 52: Processor 53: Input / output interface 54: Communication interface 55: Display 56: Heart rate sensor 57: Acceleration sensor

Claims

1. Computer, means for acquiring first data based on a sensing result when a user is performing a first exercise event; means for acquiring a first exercise intensity when the user is performing the first exercise type based on the first data; a means for acquiring second data based on sensing while the user is performing a second exercise type different from the first exercise type or while the user is at rest; a means for acquiring a second exercise intensity when the user is performing the second exercise type or at rest based on the second data; means for determining a first personal index of the user by performing a predetermined calculation on a plurality of exercise intensities acquired for the user, including the first exercise intensity and the second exercise intensity; means for calculating a parameter representing a characteristic of the user relating to exercise tolerance based on a first personal index of the user and a first reference value; A program that functions as a

2. The computer a means for selecting a recommended exercise type for the user from the plurality of exercise types using the parameters, the user's target exercise intensity, and a database that associates a plurality of exercise types with standard exercise intensities; a means for outputting information indicating the recommended exercise type; To function as, The program according to claim 1.

3. the selecting means selects, as the recommended exercise type, an exercise type associated with an exercise intensity that does not exceed a corrected exercise intensity obtained by correcting the target exercise intensity using the parameter. The program according to claim 2.

4. the selecting means selects the recommended exercise types so as to include, from among the plurality of exercise types, an exercise type associated with a maximum exercise intensity that does not exceed the corrected exercise intensity. The program according to claim 3.

5. The target exercise intensity is determined according to an exercise intensity designated by a person who plans or instructs the exercise therapy of the user or an algorithm based on the measurement result of the exercise tolerance of the user. The program according to claim 2.

6. the first reference value is a representative value of first individual indicators of a plurality of people calculated using the same calculation formula as that of the first individual indicator of the user; The program according to claim 1.

7. The plurality of people are defined to have at least one attribute identical to that of the user. The program according to claim 6.

8. The computer means for making an estimation regarding the movement of the user when performing the first exercise type based on the first data; means for providing feedback to the user when the result of the estimation regarding the movement does not conform to at least one of a form or a pace defined for the first exercise event; The program according to claim 1,

9. The plurality of exercise events include an exercise event that can be performed without using a device that can adjust the exercise load. The program according to claim 2.

10. The first exercise type has a different standard exercise intensity from the second exercise type. The program according to claim 1.

11. The plurality of exercise intensities are expressed using at least one of oxygen consumption and energy consumption. The program according to claim 1.

12. The computer a means for acquiring third data based on a sensing result when the user is performing a third exercise selected from the recommended exercises; means for acquiring a third exercise intensity when the user is performing the third exercise type based on the third data; means for determining a second personal index of the user by performing a predetermined calculation on a plurality of exercise intensities acquired for the user, including the third exercise intensity; means for calculating the parameter based on the second personal index and a second reference value; To function as, The program according to claim 2.

13. means for acquiring first data based on a sensing result when a user is performing a first exercise event; means for acquiring a first exercise intensity when the user is performing the first exercise type based on the first data; means for acquiring second data based on sensing while the user is performing a second exercise type different from the first exercise type or while the user is at rest; a means for acquiring a second exercise intensity when the user is performing the second exercise type or at rest based on the second data; means for determining a first personal index of the user by performing a predetermined calculation on a plurality of exercise intensities acquired for the user, including the first exercise intensity and the second exercise intensity; means for calculating a parameter representing a characteristic of the user relating to exercise tolerance based on a first personal index of the user and a first reference value; An information processing device comprising:

14. The computer acquiring first data based on a sensing result when the user is performing a first exercise event; acquiring a first exercise intensity when the user is performing the first exercise event based on the first data; acquiring second data based on sensing while the user is performing a second exercise type different from the first exercise type or while the user is at rest; acquiring a second exercise intensity when the user is performing the second exercise type or at rest based on the second data; determining a first personal index of the user by performing a predetermined calculation on a plurality of exercise intensities acquired for the user, including the first exercise intensity and the second exercise intensity; calculating a parameter representing a characteristic of the user related to exercise tolerance based on a first personal index of the user and a first reference value; How to do it.

15. A system including a plurality of information processing devices, means for acquiring first data based on a sensing result when a user is performing a first exercise event; means for acquiring a first exercise intensity when the user is performing the first exercise type based on the first data; means for acquiring second data based on sensing while the user is performing a second exercise type different from the first exercise type or while the user is at rest; a means for acquiring a second exercise intensity when the user is performing the second exercise type or at rest based on the second data; means for determining a first personal index of the user by performing a predetermined calculation on a plurality of exercise intensities acquired for the user, including the first exercise intensity and the second exercise intensity; means for calculating a parameter representing a characteristic of the user relating to exercise tolerance based on a first personal index of the user and a first reference value; A system comprising:

Images