Information processing apparatus, information processing method, program, and information processing system

Abstract

To provide an information processing apparatus, an information processing method, a program, and an information processing system that can easily and more accurately evaluate the motor function of an evaluatee, propose effective rehabilitation, and improve the quality of rehabilitation. 【Solution means】 A video acquisition unit that acquires an evaluation video obtained by imaging the entire body of an evaluation subject who performs a sitting and standing motion from a sitting state, and period information of the sitting and standing motion; a video processing unit that estimates key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; and an evaluation unit that generates an evaluation result of the muscle strength and balance ability of the evaluation subject using a mathematical model from the key points and the period information of the sitting and standing motion. The sitting and standing motion includes a rising phase and a sitting phase as motion phases, and the evaluation unit generates the evaluation result for each motion phase.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, a program, and an information processing system.

Background Art

[0002] Social security costs related to nursing care in Japan have been continuously increasing, and the government is aiming to shift to a sustainable social security system by promoting self-support and prevention of exacerbation. In the future nursing care insurance system, the shift to an outcome-based reward system that emphasizes results is accelerating, and nursing care facilities are required to improve the quality of rehabilitation in order to achieve results. However, there are few nursing care facilities that have rehabilitation professionals, and in particular, about 80% of day-care nursing care facilities are in a situation where it is difficult to provide "effective rehabilitation" due to the absence of rehabilitation professionals. In proposing effective rehabilitation, evaluation for appropriately grasping problems and conditions is important, but motion analysis requires advanced professional skills of rehabilitation professionals.

[0003] Regarding such problems in the evaluation of rehabilitation, Patent Document 1 discloses a technique for objectively evaluating the center-of-gravity control motor ability of a subject and diagnosing the risk of falling in advance by calculating the speed of the reaction motion until the start of center-of-gravity movement. Further, Patent Document 2 has started a technique for photographing a video of a subject and estimating the movement and skeletal information of the subject.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the technique disclosed in Patent Document 1 has a problem that it does not consider feature amounts other than the center of gravity. Further, in the technique disclosed in Patent Document 2, in order to appropriately grasp the subject's problems and condition from the estimated movement and skeletal information of the subject, it is still necessary for a rehabilitation professional to evaluate.

[0006] By the way, in the conventional evaluation method, it is difficult to diagnose an evaluatee whose motor function has declined, such as an evaluatee who cannot maintain a standing posture, and it has not been possible to appropriately grasp the problems and condition of such an evaluatee. Therefore, it has not been possible to propose effective rehabilitation either. Further, among the evaluation methods, there is also an evaluation performed using tools (for example, furniture such as chairs and bedding), but in order to perform an accurate evaluation by such an evaluation method, it is necessary to consider the relationship between the tool and the evaluatee.

[0007] Therefore, the present disclosure has been made to solve the above problems, and an information processing apparatus, an information processing method, a program, and an information processing system that easily and more accurately evaluate the motor function of an evaluatee, propose effective rehabilitation, and improve the quality of rehabilitation are provided.

Means for Solving the Problems

[0008] In order to achieve the above object, the information processing apparatus according to the present invention includes a video acquisition unit that acquires an evaluation video obtained by imaging the entire body of an evaluation target person who performs a sitting-to-standing movement from a sitting state, and period information of the sitting-to-standing movement, a video processing unit that estimates key points including joint points and the center of gravity of the evaluation target person shown in the evaluation video, and an evaluation unit that generates an evaluation result of the muscle strength and balance ability of the evaluation target person using a mathematical model from the key points and the period information of the sitting-to-standing movement, the sitting-to-standing movement includes a standing-up phase and a sitting-down phase as movement phases, and the evaluation unit generates the evaluation result for each of the movement phases.

[0009] In order to achieve the above object, an information processing method according to the present invention is an information processing method executed by an information processing apparatus including a processor and a storage unit, the method including: causing the processor to obtain an evaluation video obtained by imaging the entire body of an evaluation subject who performs a standing-sitting motion from a seated state, and period information of the standing-sitting motion; estimating key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; and generating an evaluation result of the muscle strength and balance ability of the evaluation subject using a mathematical model from the key points and the elapsed time of the standing-sitting motion. The standing-sitting motion includes a rising phase and a sitting phase as motion phases, and the step of generating the evaluation result is a step of generating the evaluation result for each of the motion phases.

[0010] In order to achieve the above object, a program according to the present invention is a program for causing a computer including a processor and a storage unit to execute, the program causing the processor to obtain an evaluation video obtained by imaging the entire body of an evaluation subject who performs a standing-sitting motion, and period information of the standing-sitting motion; estimating key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; and generating an evaluation result of the muscle strength and balance ability of the evaluation subject using a mathematical model from the key points and the elapsed time of the standing-sitting motion. The standing-sitting motion includes a rising phase and a sitting phase as motion phases, and the step of generating the evaluation result is a step of generating the evaluation result for each of the motion phases.

[0011] In order to achieve the above object, an information processing system according to the present invention is an information processing system including an information processing apparatus and a user terminal each including a processor and a storage unit. The processor is caused to execute steps of: acquiring an evaluation video obtained by imaging the entire body of an evaluation target person performing a sitting and standing motion, and period information of the sitting and standing motion; estimating key points including joint points and a center of gravity of the evaluation target person shown in the evaluation video; and generating an evaluation result of the muscle strength and balance ability of the evaluation target person using a mathematical model from the key points and the elapsed time of the sitting and standing motion. The sitting and standing motion includes a rising phase and a sitting phase as motion phases, and the step of generating the evaluation result is a step of generating the evaluation result for each of the motion phases.

Advantages of the Invention

[0012] According to the present disclosure using the above means, it is possible to easily and more accurately evaluate the motor function of an evaluated person, propose effective rehabilitation, and improve the quality of rehabilitation.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0015] <System Configuration> The information processing system 1 in the present disclosure photographs an evaluation target person who performs a predetermined functional operation related to a motor function, evaluates the motor function of the evaluation target person based on the photographed video, and proposes an exercise menu for improving the motor function of the evaluation target person based on the evaluation result. It is an information processing system that provides a motor function evaluation service.

[0016] The evaluation target person is mainly an elderly person. The users who use the motor function evaluation service are mainly business operators, local governments, individuals, etc. who provide care and assistance services to the evaluation target person. In the following description, the user will be described as a care business operator who provides care services.

[0017] FIG. 1 is a diagram showing the overall configuration of the information processing system 1 according to an embodiment of the present disclosure. The information processing system 1 includes information processing devices such as an information processing server 10 and a user terminal 20 connected via a network NW.

[0018] Each information processing device is configured by a computer including an arithmetic device and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later.

[0019] The network NW is a network such as the Internet, a VPN (Virtual Private Network), an intranet, or short-range wireless communication. For the sake of simplicity in explanation, in FIG. 1, only one user terminal 20 is shown, but the information processing server 10 can be connected to two or more user terminals 20 via the network NW. Also, one user terminal 20 may be used by a plurality of users or evaluation subjects.

[0020] The operator of the motor function evaluation service provides the motor function evaluation service to users using the information processing system 1. Specifically, the information processing system 1 provides an application that operates on the user terminal 20 used by the user. The user uses the application to measure the motor function of the evaluation subject. In this measurement, the user images the evaluation subject performing a predetermined functional movement related to the motor function, and transmits the captured evaluation video and the information of the evaluation subject to the information processing server 10. The information processing server 10 evaluates the motor function of the evaluation subject based on the received evaluation video and the like. Also, the information processing server 10 selects an appropriate exercise menu based on the evaluation result. The evaluation result and the selected exercise menu are transmitted to the user terminal 20. The motor function includes balance ability, walking ability, functional mobility, flexibility, whole-body coordination, whole-body durability, muscle strength, etc. The balance ability includes static balance ability and dynamic balance ability. In this embodiment, the evaluation of muscle strength and balance ability is performed as the motor function.

[0021] FIG. 2 is a block diagram showing the functional configuration of the information processing server 10. The information processing server 10 (hereinafter simply referred to as the server 10) is an information processing device that provides a motor function evaluation service. The server 10 includes a storage unit 101 and a control unit 103.

[0022] <Configuration of the storage unit of the information processing server> The storage unit 101 of server 10 is a storage medium or storage device that stores information used to execute management by information processing system 1. Although shown integrated with server 10 in FIG. 2, it may be a storage device physically independent of server 10. The storage unit 101 includes an application program 1011, a video table 1012, an evaluation table 1013, a motion information table 1014, a user table 1015, an evaluation target person table 1016, a video analysis mathematical model 1021, an evaluation mathematical model 1022, and a proposal mathematical model 1023. The storage unit 101 also stores other information that does not belong to these storage units.

[0023] FIG. 3 is a diagram showing an example of information stored in the storage unit 101 of the information processing server 10. Hereinafter, each table of the storage unit 101 will be described with reference to FIG. 3.

[0024] The video table 1012 is a table that stores and manages the evaluation videos acquired by the video acquisition unit 1031. The video table 1012 includes items such as video ID, video data, video basic information, creating user, imaging target person information, converted video, and evaluated video as items of information to be stored.

[0025] The video ID is an item that stores video identification information for identifying the evaluation video. The video identification information is an item for which a unique value is set for each evaluation video. The video ID is generated for each acquired evaluation video. The video data is an item that stores the file body of the evaluation video. The file body may be stored in another server, and its path may be stored. The video basic information is an item that stores video basic information such as the shooting date and time, file format, compression method, capacity, resolution, frame rate, bit rate, shooting time, etc. of the evaluation video. The creating user is an item that stores the user ID of the user who shoots the evaluation video. Thus, the evaluation video can be associated with the user. The imaging subject information is an item that stores the imaging subject information, which is the related information between the subject to be evaluated, who is the subject of the evaluation video, and the evaluation video. The imaging subject information includes the functional operation type, the total operation period Ta, and the evaluation subject ID, which will be described later. By including the evaluation subject ID, the evaluation video and the evaluation subject can be linked. The functional operations included in the functional operation type are, for example, Timed UP&Go Test, Berg Balance Scale, sit-to-stand test, single-leg or double-leg stand-up test. Each functional operation is related to a specific motor function. Specifically, the Timed UP&Go Test is related to muscle strength, balance ability, walking ability, and functional mobility ability. The Berg Balance Scale is related to balance ability, muscle strength, endurance, flexibility, and interval. The sit-to-stand test is related to lower limb muscle strength and balance ability. The single-leg or double-leg stand-up test is related to lower limb muscle strength and mobility ability. That is, all of the Timed UP&Go Test, Berg Balance Scale, sit-to-stand test, and single-leg or double-leg stand-up test are related to balance ability and muscle strength, and can be used alone or in combination as tests for judging the risk of falls in the elderly. The total operation period Ta is period information indicating numerically the period (time) from the start time to the end time of such a functional operation. The converted video is an item that stores the converted video generated by performing preprocessing on the evaluation video acquired by the video acquisition unit 1031. The evaluated video is an item that stores and manages the evaluated video generated by the inference process of the evaluation unit 1034. The evaluated video includes the report video, which will be described later.

[0026] The evaluation table 1013 is a table that stores and manages the evaluation information related to the analysis and evaluation of the motor function of the evaluation subject. The evaluation table 1013 includes, as items of the information to be stored, the items of evaluation information ID, analysis target video, joint point information, environmental information, phase period information, evaluation information, and evaluation report information.

[0027] The evaluation information ID is an item that stores evaluation identification information for identifying evaluation information. The evaluation identification information is an item for which a unique value is set for each piece of evaluation information. The evaluation information is generated for each measurement of the motor function. The video to be analyzed is an item that stores the video ID of the evaluation video that is the target of analysis. Thereby, the evaluation information and the evaluation video are linked. The joint point information is a table that associates and manages, for each frame of the converted video, a set of keypoints and coordinates generated by the inference process of the evaluation unit 1034 described later, linked to the frame. The keypoints include joint points and the center of gravity. The correspondence between the human body and the inferred keypoints will be described later. The environmental information is a table that stores and manages information about the furniture 30 used by the evaluation target person for each measurement, acquired by the environmental information acquisition unit 1033 described later. The information about the furniture 30 includes the type of furniture, the dimensional information of the furniture, the presence or absence of a backrest, and the presence or absence of armrests. Examples of the type of furniture are a chair and a bed, and examples of the dimensional information of the furniture are the height of the seat surface and the length of the depth of the seat surface. The phase period information is an item that stores the period information of the converted video to be analyzed. The period information of the converted video includes the delimiter position and the nth phase period Tn described later. In FIG. 3, as the nth set phase period Tn when the functional operation type is "sitting and standing test", the first set phase period T1 to the fifth set phase period T5 are shown. The evaluation information is an item that stores the evaluation information that is the result of analyzing the period information and the converted video. The evaluation information includes the overall operation period determination result, the phase period determination result, the phase evaluation result, the phase part-by-part evaluation result, the compensatory movement evaluation result, and the comprehensive evaluation, which will be described later. The evaluation report information is an item that stores the evaluation report information as information related to the evaluation report described later. The evaluation report information includes the report video described later, the feedback data described later, and the proposed exercise menu information described later.

[0028] The exercise information table 1014 is an item that stores information regarding an exercise menu for proposal according to the evaluation result of the exercise function of the person to be evaluated. The exercise information table 1014 includes items such as exercise information ID, exercise basic information, and exercise content as items of information to be stored. The exercise menu is classified into two types: balance exercise and movement practice, and exercise for improving muscle strength and flexibility. The exercise for improving muscle strength and flexibility includes exercises for the trunk, hip joint, thigh, ankle, and foot. Examples of the exercise menu will be described later.

[0029] The exercise information ID is an item that stores exercise identification information for identifying the exercise menu. The exercise identification information is an item for which a unique value is set for each exercise menu. The exercise basic information is an item that stores basic information regarding the exercise menu. Specifically, it includes the name of the exercise, target part, difficulty level, number of repetitions or sets to be performed, precautions, other remarks, and the like. The exercise content is an item that stores content for explaining the content of the exercise composed of an image, explanatory text, video, explanatory audio, or a combination thereof regarding the exercise menu.

[0030] The user table 1015 is a table that stores and manages information of users (user members) who use the exercise function evaluation service. The user table 1015 includes items such as user ID and user basic information as items of information to be stored. By registering for use of the service, the information of the user is stored in a new record of the user table 1015. Thereby, the user can use the exercise function evaluation service according to the present disclosure.

[0031] The user ID is an item that stores user identification information for identifying the user. The user identification information is an item for which a unique value is set for each user. The user basic information is an item that stores basic information such as the name, address, contact information, password, etc. of the user.

[0032] The evaluation target person table 1016 is a table that stores and manages information of evaluation target persons who are the targets of evaluating motor functions using a motor function evaluation service. The evaluation target person table 1016 includes, as items of information to be stored, an evaluation target person ID, basic information of the evaluation target person, a management user, and an evaluation history. The information of the evaluation target person is mainly registered by the user of the motor function evaluation service. In this embodiment, a care provider registers the information of the evaluation target person or links the information of the evaluation target person registered in another system or service used by the care provider.

[0033] The evaluation target person ID is an item that stores evaluation target person identification information for identifying the evaluation target person. The evaluation target person identification information is an item for which a unique value is set for each evaluation target person. The basic information of the evaluation target person is an item that stores the name, date of birth, age, gender, height, weight, care level, health status including medical history, physical and mental functions, ADL / IADL, environmental and personal factors (usage status of welfare appliances, home environment), etc. of the evaluation target person. The management user is an item that stores the user ID of the care provider to which the evaluation target person belongs. Thereby, the information of the evaluation target person and the information of the user can be linked. The evaluation history is an item in which the present information processing system 1 stores the evaluation information ID as the evaluation history. Thereby, the evaluation result of the motor function and the evaluation target person can be linked.

[0034] Returning to FIG. 2, each mathematical model of the storage unit 101 will be described.

[0035] The video analysis mathematical model 1021 is a mathematical model that uses a video as input data to estimate the joint points, center of gravity, skeleton, posture, etc. of the human body or animal shown in the video. Details of the video processing using the video analysis mathematical model 1021 will be described later.

[0036] The video analysis mathematical model 1021 of this embodiment is a deep learning model based on a deep neural network in deep learning. The video analysis mathematical model 1021 can be realized using a pre-trained general-purpose skeleton estimation model. As the general-purpose skeleton estimation model, for example, Openpose, PoseNet, AlphaPose, HRNet, MMPose, BlazePose, etc. can be used. The video analysis mathematical model 1021 does not necessarily have to be a deep learning model, and any machine learning or artificial intelligence model may be used. The video analysis mathematical model 1021 may be generated for each functional operation, gender, age, degree of care required, and health condition.

[0037] The evaluation mathematical model 1022 is a mathematical model that takes the key points estimated for the converted video as input data and outputs whether the functional operation performed by the person to be evaluated shown in the converted video is good or bad. The details of the inference process using the evaluation mathematical model 1022 will be described later. The evaluation mathematical model 1022 is, for example, one kind or a combination of machine learning, artificial intelligence, deep learning models, rule-based, etc. The evaluation mathematical model 1022 does not have to be a single mathematical model, and may be realized by combining or switching a plurality of independent types of mathematical models. The evaluation mathematical model 1022 may be generated for each motor function or functional operation to be evaluated, gender, age, degree of care required, and health condition.

[0038] The proposed mathematical model 1023 is a mathematical model that takes the period information and the evaluation result as input data and selects and outputs one or more exercise menus. The proposed mathematical model 1023 is, for example, one kind or a combination of machine learning, artificial intelligence, deep learning models, rule-based, etc. The proposed mathematical model 1023 does not have to be a single mathematical model, and may be realized by combining or switching a plurality of independent types of mathematical models. The proposed mathematical model 1023 may be generated for each motor function or functional operation to be evaluated, gender, age, degree of care required, and health condition.

[0039] <Configuration of the control unit of the information processing server> The control unit 103 of the server 10 includes, as functional units, a video acquisition unit 1031, a video processing unit 1032, an environmental information acquisition unit 1033, an evaluation unit 1034, an exercise proposal unit 1035, an evaluation result output unit 1036, and a mathematical model management unit 1037. The control unit 103 is a computer equipped with a CPU. The control unit 103 realizes each functional unit by executing the application program 1011 stored in the storage unit 101. The control unit 103 executes the exercise function evaluation process described later by coordinating each functional unit. Although not shown, the control unit 103 also has a function of controlling communication with external devices connected to the server 10 via a network NW or the like.

[0040] The video acquisition unit 1031 acquires, from the user terminal 20 connected to the server 10 via the network NW, an evaluation video that captures an evaluation target person performing a predetermined functional operation, and period information of the predetermined functional operation.

[0041] The video processing unit 1032 preprocesses the evaluation video and stores the preprocessed video in the storage unit 101 as a converted video. Further, the video processing unit 1032 estimates key points including joint points and the center of gravity for the evaluation target person shown in the converted video.

[0042] The environmental information acquisition unit 1033 acquires, from the user terminal 20 connected to the server 10 via the network NW, environmental information including information about the furniture 30 used by the evaluation target person to perform a predetermined functional operation. For example, the user inputs information about the furniture (e.g., the type of furniture and the dimensional information of the furniture) to the user terminal 20. Note that the acquisition of the environmental information may also be performed by indirectly measuring, using a mathematical model from the converted video, based on the age, gender, height, etc. of the basic information of the evaluation target person of the evaluation target person, and acquiring the measurement result as environmental information.

[0043] The evaluation unit 1034 performs an inference process of inferring the evaluation results of a predetermined functional operation of the person to be evaluated and the motor function related to the predetermined functional operation using a mathematical model from the key points, the period information of the predetermined functional operation, and the environmental information, and stores the inference results in the storage unit 101 as evaluation information. The details of the inference process will be described later.

[0044] Based on the period information of the predetermined functional operation and the evaluation information, the exercise proposal unit 1035 selects an exercise menu suitable for improving, maintaining, and enhancing the motor function of the person to be evaluated.

[0045] Based on the evaluation information and the selected exercise menu, the evaluation result output unit 1036 generates evaluation report information and outputs it to the user terminal 20.

[0046] The mathematical model management unit 1037 generates and updates each mathematical model in the storage unit 101. Note that the generation of the mathematical model also includes the meaning of learning.

[0047] FIG. 4 is a block diagram showing the functional configuration of the user terminal 20. The user terminal 20 includes a storage unit 201, a control unit 203, an input device 205, and an output device 207 as functional units. The user terminal 20 is an information processing device having an imaging function that is operated by a user who uses the service. The user terminal 20 may be, for example, a mobile terminal such as a smartphone or a tablet, or a stationary PC (Personal Computer) or a laptop PC. It may also be a wearable terminal such as an HMD (Head Mount Display) or a wristwatch-type terminal.

[0048] <Configuration of the storage unit of the user terminal> The storage unit 201 of the user terminal 20 is a storage medium or a storage device that can store information used for providing the motor function evaluation service by the information processing system 1. Although it is shown integrated with the user terminal 20 in FIG. 4, it may be a storage device physically independent of the user terminal 20. The storage unit 201 stores an application program 2011 and a user ID 2012. The storage unit 201 also stores other information that does not belong to these storage units.

[0049] The application program 2011 may be pre-stored in the storage unit 201, or may be configured to be downloaded from a web server or the like operated by a service provider via a communication IF. The application program 2011 includes applications such as a web browser application. The application program 2011 includes an interpreter-type programming language such as JavaScript (registered trademark) that is executed on a web browser application stored in the user terminal 20.

[0050] The user ID 2012 stores the user ID of the user who uses the user terminal 20 as identification information for the user to use the motor function evaluation service. The user transmits the user ID from the user terminal 20 to the server 10. The server 10 identifies the user by querying the received user ID in the user table 1015, and provides the service according to the present disclosure to the user. Note that the user ID 2012 may include information such as a session ID temporarily assigned from the server 10 for identifying the user who uses the user terminal 20.

[0051] <Configuration of the control unit of the user terminal> The control unit 203 of the user terminal 20 includes an input control unit 2031 and an output control unit 2032. The control unit 203 is a computer including a CPU. The control unit 203 realizes each functional unit by executing the application program 2011 stored in the storage unit 201. Although not shown, the control unit 203 also has a function of controlling communication with an external device connected to the user terminal 20 via a network NW or the like.

[0052] The input control unit 2031 controls the input device 205. The output control unit 2032 controls the output device 207.

[0053] The input device 205 of the user terminal 20 includes a camera 2051, a microphone 2052, a position information sensor 2053, a motion sensor 2054, and a touch device 2055. The input device 205 is a variety of input devices that enable the input and selection of information by the user's operation. The input and selection of information are, for example, input by entering numerical values, characters, or symbols, selection of items, input for determining processing, and the like.

[0054] The camera 2051 is, for example, a visible light camera and is an input device having a function of detecting light reflected from a subject and generating image (still image or moving image) information. The camera 2051 may be built into the user terminal 20 or may be an independent camera connected to the user terminal 20.

[0055] The microphone 2052 is an input device having a function of collecting sound generated in the vicinity of the installation location and converting it into an electrical signal to generate sound information. For example, it may be used when receiving an instruction input from the user by voice.

[0056] The position information sensor 2053 is an input device having a function of generating the position information of the user terminal 20.

[0057] The motion sensor 2054 has a function of generating the posture information of the user terminal 20 by detecting and tracking the movement of the user terminal 20.

[0058] The touch device 2055 is an input device having a function of generating operation information of an operation performed on the screen by the user of the user terminal 20 using a finger, a stylus pen, or the like.

[0059] The output device 207 of the user terminal 20 includes a display 2071 and a speaker 2072. The output device 207 is an output device capable of displaying, reproducing, and notifying information and the like to the user.

[0060] The display 2071 is a display output device that displays characters, graphics, images, videos, etc. The display 2071 may be a display output device independent of the user terminal 20, or may be a display output device such as a liquid crystal display or an organic EL display in a smartphone or tablet.

[0061] The speaker 2072 is an audio output device that outputs sound to the surroundings by converting an electrical signal into sound. The speaker 2072 may be built into the user terminal 20, or may be an independent audio output device connected to the user terminal 20.

[0062] <Measurement procedure> FIG. 5 is a diagram showing an example of measuring a motor function using the information processing system according to an embodiment of the present disclosure. FIG. 6 is a diagram showing a measurement procedure of the motor function and key points estimated at the time of measurement. With reference to FIGS. 5 and 6, the measurement procedure of the motor function, that is, the specific method of shooting the evaluation video, will be described below.

[0063] As shown in FIG. 5, the user fixes the shooting position and shooting posture of the user terminal 20 at a predetermined position and posture using a tripod or the like. The user selects (inputs) and transmits, as information about the person to be evaluated, the registered identification information (such as name) of the person to be evaluated and the type of functional operation to be evaluated on the display 2071 of the user terminal 20. Then, the user guides the person to be evaluated to a position at a predetermined distance from the user terminal 20 (hereinafter referred to as the operation position). At this time, the user may use the guide displayed during the imaging of the evaluation video. When the type of functional operation is the "sitting and standing test", the user guides the person to be evaluated to the furniture 30 used for evaluation and seats the person to be evaluated on the furniture 30. At this time, on the display 2071 of the user terminal 20, the position where the whole body of the person to be evaluated fits and at least a part of the furniture 30 used for evaluation fits is the operation position, that is, the position where the person to be evaluated is made to perform an operation. Also, when the type of functional operation is the "sitting and standing test", the person to be evaluated is photographed from the right or left side (see FIG. 5).

[0064] When the subject to be evaluated enters the standby state at the operating position, the user starts imaging on the user terminal 20 and instructs the subject to be evaluated to start the functional operation of the type selected on the display 2071. The subject to be evaluated starts the functional operation at their own timing. The user measures the period from the start to the end of the functional operation of the subject to be evaluated with a stopwatch or the like at hand. When the functional operation type is the "sitting and standing test", the operation of standing up from the sitting state and the operation of sitting down from the standing state are regarded as one time, and imaging of an evaluation video in which this sitting and standing operation is repeated a plurality of predetermined times and measurement of the total operation period Ta are performed.

[0065] When a predetermined time has elapsed since the start of video imaging, the imaging automatically stops. If the state of the functional operation of the subject to be evaluated meets the predetermined end condition before the predetermined time has elapsed, the user stops the imaging on the user terminal 20 and transmits the evaluation video to the server 10. The user inputs and transmits the measurement result as the total operation period Ta on the display screen of the user terminal 20. Note that, at this timing, the functional operation type and the identification information of the subject to be evaluated may be selected and transmitted together.

[0066] In this embodiment, when the functional operation type is the "sitting and standing test", a test in which the sitting and standing operation is repeated 5 times is performed. And the predetermined end condition of this embodiment is when the total measurement time exceeds a predetermined time (for example, 60 seconds). Note that the predetermined end condition of this embodiment is not limited to the case where it exceeds the predetermined time, and the predetermined end condition may be changed by the subject to be evaluated or the like. Also, the predetermined end condition differs depending on the functional operation type, and further, even when the functional operation type is the "sitting and standing test", the predetermined end condition also differs depending on the evaluation conditions such as the number of times the sitting and standing operation is repeated or the number of sitting and standing operations within a predetermined time (for example, how many sitting and standing operations can be performed in 30 seconds (or 60 seconds)).

[0067] <Motor function evaluation process> FIG. 7 is a flowchart showing the operation of the motor function evaluation process. The motor function evaluation process mainly involves obtaining an evaluation video that captures an evaluation subject performing predetermined functional movements related to motor function, evaluating the motor function of the evaluation subject based on the obtained evaluation video, selecting an appropriate exercise menu based on the evaluation results, and providing the evaluation results and the exercise menu. This is a series of processes. In the following description, it is assumed that "sitting-to-standing test" is selected as the predetermined functional movement in order to evaluate balance ability and muscle strength as motor functions. The motor function evaluation process includes a video acquisition step, an imaging subject information acquisition step, a video preprocessing step, an environmental information acquisition step, an inference processing step, a report video generation step, a feedback data generation step, a movement determination step, and an evaluation result output step. The operation of the motor function evaluation process will be described below according to the same flowchart.

[0068] In step S101, the video acquisition unit 1031 of the server 10 acquires an evaluation video from the user terminal 20 (video acquisition step). Specifically, the video acquisition unit 1031 acquires an evaluation video of an evaluation subject performing a predetermined functional movement from the user terminal 20 and stores it in the video table 1012 of the storage unit 101.

[0069] In step S102, the video acquisition unit 1031 of the server 10 acquires imaging subject information from the user terminal 20 (imaging subject information acquisition step). Specifically, the video acquisition unit 1031 acquires, from the user terminal 20, as imaging subject information, identification information of the evaluation subject (evaluation subject ID or information associated with the evaluation subject ID) for identifying the evaluation subject, the type of functional movement input by the user, and the total movement period Ta of that functional movement, and stores them in the video table 1012 of the storage unit 101.

[0070] In step S103, the video processing unit 1032 of the server 10 performs preprocessing on the evaluation video acquired in step S101 (video preprocessing step). Specifically, the video processing unit 1032 performs video conversion to a predetermined frame rate and a predetermined format as preprocessing, and stores it in the video table 1012 of the storage unit 101 as the converted video.

[0071] In step S104, the environment information acquisition unit 1033 of the server 10 acquires environment information from the user terminal 20 (environment information acquisition step). Specifically, the environment information acquisition unit 1033 acquires information regarding furniture used for evaluation (such as the type of furniture and the dimensional information of the seat surface) as environment information from the user terminal 20, and stores it in the environment information item of the evaluation table 1013 in the storage unit 101.

[0072] In step S105, the evaluation unit 1034 of the server 10 executes an inference process based on the converted video generated in step S103 and the environment information acquired in step S104, and generates an evaluation result (inference process step). Details of the inference process will be described later.

[0073] In step S106, the video processing unit 1032 of the server 10 executes generation of a video for report based on the converted video generated in step S103 and the evaluation result in step S105, and stores it in the evaluation report information item of the evaluation table 1013 in the storage unit 101 (video generation step for report). The video for report may be either a video or a still image. Two types of videos for report are generated: one for the user and one for the person being evaluated. The video for report for the user is mainly information for sharing with rehabilitation professionals, and is mainly an extracted partial video or frame image related to the evaluation result, with key points superimposed. In this way, by adding key points to the video for report for the user, it becomes possible to request detailed motion analysis from rehabilitation professionals (physical therapists, occupational therapists). The video for report for the person being evaluated is mainly information for sharing with the person being evaluated and their family / caregivers, and is displayed together with key points in the evaluation report described later. This promotes understanding of the posture and evaluation result of the person being evaluated, etc. at the time of measurement (video imaging) when the evaluation report is provided, and can improve the quality of rehabilitation proposals and the motivation of the person being evaluated.

[0074] In step S107, the evaluation unit 1034 of the server 10 analyzes the motor function of the person to be evaluated based on the evaluation result (evaluation information) in step S105 and the past evaluation result (evaluation information) stored in the storage unit 101, generates feedback data, and stores it in the item of the evaluation report information in the evaluation table 1013 of the storage unit 101 (feedback data creation process). For generating the feedback data, for example, a rule-based method, a general-purpose text generation AI model, or the like can be used. The feedback data includes information corresponding to each item displayed in the evaluation report. An example of the evaluation report when the function operation type is "sitting and standing test" will be described later.

[0075] In step S108, the exercise proposal unit 1035 of the server 10 executes an exercise determination based on the evaluation result in step S105 (exercise determination process). Specifically, the exercise proposal unit 1035 applies the evaluation result in step S105 as input data to the proposed mathematical model 1023, acquires (selects) one or more exercise menus as proposed exercise menu information as output data, and stores it in the item of the evaluation report information in the evaluation table 1013 of the storage unit 101. The output data is preferably at least one selected from the categories of balance exercises and movement practice. Note that past evaluation results may be added as input data.

[0076] In step S109, the evaluation result output unit 1036 of the server 10 outputs an evaluation report based on the evaluation report information stored in the evaluation table 1013 of the storage unit 101 (evaluation result output process).

[0077] <Inference processing> The inference processing is mainly a process for generating an evaluation result for evaluating the motor function of the person to be evaluated by analyzing the video. The inference processing is executed for each measurement. FIG. 8 is a flowchart showing the operation of the inference processing. The inference process is a series of processes consisting of a joint point estimation process, an action phase inference process, a full action period determination process, a phase period determination process, a phase evaluation process, a phase part-by-part evaluation process, a compensation evaluation process, and an evaluation result integration process. In the following description, it is assumed that "sitting and standing test" is selected as a predetermined functional operation. Hereinafter, the operation of the inference process will be described according to the same flowchart.

[0078] In step S201, the video processing unit 1032 of the server 10 executes a joint point estimation process for each frame of the converted video (joint point estimation process). When two or more videos are captured in one measurement, the joint point estimation process is executed for each video. Specifically, the video processing unit 1032 applies each frame image of the converted video as input data to the video analysis mathematical model 1021, obtains the set of keypoints estimated for each frame and their coordinates (hereinafter simply referred to as keypoint coordinates, joint point coordinates, and center of gravity coordinates) as output data, and stores them in the joint point information item of the evaluation table 1013 in the storage unit 101 of the server 10 in association with the identification information of each frame. Note that the center of gravity may be calculated separately from the coordinates of the joint points.

[0079] FIG. 9 is a diagram showing an example of the correspondence between the human body (evaluation target person) shown in the video and the estimated keypoints (joint points and center of gravity). The keypoints estimated by the inference process will be described with reference to FIG. 9. In the present embodiment, the coordinates of each keypoint shown in FIG. 9 can be estimated for each frame by the video analysis mathematical model 1021. Note that the numbers displayed near each keypoint in FIG. 9 are used only to indicate the correspondence with the numbers in the keypoint list of the same figure, and are not related to the reference numbers in other drawings. In Fig. 9, 33 points are shown as black circles as the estimated key points. The human body is large and is divided into the head and neck region (the dashed line region in Fig. 9), the torso (not shown), and the limbs (not shown). The limbs are divided into the upper limbs and the lower limbs (not shown). The key points belonging to the head and neck region are the nose, the inner end of the left eye, the left eye, the outer end of the left eye, the inner end of the right eye, the right eye, the outer end of the right eye, the left ear, the right ear, the left end of the mouth, and the right end of the mouth. The key points belonging to the torso region are the left shoulder, the right shoulder, the left waist, and the right waist. The key points belonging to the upper limb region are the left elbow, the right elbow, the left wrist, the right wrist, the left little finger, the right little finger, the left index finger, the right index finger, the left thumb, and the right thumb. The key points belonging to the lower limb region are the left knee, the right knee, the left ankle, the right ankle, the left heel, the right heel, the left index toe, and the right index toe.

[0080] In step S202, the video processing unit 1032 of the server 10 divides the converted video for each motion phase based on the time change of the key point coordinates acquired in step S201 (motion phase inference process). When shooting two or more videos in one measurement, the motion phase inference process is executed for each converted video. A functional operation includes a plurality of motion phases from the start to the end of the functional operation and transitions between the motion phases as time elapses. The number of included motion phases, the coupling relationship and transition direction between the motion phases, differ depending on the type of the functional operation. When the functional operation type is the "sitting and standing test", as shown in Fig. 6, there are two motion phases, the first phase and the second phase, and the motion (set motion) that transitions from the first phase to the second phase is counted as one set motion, and this set motion is repeated a plurality of times. In this embodiment, one video (converted video) is one in which the set motion is repeated 5 times. Note that the number of times the set motion is repeated in one video (converted video) is not limited to 5 times. Also, even if the set motion cannot be achieved 5 times within a predetermined time, if a predetermined end condition is satisfied, it is regarded as one video (converted video). Note that, as described above, the number of times and the end condition of the set motion of the "sitting and standing test" are not limited to this.

[0081] Specifically, the video processing unit 1032 applies the time transition of predetermined one or more keypoint coordinates in all frames of the converted video as input data to the video analysis mathematical model 1021, and acquires the segmentation positions (time or frame number) for each estimated motion phase as output data. In this process, the number of predetermined one or more keypoints is preferably 3 or more and 8 or less. When the functional operation type is "sitting - standing test", it preferably includes 2 or more keypoints in the upper limb region and 2 or more keypoints in the lower limb region. Further, the video processing unit 1032 calculates the phase period of each motion phase based on the segmentation position. Furthermore, the video processing unit 1032 stores the estimated segmentation position and the calculated phase period in the phase period information item of the evaluation table 1013 in the storage unit 101. The phase period is the elapsed period (time) from the start to the end of the motion phase. FIG. 10 shows each phase period in the case where the functional operation type is "sitting - standing test". As shown in FIG. 10, let the elapsed time of the first set of operations be the first set phase period T1, the elapsed time of the second set of operations be the second set phase period T2, the elapsed time of the third set of operations be the third set phase period T3, the elapsed time of the fourth set of operations be the fourth set phase period T4, and the elapsed time of the fifth set of operations be the fifth set phase period T5. For simplicity, in the following description, let n be a natural number. The phase period corresponding to the nth phase is denoted as the nth phase period Tn, and the video of the period corresponding to the nth phase is denoted as the nth phase video. When the functional operation type is "sitting - standing test", the phase period corresponding to the nth set of operations is denoted as the nth set phase period Tn, and the video of the period corresponding to the nth video phase is denoted as the nth set phase video.

[0082] When the functional operation type is "sitting and standing test" and the video includes 5 sets of operations, in step S202, the video processing unit 1032 obtains the delimiter position between the first phase and the second phase of the set operation, the first set phase period T1, the second set phase period T2, the third set phase period T3, the fourth set phase period T4, and the fifth set phase period T5, and stores them in the item of the phase period information in the evaluation table 1013 of the storage unit 101. The phase period information to be obtained and stored is determined according to the number of repetitions of the set operation described above. In the following specific example of steps, the case where the set operation is repeated 5 times will be described.

[0083] In step S203, the evaluation unit 1034 of the server 10 determines the total operation period Ta obtained from the user terminal 20 in step S101 of the motor function evaluation process by a five-level scoring method (total operation period determination step). When shooting two or more evaluation videos in one measurement, the total operation period determination step is executed for the total operation period Ta obtained for each evaluation video. Specifically, the evaluation unit 1034 determines the total operation period Ta in five levels based on a preset reference value (reference range). For example, when the reference value (reference range) is Taa, it is "3 points", when it is slightly better than the reference value (reference range) Taa, it is "4 points", when it is better than the reference value (reference range) Taa, it is "5 points", and conversely, when it is slightly worse than the reference value (reference range) Taa, it is "2 points", and when it is worse than the reference value (reference range) Taa, it is determined as "1 point". The evaluation unit 1034 stores the determination result in the item of the evaluation information in the evaluation table 1013 of the storage unit 101 as the total operation period determination result. Note that the total operation period determination is not limited to being determined by a five-level scoring method, and may be, for example, only a two-level good or bad determination, or a three-level excellent, good, or bad evaluation.

[0084] In step S204, the evaluation unit 1034 of the server 10 determines each nth phase period Tn calculated in step S202 (phase period determination step). When shooting two or more evaluation videos in one measurement, the phase period determination step for each phase period calculated for each evaluation video is executed. Specifically, the evaluation unit 1034 determines the nth phase period Tn in a five - level scoring system based on a preset reference value (reference range). Then, the evaluation unit 1034 stores the determination result in the item of the phase period determination result in the evaluation table 1013 of the storage unit 101.

[0085] When the functional operation type is "sitting - standing test", the evaluation unit 1034 determines the first set phase period T1 in five levels based on a preset reference value (reference range). For example, when the reference value (reference range) is T1a, it is judged as "3 points"; when it is slightly better than the reference value (reference range) T1a, it is judged as "4 points"; when it is better than the reference value (reference range) T1a, it is judged as "5 points". Conversely, when it is slightly worse than the reference value (reference range) T1a, it is judged as "2 points"; when it is worse than the reference value (reference range) T1a, it is judged as "1 point". The second set phase period T2 to the fifth set phase period T5 are judged in the same way. Also, for each operation set, the determination is made for each of the first phase and the second phase. Note that the determination may be made for each of the first phase and the second phase only for the nth operation set. For example, the determination may be made for each of the first phase and the second phase for the first operation set or the last operation set. In this case, the evaluation unit 1034 stores the determination results of the first phase and the second phase respectively in the item of the evaluation information in the evaluation table 1013 of the storage unit 101, for example, as the determination result of the first phase of the nth time and the determination result of the second phase of the nth time. Note that the phase period operation determination is not limited to the five - level scoring system, and may be, for example, an evaluation with only two levels of good or bad, or a multi - level evaluation with three or more levels.

[0086] In step S205, the evaluation unit 1034 of the server 10 infers the quality of the operation of the person to be evaluated in each operation phase based on the keypoint coordinates acquired in step S201 for each operation phase (phase evaluation process). When shooting two or more videos in one measurement, the phase evaluation process is executed for each video. Specifically, the evaluation unit 1034 calculates, for a predetermined part, a physical quantity including coordinate positions, a statistical value of the physical quantity, and a calculation result of the physical quantity and / or the statistical value as feature quantities, using the keypoint coordinates of each frame of the nth-phase video. The evaluation unit 1034 applies the calculated feature quantities as input data to the evaluation mathematical model 1022, and acquires the goodness or badness of the inferred action as output data. Then, the evaluation unit 1034 associates the goodness or badness of the inferred action with the nth action phase and stores it in the item of the phase evaluation result in the evaluation table 1013 of the storage unit 101. When the functional action type is "sitting and standing test", it may also include evaluation results based on the variation of each evaluation of multiple sets of actions.

[0087] The predetermined part in this process is selected from the above-mentioned 33 keypoints and six regions (head and neck, trunk, left upper limb, right upper limb, left lower limb, right lower limb), and the number of selected parts is preferably in the range of 3 to 8. The predetermined part to be selected is set and registered in the server 10 in advance for each action phase. The physical quantities of the predetermined part include position, distance, and angle. The angle may be a keypoint angle which is the angle formed by two lines connecting a certain keypoint and two adjacent keypoints respectively. For example, when referring to the keypoint angle of the right knee, it refers to the angle formed by the line connecting the right knee and the right waist and the line connecting the right knee and the right ankle. Also, the angle may be an inter-region angle which is the angle formed by two lines connecting a certain region and two adjacent regions respectively. For example, when referring to the trunk angle, it refers to the angle formed by the line connecting the center of gravity and the center of the head and neck (it may also be the nose) and the line connecting the center of gravity and the tip of one lower limb (it may also be the ankle). Hereinafter, the keypoint angle and the inter-region angle are collectively referred to as part angles. Also, the statistical values of the physical quantities include movement amount, movement range, average value, maximum value, and minimum value. The calculation includes arithmetic operations. The feature quantities calculated for the selected predetermined part are set and registered in the server 10 in advance for each action phase. The feature quantity is defined by selecting from a predetermined part, physical quantity, statistical value, and calculation respectively, for example, like "movement amount of the center of gravity within a period - movement range of the center of gravity within a period".

[0088] When the functional operation type is "sitting and standing test", the first phase is also called the rising phase, and the second phase is also called the sitting phase. The rising phase is the period when the subject adjusts the rising posture according to the inclination of the upper body and the inclination below the knees when rising from the sitting state. The sitting phase is the period when the subject adjusts the sitting posture according to the speed of the center-of-gravity movement when sitting down from the standing state. In the evaluation report, the inclination of the upper body is rephrased and expressed as "bowing", the inclination below the knees is expressed as "dragging the feet", and the speed of the center-of-gravity movement is expressed as "thumping down to sit".

[0089] In the evaluation report, "bowing" is used as an indicator showing the angle at which the trunk is inclined forward when rising. From this indicator, it can be known whether the rising movement can be carried out smoothly. Specifically, as shown in Figure 6, when the angle at which the trunk is inclined forward (the trunk forward inclination angle, the so-called bowing angle) is less than the first forward inclination angle (a predetermined angle among, for example, 10° to 30°), it becomes difficult to perform a smooth rising movement. When the angle is greater than or equal to the second forward inclination angle (a predetermined angle among, for example, 30° to 50°), there may be a compensation such as a decrease in the muscle strength of the lower body or pain in the knees and hip joints. Ideally, the trunk should be at an angle of more than 15° and less than 40° when rising. Also, "dragging the feet" is used as an indicator showing the angle at which the part below the knees is inclined toward the furniture side when rising (the lower leg forward inclination angle, the so-called dragging the feet angle). Similar to "bowing", from this indicator, it can be known whether the rising movement can be carried out smoothly. Specifically, as shown in Figure 6, when the inclination angle from the heel of the foot to the knee with respect to the ground (the dragging the feet angle) is greater than or equal to the first dragging the feet angle (a predetermined angle among, for example, 60° to 80°), it is considered that the ankle is stiff and it becomes difficult to perform a smooth rising movement. Ideally, the dorsiflexion angle should be less than a predetermined dorsiflexion angle (a predetermined angle among, for example, 60° to 80°) when rising. And "thumping down to sit" is used as an indicator showing the speed of the center-of-gravity movement during the sitting movement. From this indicator, it can be known whether there is the ability to control the speed of the center-of-gravity movement during the sitting movement.

[0090] When the functional operation type is "sitting and standing test", it is preferable to use the statistical value of one or more joint angles or the calculation result thereof as the input data for the first phase. Similarly, for example, as the input data for the second phase, it is preferable to use the statistical value of the center of gravity or the calculation result thereof and the statistical value of one or more joint angles or the calculation result thereof.

[0091] In step S206, the evaluation unit 1034 of the server 10 infers the quality of the movement of the specific part of the person to be evaluated in each movement phase based on the keypoint coordinates acquired in step S201 for each movement phase (phase part-by-part evaluation process). When two or more videos are captured in one measurement, the phase evaluation process is executed for each video. Specifically, the evaluation unit 1034 uses the keypoint coordinates of each frame of the nth movement phase video to calculate, for a predetermined part, a physical quantity including the coordinate position, a statistical value of the physical quantity, and a calculation result of the physical quantity and / or the statistical value as a feature quantity. Then, the evaluation unit 1034 applies the calculated feature quantity as input data to the evaluation mathematical model 1022 and obtains the quality of the movement of the specific part inferred as output data. Then, the evaluation unit 1034 associates the quality of the movement of the inferred specific part with the movement phase and stores it in the item of the phase part-by-part evaluation result in the evaluation table 1013 of the storage unit 101. The predetermined part in this process varies for each specific part to be evaluated. Also, the predetermined part in this process is selected from the above-mentioned 33 keypoints and six regions (head and neck, trunk, left upper limb, right upper limb, left lower limb, right lower limb), and the number of selected parts is preferably in the range of 2 to 8. The predetermined part to be selected is set and registered in the server 10 in advance for each specific part to be evaluated.

[0092] When the functional operation type is "sitting and standing test", in this embodiment, as specific parts, the movements of "shoulder", "hip joint", "knee", "ankle", "heel", and "index finger" are evaluated as good or bad. As input data, it is preferable to use the statistical values of the angles of one or more parts and the statistical value of the center of gravity, or the calculation results thereof. For example, using the keypoint coordinates of each frame of the nth set of operation phase videos, a physical quantity including the position of the shoulder of the person to be evaluated, the statistical value of the physical quantity, the physical quantity and / or the calculation result of the statistical value are calculated as feature quantities. The feature quantity obtained from the position of the shoulder is applied to the evaluation mathematical model 1022 as input data, and the evaluation unit 1034 acquires the evaluation result of the quality of the inferred shoulder movement as output data. As input data for the evaluation of the "ankle", the physical quantity, statistical value, and calculation result of the keypoints of the lower limb may be excluded. For example, as shown in FIG. 6, in the first phase of "bowing", the statistical value of the angle formed by the line connecting the "shoulder" and the "hip joint" with respect to the vertical virtual line P, the statistical value of the center of gravity, or the calculation result thereof is used. Also, in the "leg drag", the statistical value of the angle formed by the line connecting the "knee" and the "heel" with respect to the horizontal virtual line Q, the statistical value of the center of gravity, or the calculation result thereof is used. Note that the keypoints and the six regions to be used are not limited to these.

[0093] In step S207, the evaluation unit 1034 of the server 10 evaluates the presence or absence of compensatory movements in the functional movements of the person to be evaluated based on the converted video (compensation evaluation step). Compensatory movements include upper limb compensatory movements and lower limb compensatory movements. This evaluation is performed for each movement phase. When shooting two or more videos in one measurement, the full movement period determination step is executed for each converted video corresponding to each video. Specifically, the evaluation unit 1034 calculates, for a predetermined part, a physical quantity including coordinate positions, a statistical value of the physical quantity, and a calculation result of the physical quantity and / or the statistical value as feature quantities using the keypoint coordinates of each frame of the nth motion phase video. The evaluation unit 1034 applies the calculated feature quantities as input data to the evaluation mathematical model 1022, and acquires the presence or absence of the estimated upper limb compensatory motion and / or lower limb compensatory motion as output data. Then, the evaluation unit 1034 stores the presence or absence of the acquired compensatory motion in the evaluation information item of the evaluation table 1013 in the storage unit 101 of the server 10 as the nth phase compensatory motion evaluation result.

[0094] When the functional motion type is the "sitting and standing test", the evaluation unit 1034 first applies, as input data, the feature quantities of a predetermined part of the first set phase video to the evaluation mathematical model 1022, and acquires the presence or absence of the estimated upper limb compensatory motion as output data. The acquired output data is stored in the evaluation result item of the evaluation table 1013 in the storage unit 101 of the server 10 as the first phase compensatory motion evaluation result. The second phase is acquired in the same manner and stored in the evaluation result item of the evaluation table 1013 as the second phase compensatory motion evaluation result. Also, the evaluation is performed for each of the time of standing up and sitting down.

[0095] The predetermined part in this process is selected from the above-mentioned 33 keypoints and six regions (head and neck, trunk, left upper limb, right upper limb, left lower limb, right lower limb). The number of selected predetermined parts is preferably in the range of 2 to 5, and is preset for each of the upper limb compensatory motion and the lower limb compensatory motion. Examples of the feature quantities of the upper limb compensatory motion include the movement amount of a predetermined part selected from the upper limb region and the statistical value of the part angles of a plurality of predetermined parts selected from the upper limb region. Examples of the feature quantities of the lower limb compensatory motion include the statistical value of the center of gravity coordinates and the statistical value of a plurality of part angles. Also, the feature quantities of the lower limb compensatory motion may exclude the keypoints of the lower limb region. When the functional operation type is "sitting and standing test", as the feature quantity of the upper limb compensatory movement, at least one part such as "shoulder", "elbow", "wrist", etc. is selected from the upper limb area, and the statistical values of the positions and part angles of the selected parts are cited.

[0096] In step S208, the evaluation unit 1034 of the server 10 evaluates the relative positional relationship between the person to be evaluated and the environmental information based on the converted video (environmental evaluation process). The relative positional relationship is the positional relationship between the person to be evaluated and the furniture. Specifically, the evaluation unit 1034 calculates the positional relationship based on the environmental information acquired by the environmental information acquisition unit 1033 and the coordinate positions of the predetermined parts of the person to be evaluated reflected in the converted video. The calculated positional relationship is applied to the evaluation mathematical model 1022 as input data, and the estimated relative positional relationship is acquired as output data. Then, the evaluation unit 1034 stores the acquired relative positional relationship in the item of the evaluation information in the evaluation table 1013 of the storage unit 101 of the server 10 as the relative positional relationship evaluation result. When the functional operation type is "sitting and standing test", the relative positional relationship is the position where the person to be evaluated sits, that is, whether the person to be evaluated sits shallowly or deeply. For example, it is possible to evaluate whether the person to be evaluated is sitting shallowly or deeply based on the distance between the leg of the furniture and below the knees. In this case, the feature points of the end or end face of the furniture are extracted from the converted video, the horizontal distance from there to the key point of the knee is calculated, and if it is equal to or greater than a predetermined distance, it is determined as shallow sitting, and if it is less than the predetermined distance, it is determined as deep sitting. In addition, the determination of the depth of sitting may be further provided with a threshold value and determined in three or more levels such as shallow, normal, and deep. Also, when there is a backrest, it is possible to evaluate whether the person to be evaluated is leaning forward or not based on the distance between the backrest of the furniture and the back of the person to be evaluated. In this case, the feature points of the backrest of the furniture are extracted from the converted video, the horizontal distance from there to the key point of the shoulder is calculated, and if it is less than a predetermined distance, it is determined that the person is leaning forward, and if it is equal to or greater than the predetermined distance, it is determined that the person is not leaning forward.

[0097] In step S209, the evaluation unit 1034 of the server 10 generates an overall evaluation from the determination result of the entire operation period Ta in step S203, the determination result of each phase period Tn in step S204, the evaluation result of each phase in step S205, the evaluation result for each part of each phase in step S206, and the evaluation result of the compensation operation in step S207 (evaluation result integration step). The overall evaluation is, for example, an index indicating the evaluation of the quality of the motor function generated by scoring each result and performing aggregation processing and statistical processing. The overall evaluation may also be generated using the evaluation mathematical model 1022.

[0098] <Output example> FIG. 11 is a diagram showing an example of an evaluation report. The evaluation report may be displayed on the display 2071 of the user terminal 20, may be displayed on the display of another user terminal (not shown) used by an elderly person or a third party, or may be printed out on paper. By providing these evaluation reports and the like to the person being evaluated, their family members, and caregivers, the user can improve the reliability and smooth communication.

[0099] The evaluation report in Fig. 11 exemplifies the evaluation results (denoted as measurement results in Fig. 11) for the "sitting and standing motion test" among the functional operation types. The evaluation report in Fig. 11 is the evaluation result when the set operation of the sitting and standing motion is repeated 5 times. The "lower body score" shows the current result on a scale of 1 to 5 points for the comprehensive evaluation of the evaluation information in evaluation table 1013. "1 point" indicates that the set operation could not be performed, "2 points" indicates that the set operation was performed 1 or more times but less than 5 times, "3 points" indicates that there was hand assistance and the set operation could be performed 5 times to reach the standard (average) value Tna, "4 points" indicates that there was no hand assistance and the set operation could be performed 5 times to reach the standard (average) value Tna, and "5 points" indicates that in the state of crossing the arms in front of the chest, it was better than the standard (average) value Tna. The "comprehensive comment" shows the feedback data generated based on the evaluation information. Note that although the "lower body score" shows the comprehensive evaluation on a scale of 1 to 5 points, it is not limited to a 5 - level scoring method. It could also be an evaluation with only two levels of good or bad, or a graded evaluation with three or more levels. Also, although the presence or absence of hand assistance is the object of evaluation, it could be evaluated and the results displayed without taking the presence or absence of hand assistance as the object of evaluation. The "change in lower body score" and the "change in seconds" display the history of the full - motion period determination results in a graph. The results of the "change in lower body score" and the "change in seconds" show whether the evaluation is "decrease", "maintain", or "improve" compared to the previous result. The "standing state" shows the current and previous results of "shallow bow", "appropriate", and "deep bow" in the "bow" of the evaluation information in evaluation table 1013. Similarly, it shows the current and previous results of "insufficient pull" and "sufficient pull" in the "foot pull". The "comment" shows the feedback data generated based on the evaluation information of "bow" and "foot pull". The "improvement advice" shows the feedback data generated for the "functional aspect" and the "environmental aspect" based on the comprehensive evaluation of the evaluation information in evaluation table 1013.

[0100] In addition, a list of exercise menus is displayed (not shown) from the comprehensive evaluation of the evaluation information in the evaluation table 1013. In the list of exercise menus, for example, as recommended exercises, a list of exercise menus is shown for each category of balance exercises, movement practice, and muscle strength / flexibility improvement exercises. The displayed exercise menus correspond to the proposed exercise menu information in the report information of the evaluation table 1013. When the person to be evaluated is an elderly person, it is preferable that at least one exercise is presented in the category of balance exercises and movement practice. When the user or the person to be evaluated selects an exercise menu on the user terminal 20, the content of the exercise menu is displayed. The items displayed in the exercise menu include basic information including "points to be aware of", "difficulty level", "number of times (number of times and sets)", "points to note", and "free description", and exercise content that explains the content of the exercise. The content of the exercise may be a combination of one or more images and explanatory text, or may be a video.

[0101] As described above, the information processing system 1 according to the present disclosure can evaluate exercise functions such as the balance function, analyze the evaluation, and provide the user or the person to be evaluated with an exercise menu selected based on the evaluation of the exercise function and the analysis of the evaluation, with only video shooting and minimal necessary input. That is, the information processing system 1 can easily and more accurately evaluate the exercise function, propose effective rehabilitation, and improve the quality of rehabilitation. In addition, according to the information processing system 1, the user can easily measure the exercise function such as the balance ability of the person to be evaluated from the user terminal 20.

[0102] <Program> FIG. 12 is a schematic block diagram showing the configuration of the computer 801. The computer 801 includes a CPU 802 (processor), a main storage device 803, an auxiliary storage device 804, an interface 805, and an image processing device (GPU) 806.

[0103] Here, the details of the program for realizing each function constituting the server 10 according to the above embodiment will be described.

[0104] Server 10 is implemented in computer 801. The operations of each component of server 10 are stored in auxiliary storage device 804 in the form of a program. CPU 802 reads the program from auxiliary storage device 804, expands it in main storage device 803, and executes the above processing according to the program. Also, CPU 802 secures a storage area corresponding to the above-described storage unit 101 in main storage device 803 or auxiliary storage device 804 according to the program.

[0105] Specifically, the program is a program for causing a computer including a processor and a storage unit to execute. The program causes the processor to perform steps of acquiring an evaluation video obtained by imaging the entire body of an evaluation target person performing a sitting and standing motion and period information of the sitting and standing motion, estimating key points including joint points and the center of gravity of the evaluation target person reflected in the evaluation video, and generating an evaluation result of the muscle strength and balance ability of the evaluation target person using a mathematical model from the key points and the elapsed time of the sitting and standing motion. The sitting and standing motion includes a rising phase and a sitting phase as motion phases, and the step of generating the evaluation result is a step of generating the evaluation result for each motion phase.

[0106] Note that auxiliary storage device 804 is an example of a non-transitory tangible medium. Other examples of non-transitory tangible media include magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, semiconductor memories, etc. connected via interface 805. Also, when this program is distributed to computer 801 via network NW, the receiving computer 801 may expand the program in main storage device 803 and execute the above processing.

[0107] Also, the program may be for realizing a part of the above-described functions. Furthermore, the program is realized in combination with other programs already stored in auxiliary storage device 804, a so-called difference file (difference program). It may be.

[0108] As described above, the embodiments of the present invention have been explained. However, these embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are to be included in the scope and gist of the invention, and are also to be included in the invention described in the claims and the equivalent scope thereof.

[0109] With the above, the description of the embodiments of the present invention is completed. However, the aspects of the present invention are not limited to this embodiment.

[0110] For example, in the measurement procedure in the above embodiment, during the imaging of the evaluation video, the video processing unit 1032 recognizes a specific part of the body of the person to be evaluated reflected in the evaluation video and an object around the person to be evaluated, respectively. The video acquisition unit 1031 may automatically end the imaging of the evaluation video when the positional relationship between the specific part of the body and the object around satisfies a predetermined condition.

[0111] Also, in the measurement procedure of the exercise function in the above embodiment, the user measures the total operation period Ta. However, the video acquisition unit 1031 may recognize the operation of the person to be evaluated in real time and calculate the total operation period Ta. This makes the measurement easier.

[0112] Also, in step S209 of the above embodiment, the comprehensive evaluation was generated from the determination result of the total operation period, the determination result of each phase period, the comprehensive evaluation result of each phase, the part-by-part evaluation result of each phase, and the compensatory operation evaluation result. However, one of these determination results and evaluation results may be used as the comprehensive evaluation, or it may be generated from a combination of two or more.

[0113] Also, in step S202 of the above embodiment, the position for dividing the operation phase was estimated using all the frames of the converted video. However, it is not necessary to use all the frames as long as the range from the start to the end of the functional operation is reflected in the frames of the converted video, and a part may be sufficient.

[0114] In the above-described embodiment, no score is provided for the relative position relationship evaluation result, but a relative position score with the furniture corresponding to the relative position relationship evaluation result may be provided.

[0115] <Supplementary Note> The configuration of this embodiment is illustrated as follows. [1] A video acquisition unit that acquires an evaluation video obtained by imaging the entire body of an evaluation subject performing a standing-sitting motion from a seated state and period information of the standing-sitting motion; A video processing unit that estimates key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; An evaluation unit that generates an evaluation result of the muscle strength and balance ability of the evaluation subject using a mathematical model from the key points and the period information of the standing-sitting motion; Comprising The standing-sitting motion includes a rising phase and a sitting phase as motion phases. The evaluation unit generates the evaluation result for each of the motion phases. An information processing device. Thereby, when evaluating the balance ability as a motor function, it is possible to evaluate for each motion phase of motor functions with different feature amounts while considering both joint points and the center of gravity, and the accuracy of the evaluation of the motor function can be improved. In addition, since the motor function can be measured only by a simple operation of video shooting, the measurement can be labor-saving. That is, the motor function of the evaluated person can be evaluated easily and more accurately. [2] The video acquisition unit acquires, as the evaluation video, an upper body evaluation video for evaluating the inclination of the upper body and a lower knee evaluation video for evaluating the inclination of below the knees at the time of rising. The evaluation unit generates the evaluation result based on the upper body evaluation video and the lower knee evaluation video. The information processing device according to [1]. Thereby, it is possible to present evaluations of the inclination of the upper body and the inclination of below the knees respectively. [3] The evaluation unit inputs at least any one of the physical quantity of the key point during the rising phase, the statistical value of the physical quantity, and the calculation result of the physical quantity and the statistical value as input data of the mathematical model, and obtains the evaluation result in the rising phase as output data from the mathematical model. The information processing apparatus according to [1] or [2]. Thereby, since the motor function is evaluated in consideration of the feature amount affecting the rising phase, an evaluation result specialized for the rising phase can be provided. [4] The evaluation unit inputs the physical quantity of the key point during the sitting phase and / or the statistical value of the physical quantity of the key point as input data to the mathematical model, and obtains the evaluation result in the sitting phase as output data from the mathematical model. The information processing apparatus according to any one of [1] to [3], comprising the above. Thereby, since the motor function is evaluated in consideration of the feature amount affecting the sitting phase, an evaluation result specialized for the sitting phase can be provided. [5] The evaluation unit inputs the physical quantity of the key point related to the upper limb compensatory movement of the evaluation target person and / or the statistical value of the physical quantity of the key point as input data to the mathematical model, and obtains the presence or absence of the upper limb movement as output data from the mathematical model. The information processing apparatus according to any one of [1] to [4]. Thereby, since the motor function is evaluated in consideration of the upper limb compensatory movement, a more specific evaluation result can be presented. Further, a more appropriate exercise menu can be proposed for the upper limb compensatory movement. [6] Furthermore, it has an environmental information acquisition unit that acquires environmental information including information about furniture used by the evaluation target person for evaluating the sitting and rising movements. The evaluation unit inputs the environmental information acquired by the environmental information acquisition unit as input data to the mathematical model, and obtains the evaluation result in the movement phase as output data from the mathematical model. The information processing apparatus according to any one of [1] to [5]. Accordingly, since the exercise function is evaluated in consideration of the furniture on which the person to be evaluated is seated, a more specific evaluation result can be presented. [7] The information about the furniture includes at least any one of the dimensional information of the furniture on which the person to be evaluated is seated, the presence or absence of a backrest of the furniture, and the presence or absence of armrests. The information processing apparatus according to [6]. Accordingly, since the exercise function is evaluated in consideration of the furniture used by the person to be evaluated, a more specific evaluation result can be presented. Furthermore, more appropriate furniture can be proposed. [8] The evaluation unit includes, in the evaluation result, an evaluation based on the environmental information acquired by the environmental information acquisition unit and the relative positional relationship between the environmental information and the person to be evaluated shown in the evaluation video. The information processing apparatus according to [6] or [7]. Accordingly, by including environmental information, for example, the positional relationship with the person to be evaluated seated on a chair, in the evaluation result, a more specific evaluation result can be presented. [9] The video acquisition unit acquires the evaluation video that captures a plurality of standing and sitting motions of the person to be evaluated. The evaluation unit includes, in the evaluation result, an evaluation based on the variation in the evaluation of each of the plurality of standing and sitting motions. The information processing apparatus according to any one of [1] to [8]. Accordingly, each standing and sitting motion can be evaluated, and a more detailed and specific evaluation result can be presented.

[10] The information processing apparatus further includes a storage unit that stores the evaluation result. The evaluation unit analyzes the current evaluation result and the past evaluation results to generate report information. The information processing apparatus according to any one of [1] to [9]. Accordingly, the effect of rehabilitation can be confirmed.

[11] The storage unit stores a plurality of exercise menus. Further comprising an exercise proposal unit that selects one or more of the exercise menus based on the evaluation results and the period information. The information processing apparatus according to

[10] . Thereby, an exercise menu appropriate for the current state of the exercise function can be presented.

[12] An information processing method executed by an information processing apparatus including a processor and a storage unit, the method comprising causing the processor to: Obtaining an evaluation video that captures the entire body of an evaluation subject performing a sit-to-stand motion from a seated state and period information of the sit-to-stand motion; Estimating key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; Generating evaluation results of the muscle strength and balance ability of the evaluation subject using a mathematical model from the key points and the elapsed time of the sit-to-stand motion; To execute, The sit-to-stand motion includes a rising phase and a sitting phase as motion phases. The step of generating the evaluation results is a step of generating the evaluation results for each of the motion phases. Information processing method.

[13] A program for causing a computer including a processor and a storage unit to execute, the program causing the processor to: Obtaining an evaluation video that captures the entire body of an evaluation subject performing a sit-to-stand motion and period information of the sit-to-stand motion; Estimating key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; Generating evaluation results of the muscle strength and balance ability of the evaluation subject using a mathematical model from the key points and the elapsed time of the sit-to-stand motion; To execute, The sit-to-stand motion includes a rising phase and a sitting phase as motion phases. To execute, The step of generating the evaluation result is the step of generating the evaluation result for each of the operation phases. Program.

[14] An information processing system comprising an information processing device and a user terminal, the information processing device including a processor and a storage unit, causing the processor to acquire an evaluation video obtained by imaging the entire body of an evaluation subject performing a sitting-down and standing-up motion, and period information of the sitting-down and standing-up motion; estimate key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; generate an evaluation result of the muscle strength and balance ability of the evaluation subject from the key points and the elapsed time of the sitting-down and standing-up motion using a mathematical model; execute, wherein the sitting-down and standing-up motion includes a standing-up phase and a sitting-down phase as operation phases, and the step of generating the evaluation result is the step of generating the evaluation result for each of the operation phases. Information processing system.

Explanation of Signs

[0116] 1 Information processing system 10 Information processing server 20 User terminal 30 Furniture 101 Storage unit 103 Control unit 201 Storage unit 203 Control unit 205 Input device 207 Output device 801 Computer 802 CPU 803 Main storage device 804 Auxiliary storage device 805 Interface 1011 Application program 1012 Video table 1013 Evaluation table 1014 Motion Information Table 1015 User Table 1016 Evaluated Person Table 1021 Video Analysis Mathematical Model 1022 Evaluation Mathematical Model 1023 Proposal Mathematical Model 1031 Video Acquisition Unit 1032 Video Processing Unit 1033 Environment Information Acquisition Unit 1034 Evaluation Unit 1035 Exercise Proposal Unit 1036 Evaluation Result Output Unit 1037 Mathematical Model Management Unit 2011 Application Program 2012 User ID 2031 Input Control Unit 2032 Output Control Unit 2051 Camera 2052 Microphone 2053 Position Information Sensor 2054 Motion Sensor 2055 Touch Device 2071 Display 2072 Speaker NW Network P Vertical Virtual Line Q Horizontal Virtual Line

Claims

1. A video acquisition unit that acquires an evaluation video that captures the entire body of an evaluation subject performing a standing-sitting motion from a seated state, and period information of the standing-sitting motion; A video processing unit that estimates key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; An evaluation unit that generates an evaluation result of the muscle strength and balance ability of the evaluation subject using a mathematical model from the key points and the period information of the standing-sitting motion; Comprising: The standing-sitting motion includes, as motion phases, a rising phase and a sitting phase; Including: The evaluation unit generates the evaluation result for each of the motion phases; The evaluation unit inputs at least any one of the physical quantity of the key points during the rising phase, the statistical value of the physical quantity, and the calculation result of the physical quantity and the statistical value as input data of the mathematical model, and obtains the evaluation result in the rising phase as output data from the mathematical model; The evaluation unit inputs, as input data to the mathematical model, the physical quantity of the key points related to the upper limb compensatory motion of the evaluation subject and / or the statistical value of the physical quantity of the key points, and obtains the presence or absence of the upper limb compensatory motion as output data from the mathematical model, an information processing device.

2. A video acquisition unit that acquires an evaluation video that captures the entire body of an evaluation subject performing a standing-sitting motion from a seated state, and period information of the standing-sitting motion; A video processing unit that estimates key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; An evaluation unit that generates an evaluation result of the muscle strength and balance ability of the evaluation subject using a mathematical model from the key points and the period information of the standing-sitting motion; Comprising: The standing-sitting motion includes, as motion phases, a rising phase and a sitting phase; Including: The evaluation unit generates the evaluation result for each of the motion phases; The evaluation unit inputs, as input data to the mathematical model, the physical quantity of the key points during the sitting phase and / or the statistical value of the physical quantity of the key points, and obtains the evaluation result in the sitting phase as output data from the mathematical model; The evaluation unit inputs, as input data to the mathematical model, the physical quantity of the key point related to the upper limb compensatory movement of the person to be evaluated, and / or the statistical value of the physical quantity of the key point, and obtains the presence or absence of the upper limb compensatory movement as output data from the mathematical model. An information processing device.

3. A video acquisition unit that acquires an evaluation video that captures the entire body of a person to be evaluated performing a standing-sitting movement from a seated state, and period information of the standing-sitting movement; A video processing unit that estimates key points including joint points and the center of gravity of the person to be evaluated reflected in the evaluation video; An evaluation unit that generates an evaluation result of the muscle strength and balance ability of the person to be evaluated using a mathematical model from the key points and the period information of the standing-sitting movement; Comprising: The standing-sitting movement includes, as movement phases, a rising phase and a sitting phase; Including: The evaluation unit generates the evaluation result for each of the movement phases; Furthermore, it has an environment information acquisition unit that acquires environment information including information about the furniture used by the person to be evaluated for evaluating the standing-sitting movement; The evaluation unit inputs the environment information acquired by the environment information acquisition unit as input data to the mathematical model, and acquires the evaluation result in the movement phase as output data from the mathematical model. An information processing device.

4. The information about the furniture includes at least any one of the dimensional information of the furniture on which the person to be evaluated sits, the presence or absence of the backrest of the furniture, and the presence or absence of armrests. The information processing device according to claim 3.

5. The evaluation unit includes, in the evaluation result, an evaluation based on the relative positional relationship between the environment information acquired by the environment information acquisition unit and the person to be evaluated reflected in the evaluation video. The information processing device according to claim 3.

6. The video acquisition unit acquires the evaluation video that captures a plurality of standing-sitting movements of the person to be evaluated; The evaluation unit includes, in the evaluation result, an evaluation based on the variation in the evaluation of each of the plurality of standing-sitting movements. The information processing device according to any one of claims 1 to 3.

7. Furthermore, it includes a storage unit that stores the evaluation result; The evaluation unit analyzes the current evaluation result and the past evaluation result to generate report information. The information processing device according to any one of claims 1 to 3.

8. The storage unit stores a plurality of exercise menus. Further comprising an exercise proposal unit that selects one or more of the exercise menus based on the evaluation result and the period information. The information processing apparatus according to claim 7.

9. A video acquisition unit that acquires an evaluation video obtained by imaging the entire body of an evaluation subject who performs a standing-sitting motion a plurality of times from a seated state; A video processing unit that estimates key points including joint points of the evaluation subject shown in the evaluation video; An evaluation unit that generates an evaluation result of the muscle strength or balance ability of the evaluation subject using the key points as input data; Comprising; The standing-sitting motion includes, as motion phases, a rising phase and a sitting phase; Including; The evaluation unit uses the coordinate positions of 3 or more and 8 or less of the key points in the lower limb region of the plurality of standing-sitting motions and the statistical value of the angle formed by two lines connecting any two adjacent key points to generate the evaluation result of the muscle strength or balance ability of the lower limb of the evaluation subject in the rising phase or the sitting phase. Furthermore, it has an environmental information acquisition unit that acquires environmental information including information about the furniture used by the evaluation subject to evaluate the standing-sitting motion. The evaluation unit inputs the environmental information acquired by the environmental information acquisition unit into a mathematical model as input data, and acquires the evaluation result in the motion phase as output data from the mathematical model. An information processing apparatus.

10. An information processing method executed by an information processing apparatus including a processor and a storage unit, wherein the processor Obtaining an evaluation video obtained by imaging the entire body of an evaluation subject who performs a standing-sitting motion from a seated state and the period information of the standing-sitting motion; Estimating key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; Inputting the environmental information acquired by the environmental information acquisition unit including information about the furniture used by the evaluation subject to evaluate the standing-sitting motion into a mathematical model as input data; Generating an evaluation result of the muscle strength and balance ability of the evaluation subject using the mathematical model from the key points and the elapsed time of the standing-sitting motion; Causing to execute; The standing-sitting motion includes, as motion phases, a rising phase and a sitting phase; Including; The step of generating the evaluation result is the step of generating the evaluation result for each motion phase. Information processing method.

11. A program for causing a computer including a processor and a storage unit to execute, the program causes the processor to, acquire an evaluation video that captures the entire body of an evaluation subject performing a sit-to-stand motion and period information of the sit-to-stand motion; estimate key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; input, as input data, environmental information acquired by an environmental information acquisition unit that acquires environmental information including information about furniture used by the evaluation subject for evaluating the sit-to-stand motion into a mathematical model; generate an evaluation result of the muscle strength and balance ability of the evaluation subject using the mathematical model from the key points and the elapsed time of the sit-to-stand motion; execute, the sit-to-stand motion includes, as motion phases, a rising phase and a sitting phase, including, the step of generating the evaluation result is a step of generating the evaluation result for each of the motion phases. Program.

12. An information processing system including an information processing device and a user terminal each including a processor and a storage unit, the processor, acquire an evaluation video that captures the entire body of an evaluation subject performing a sit-to-stand motion and period information of the sit-to-stand motion; estimate key points including joint points and the center of gravity of the evaluation subject shown in the evaluation video; input, as input data, environmental information acquired by an environmental information acquisition unit that acquires environmental information including information about furniture used by the evaluation subject for evaluating the sit-to-stand motion into a mathematical model; generate an evaluation result of the muscle strength and balance ability of the evaluation subject using the mathematical model from the key points and the elapsed time of the sit-to-stand motion; execute, the sit-to-stand motion includes, as motion phases, a rising phase and a sitting phase, the step of generating the evaluation result is a step of generating the evaluation result for each of the motion phases. Information processing system.

Images