Exercise support system and exercise support method

Abstract

To provide precise improvement information of exercise for a user.SOLUTION: An exercise support system includes: an exercise evaluation part 103 which calculates an exercise score 204 by evaluating exercise on the basis of picture information 201 concerning user's exercise and which outputs a characteristic map 205 that is contribution for showing what articulation part is emphasized in calculating the exercise score 204; an evaluation model interpretation part 105 for calculating articulation score which is the score of exercise in the articulation part based on the characteristic map 205; a selection part 106 for selecting the articulation part based on the articulation score; an articulation time series information 210 concerning articulation exercise; a search part 107 for extracting a frame with the largest moment in the movement in the selected articulation part, of the articulation time series information 210, on the basis of the articulation identification information 207 which is the information concerning the selected articulation part; and an interpretation part 108 which generates information concerning the exercise of the articulation part in the frame extracted by the search part 107.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to the technology of an exercise support system and an exercise support method.

Background Art

[0002] In recent years, due to the trend towards health consciousness and the need to maintain a healthy lifespan in an aging society, the general public has had an increasing opportunity to engage in exercise. To meet such societal needs, technologies for supporting exercise have been proposed. As such technologies, for example, the technologies described in Patent Document 1 and Patent Document 2 are disclosed.

[0003] Patent Document 1 discloses an information processing apparatus, method, and program: "A server measures the movement of a user's body by a measurement unit, and based on an evaluation result obtained by evaluating the measurement result, identifies problems that may occur in the user's body. The server refers to a training menu associated with the problems that may occur in the body to identify a training menu associated with the problem, and presents the identified training menu to the user. The server causes the user's terminal device to display the content of the problems that may occur in the body and the training menu for addressing the problems." (See the abstract).

[0004] Further, Patent Document 2 discloses an exercise evaluation improvement system and an exercise evaluation improvement method: "A feature quantity extraction unit that extracts a feature quantity related to exercise efficiency from information obtained from a measured person, an evaluation unit that evaluates the exercise efficiency of the measured person by comparing the extracted feature quantity with a feature quantity to be a comparison target, and an improvement information providing unit that provides information such that the feature quantity of the measured person approaches the feature quantity to be a comparison target based on the evaluation result." (See the abstract).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] However, the technologies described in Patent Document 1 and Patent Document 2 do not include any interpretations that suggest improvements to the movement, and users can only obtain general information about improvements. Therefore, the technologies described in Patent Document 1 and Patent Document 2 have challenges in improving the user's improvement efficiency.

[0007] In light of this background, the present invention was made, and its objective is to provide users with accurate information on how to improve their exercise. [Means for solving the problem]

[0008] To solve the aforementioned problems, the present invention includes: an exercise evaluation unit that calculates an exercise score by evaluating the exercise based on video information relating to the user's exercise and outputs a feature map which is a contribution that indicates which joint area was emphasized when calculating the exercise score; a joint score calculation unit that calculates a joint score which is a score of the exercise at the joint area based on the feature map; a selection unit that selects the joint area based on the joint score; a search unit that extracts the frame at the moment when the movement at the selected joint area is greatest from the frames constituting the time series information, based on time series information consisting of a plurality of frames relating to the movement of the joint; an interpretation unit that generates information on the movement of the joint area in the frame extracted by the search unit; and an output processing unit that outputs the exercise score, the joint score, and the information generated by the interpretation unit to an output unit, wherein the output processing unit indicates the joint area with impairment and / or joint area with impaired movement in color based on medical record information which stores information about the user's physical condition. The search unit then uses the angle of the selected joint at the moment the movement begins as the initial value, and extracts the frame at the moment when the angle of the joint becomes the largest value from the initial value as the frame at the moment of greatest movement. It is characterized by the following: Other solutions will be described as appropriate in the embodiments. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide users with accurate information on how to improve their exercise. [Brief explanation of the drawing]

[0010] [Figure 1] This is a functional block diagram showing an example configuration of the exercise support system according to the 1-1 embodiment. [Figure 2] This is a flowchart showing the processing procedure of the exercise support system according to the first embodiment. [Figure 3] This figure shows an example of the joint score information output screen. [Figure 4] This figure shows an example of a display screen for exercise score and posture characteristics information. [Figure 5] This is a functional block diagram showing an example configuration of the exercise support system according to the first-second embodiment. [Figure 6] This is a flowchart showing the processing procedure of the exercise support system according to the first-second embodiment. [Figure 7] This is a functional block diagram showing an example configuration of the exercise support system according to the 1-3 embodiment. [Figure 8] This is a flowchart showing the processing procedure of the exercise support system according to the first to third embodiment. [Figure 9] This is a functional block diagram showing an example configuration of the exercise support system according to the 1st to 4th embodiment. [Figure 10] This is a flowchart showing the processing procedure of the exercise support system according to the first to fourth embodiment. [Figure 11] This is a functional block diagram showing an example configuration of the exercise support system according to the second-first embodiment. [Figure 12] This is a flowchart showing the processing procedure of the exercise support system according to the second-first embodiment. [Figure 13] This is a functional block diagram showing an example configuration of the exercise support system according to the second embodiment. [Figure 14]It is a flowchart showing the processing procedure of the motion assistance system according to the 2-2 embodiment. [Figure 15] It is a functional block diagram showing a configuration example of the motion assistance system according to the 2-3 embodiment. [Figure 16] It is a flowchart showing the processing procedure of the motion assistance system according to the 2-3 embodiment. [Figure 17] It is a functional block diagram showing a configuration example of the motion assistance system according to the 2-4 embodiment. [Figure 18] It is a flowchart showing the processing procedure of the motion assistance system according to the 2-4 embodiment. [Figure 19] It is a functional block diagram showing a configuration example of the motion assistance system according to the 3rd embodiment. [Figure 20] It is a flowchart showing the processing procedure of the motion assistance system according to the 3rd embodiment. [Figure 21] It is a diagram showing the hardware configuration of the motion assistance system. [Figure 22] It is a diagram (part 1) showing the configuration of the motion assistance system in the comparative example. [Figure 23] It is a diagram (part 2) showing the configuration of the motion assistance system in the comparative example.

Modes for Carrying Out the Invention

[0011] Next, modes for carrying out the present invention (referred to as "embodiments") will be described in detail with appropriate reference to the drawings.

[0012] [First - 1 Embodiment] (System Configuration) FIG. 1 is a functional block diagram showing a configuration example of the motion assistance system 1 according to the first - 1 embodiment. The movement support system 1 comprises an imaging unit 101, a joint location estimation unit 102, a movement evaluation unit 103, an output processing unit consisting of a movement evaluation result output processing unit 104, an evaluation model interpretation unit 105, and a selection unit 106. Furthermore, the movement support system 1 includes a search unit 107, an interpretation unit 108, an output processing unit consisting of an interpretation result output processing unit 109, and an output unit 110. In addition, the movement support system 1 has a trained model 301.

[0013] The following outlines the elements constituting the exercise support system 1 shown in Figure 1. Details of the processing performed by the imaging unit 101 to the output unit 110 shown in Figure 1, as well as details of the video information 201 to joint identification information 207 and joint time-series information 210 to posture feature information 212, are explained in Figure 2.

[0014] The shooting unit 101 consists of a camera and other equipment, and shoots video footage, outputting video information 201, which is the video information. The joint location estimation unit 102 extracts and outputs joint information 202 and joint time-series information 210, which is time-series information consisting of multiple frames relating to joint movement, from the video information 201. The joint time-series information 210 is time-series information for each joint location extracted from the video information 201. In this embodiment, the joint time-series information 210 is obtained by adding information about the user's joint locations to each frame of the video information 201. A frame is a frame of a video. A joint location is a place corresponding to the user's joints, such as the elbow or knee.

[0015] Joint information 202 is extracted from joint time-series information 210 and may consist of one frame, several representative consecutive frames, or all frames.

[0016] The motion evaluation unit 103 calculates and outputs a motion score 204 based on joint information 202 obtained from the joint location estimation unit 102. In other words, the motion evaluation unit 103 calculates the motion score 204 by evaluating the motion based on video information 201 related to the user's movements. The motion score 204 is an overall score for the movements performed by the user. If a neural network is used as the motion evaluation unit 103, the motion evaluation unit 103 calculates the motion score 204 based on joint information 202 and weight information 203 obtained from a trained model 301.

[0017] Furthermore, when calculating the exercise score 204, the exercise evaluation unit 103 outputs a feature map 205, which is a contribution score indicating which joint areas the exercise evaluation unit 103 itself emphasized in the calculation.

[0018] The exercise evaluation result output processing unit 104 outputs the exercise score 204 calculated by the exercise evaluation unit 103 to the output unit 110, such as a display. In this embodiment, the output unit 110 is assumed to be a display, so outputting to the output unit 110 means displaying it on a display, etc. However, the output unit 110 can also be a printer, etc.

[0019] The evaluation model interpretation unit 105, which is the joint score calculation unit, generates and outputs each joint score information 206 based on the feature map 205 obtained from the movement evaluation unit 103. Each joint score information 206 stores the movement score (joint score) for the joint site. In this way, the evaluation model interpretation unit 105 outputs the joint score, which is the movement score for the joint site, as each joint score information 206 based on the feature map 205. Thus, the interpretation performed by the evaluation model interpretation unit 105 is to calculate the joint score for each joint site.

[0020] The selection unit 106 selects joint areas for improvement in movement based on the joint score information 206 (joint scores) for each joint area. The selection unit 106 then outputs joint identification information 207, which is the ID (Identification) of the selected joint area, as information about the selected joint area. The selection unit 106 also outputs the joint score information 206 for each joint area to the interpretation result output processing unit 109. The search unit 107 searches for areas to be improved based on the joint time-series information 210 obtained from the joint area estimation unit 102 and the joint identification information 207 obtained from the selection unit 106. The search unit 107 outputs the search results for areas to be improved as joint area information 211. The interpretation unit 108 interprets the areas to be improved based on the joint area information 211 and outputs the result of this interpretation as posture feature information 212. The processing performed by the interpretation unit 108 will be described later, but specifically, the interpretation performed by the interpretation unit 108 includes calculating the angle of the joint area and the user's center of gravity from the captured image of the user.

[0021] The interpretation result output processing unit 109 then outputs the joint score information 206 and the posture feature information 212, which is information generated by the interpretation unit 108, to the output unit 110.

[0022] Furthermore, the imaging unit 101 and output unit 110 may be mounted on a user terminal such as a smartphone, tablet device, or personal computer owned by the user. The joint location estimation unit 102 to the interpretation result output processing unit 109, other than the imaging unit 101 and output unit 110, may be mounted on a server installed in a company or other organization. Alternatively, all of the imaging unit 101 to output unit 110 may be mounted on a user terminal such as a smartphone, tablet device, or personal computer owned by the user.

[0023] (flowchart) Figure 2 is a flowchart showing the processing procedure of the exercise support system 1 according to the first embodiment. Refer to Figure 1 as appropriate. First, the user films themselves exercising using the camera unit 101 (S101). The user can have someone else film them, or they can film themselves by fixing a smartphone or tablet device in place. Subsequently, the joint location estimation unit 102 acquires the video captured by the imaging unit 101 as video information 201 (S102). The joint location estimation unit 102 then performs joint location estimation processing based on the video information 201 (S103). The joint location estimation unit 102 extracts the joint locations of the person shown in each frame of the video information 201. The extracted joint locations are predetermined and include 33 locations (not limited to 33 locations), such as the right elbow, left elbow, right knee, left knee, right shoulder, and left shoulder. The joint location estimation unit 102 extracts joint locations for each frame of the video information 201. As described above, in this embodiment, the joint location estimation unit 102 outputs joint time-series information 210, which is the video information 201 with information about the joint locations added to each frame. The joint location estimation unit 102 may also generate joint time-series information 210 by connecting the extracted joint locations in frame order.

[0024] Furthermore, the joint location estimation unit 102 outputs, for each joint location, one frame, a representative series of consecutive frames, or all frames as joint information 202 from the joint time-series information 210. In this process, the joint location estimation unit 102 generates the joint information 202 by extracting the frames necessary for the movement evaluation processing performed by the movement evaluation unit 103 from the joint time-series information 210.

[0025] Furthermore, joint location estimation processing can be performed using two-dimensional or three-dimensional video information 201. If joint location estimation processing is performed using three-dimensional video information 201, it is necessary to capture the user from multiple directions in step S101.

[0026] The motion evaluation unit 103 performs motion evaluation processing based on the joint information 202 it receives (S111; motion evaluation step). In the motion evaluation processing, the motion evaluation unit 103 calculates a motion score 204. The motion score 204 calculated in step S111 is a total score calculated for the entire movement of the user captured by the imaging unit 101. When the motion evaluation unit 103 calculates the motion score 204 using a neural network, the motion evaluation unit 103 calculates the motion score 204 using the weight information 203 stored in the trained model 301 and the joint information 202. In this case, the weight information 203 becomes the weights between neurons in the neural network. The trained model 301 stores the weights between neurons that are output as a result of learning using joint information 202 for multiple people as input data and known motion scores 204 as ground truth data.

[0027] Furthermore, the motion evaluation unit 103 does not necessarily have to use machine learning with a neural network. In other words, the motion evaluation unit 103 may calculate the motion score 204 using machine learning other than a neural network. For example, the motion evaluation unit 103 may calculate the motion score 204 by comparing the joint information 202 in the sample video information with the joint information 202 input to the motion evaluation unit 103. In this case, the weight information 203 is not used.

[0028] The exercise evaluation unit 103 passes the calculated exercise score 204 to the exercise evaluation result output processing unit 104. The exercise evaluation result output processing unit 104 outputs the received exercise score 204 to the output unit 110 (S112).

[0029] Furthermore, the motion evaluation unit 103 generates a feature map 205 simultaneously with the motion score 204 (S121; motion evaluation step). As described above, the feature map 205 stores information about the weights (contributions) that indicate which joint areas were given importance when calculating the motion score 204. For example, if the right elbow and left elbow are given importance when calculating the motion score 204, large weights will be assigned to the joint areas of the right elbow and left elbow. Which joint areas are given importance is output by processing using a neural network in the motion evaluation unit 103. The feature map 205 stores the weights for all target joint areas. The motion evaluation unit 103 passes the generated feature map 205 to the evaluation model interpretation unit 105.

[0030] The evaluation model interpretation unit 105 generates (calculates) joint score information 206 based on the provided feature map 205 (S122; joint score calculation step). Each joint score information 206 stores a score (joint score) for each of the target joint parts. For example, each joint score is a score for each joint part such as the neck, right shoulder, left shoulder, right elbow, etc. Each joint score may also be the contribution of the joint part to the movement score 204 stored in the feature map 205.

[0031] The evaluation model interpretation unit 105 may calculate joint scores according to the movement score 204 calculated by the movement evaluation unit 103. For example, if the movement score 204 is 90 points, the evaluation model interpretation unit 105 calculates joint scores for joint areas that are performing well. If the movement score 204 is 40 points, the evaluation model interpretation unit 105 calculates joint scores for joint areas that are performing poorly. In other words, if the movement score 204 is above a predetermined value, the evaluation model interpretation unit 105 may output joint scores for joint areas that have joint scores above the predetermined value. Also, if the movement score 204 is below a predetermined value, the evaluation model interpretation unit 105 may output joint scores for joint areas that have joint scores below the predetermined value. Thus, although not shown in Figure 1, it is preferable that the movement score 204 is also input to the evaluation model interpretation unit 105.

[0032] Furthermore, even if the motor score 204 shows a high value, if there are joint scores below a predetermined value, the evaluation model interpretation unit 105 may output the joint scores for the joint sites that have such joint scores.

[0033] Each joint score information 206 is associated with the joint score and information about the joint location. Furthermore, the evaluation model interpretation unit 105 utilizes technologies such as XAI (Explainable AI). The evaluation model interpretation unit 105 passes the calculated joint score information 206 to the selection unit 106.

[0034] The selection unit 106 selects areas for improvement from each joint score information 206 (S123; selection step). Areas for improvement are joint areas that should be improved. In other words, the selection unit 106 selects joint areas whose joint score is less than or equal to a predetermined value. Specifically, in step S123, the selection unit 106 selects joint areas from among the joint scores that make up each joint score information 206 whose joint score is less than or equal to a predetermined value.

[0035] The selection unit 106 passes each joint score information 206 to the interpretation result output processing unit 109. The selection unit 106 may pass the joint scores selected in step S123 to the interpretation result output processing unit 109, or it may pass all joint scores, including joint areas not selected in step S123, to the interpretation result output processing unit 109. In this embodiment, the selection unit 106 is a process performed by a computer, but the user may manually select the areas to be improved based on each joint score information 206.

[0036] The interpretation result output processing unit 109 outputs the received joint score information 206 to the output unit 110 (S124).

[0037] Furthermore, the selection unit 106 passes the joint identification information 207, which is the ID of the joint area selected in step S123, to the search unit 107.

[0038] The search unit 107 obtains joint identification information 207 from the selection unit 106 and joint time-series information 210 from the joint location estimation unit 102. Then, the search unit 107 searches for areas to be improved based on the obtained joint time-series information 210 and joint identification information 207 (S131; search step). In step S131, the search unit 107 tracks the movement of the joint location corresponding to the joint identification information 207 in the joint time-series information 210. Then, for the tracked joint location, the search unit 107 extracts the frame of the moment when the movement is greatest. The moment of greatest movement is, for example, the moment when the angle of each joint location at the moment the movement begins is set as the initial value, and the joint location takes the largest value from the initial value. The angle of the joint location is the angle of the location corresponding to the two bones that are joined at the joint. For example, if the joint location is the elbow, the angle of the joint location is the angle between the upper arm and the forearm. Furthermore, in the case of knee flexion and extension, the knee is extended at the moment the movement begins, so the moment when the knee is most bent is the moment of greatest movement. In this way, the search unit 107 extracts the frame of the moment when the movement at the selected joint site is greatest from the frames that make up the joint time-series information 210.

[0039] The search unit 107 performs the processing of step S131 for all joint parts corresponding to the joint identification information 207. The search unit 107 outputs all the frames extracted in step S131 to the interpretation unit 108 as joint part information 211. In other words, the joint part information 211 contains the frame at the moment when the joint part moved the most in step S131, and the joint identification information 207. If there are multiple joint parts corresponding to the joint identification information 207, the joint part information 211 contains frames corresponding to each joint part.

[0040] The interpretation unit 108 interprets the areas to be improved based on the joint area information 211 (S132; interpretation step). In step S132, the interpretation unit 108 calculates the angles of the joint areas searched by the search unit 107 based on each frame stored in the joint area information 211. For example, in the case of knee flexion and extension, as described above, the joint area information 211 includes the frame at the moment when the knee is bent the most. The interpretation unit 108 calculates the angles of the joint areas corresponding to the frame joint area at the moment when the knee is bent the most. The interpretation unit 108 also calculates the position of the body's center of gravity in the frame from which the angles of the joint areas were calculated.

[0041] The interpretation unit 108 performs the processing in step S132 for all pairs of frames and joint identification information 207 included in the joint part information 211. The interpretation unit 108 then outputs the posture feature information 212, which contains the frames processed in step S132, the joint identification information 207, the angles of the joint parts corresponding to the joint identification information 207 calculated in step S132, and the body's center of gravity position, as a set, to the interpretation result output processing unit 109. In this way, the interpretation unit 108 generates the posture feature information 212 as information about the movement of joint parts in the frames extracted by the search unit 107. The interpretation unit 108 also calculates motion scores for the upper body, lower body, etc., based on the calculated joint part angles and center of gravity position, and generates an analysis report that includes advice for the user. This information is included in the posture feature information 212.

[0042] The interpretation result output processing unit 109 outputs the posture feature information 212 to the output unit 110 (S141).

[0043] In steps S112 and S141, the motion evaluation result output processing unit 104 and the interpretation result output processing unit 109 output the motion score 204, the individual joint score information 206, the posture feature information 212, and the user's body image. In this way, the user (for example, an elderly person) can easily recognize the individual joint score information 206 and the posture feature information 212.

[0044] (Example of output screen) Figure 3 shows an example of the joint score information output screen. Refer to Figure 1 as needed. The joint score information output screen 500 shown in Figure 3 consists of a joint score graph 510 and a joint site number display unit 520. As shown in the joint part number display unit 520, a unique number is assigned to each joint part as the joint part number.

[0045] The joint score graph 510 displays the joint scores included in each joint score information 206 output from the selection unit 106. The horizontal axis of the joint score graph 510, line L1, indicates the joint site number. In other words, each graph in the joint score graph 510 corresponds to each joint site number displayed in the joint site number display unit 520. The joint site number is information about the joint site as described above. The joint site number display unit 520 also displays the joint site number along with the user's physical information 521.

[0046] Furthermore, line L1 in the joint score graph 510 indicates that the joint score value is "0". In other words, for each graph shown in the joint score graph 510, if it is above line L1 on the paper, it indicates that the joint score is a positive value. Conversely, if it is below line L1 on the paper, it indicates that the joint score is a negative value. A positive joint score indicates that the corresponding joint area is moving well, while a negative joint score indicates that the corresponding joint area is moving poorly.

[0047] For example, the graph indicated by symbol 511 corresponds to joint number "14" (dashed arrow 531). Referring to the joint number display unit 520, joint number "14" corresponds to the left hip. Incidentally, the human body diagram in the joint number display unit 520 is assumed to be facing forward. Also, the graph indicated by symbol 511 shows a higher value compared to the other graphs. This indicates that the movement of the left hip is good.

[0048] On the other hand, the graph indicated by symbol 512 corresponds to joint number "15" (dashed arrow 532). Referring to the joint number display unit 520, joint number "15" corresponds to the right knee. Furthermore, the graph indicated by symbol 512 shows a lower value compared to the other graphs. This indicates that the movement of the right knee is poor.

[0049] In the example shown in Figure 3, the selection unit 106 passes all joint scores, including those not selected in step S123, to the interpretation result output processing unit 109. When the selection unit 106 passes the selected joint scores to the interpretation result output processing unit 109, the joint score graph 510 displays only the graph of joint scores passed to the interpretation result output processing unit 109.

[0050] Figure 4 shows an example of the motor score and posture characteristic information display screen 600. Refer to Figure 1 as needed. The exercise score and posture characteristic information display screen 600 consists of an exercise score display unit 610, a posture characteristic information display unit 620, and an analysis report display unit 630. The exercise score display unit 610 displays the exercise score 204 calculated by the exercise evaluation unit 103. The posture feature information display unit 620 displays the posture feature information 212 generated by the interpretation unit 108. The analysis report display unit 630 displays the analysis report. The analysis report is the result of the interpretation by the interpretation unit 108 (information generated by the interpretation unit 108) and is the information contained in the posture feature information 212.

[0051] In this embodiment, the posture feature information display unit 620 displays information related to "upper body," "lower body," "trunk," "endurance," and "flexibility." However, it is not limited to these; for example, information related to joint areas such as shoulders, knees, and hands may also be displayed. The items displayed in the posture feature information display unit 620 can be set by the user. The user sets the areas of interest as items to be displayed in the posture feature information display unit 620. The information displayed in the posture feature information display unit 620 is a score calculated based on the state of the joint areas in the interpretation unit 108. Incidentally, endurance is calculated by the amount of time that the same posture can be maintained.

[0052] The interpretation unit 108 calculates the number of points to be displayed on the posture feature information display unit 620 and generates an analysis report to be displayed on the analysis report display unit 630, based on the calculated joint angles, center of gravity position, etc.

[0053] As shown in Figure 3, the interpretation result output processing unit 109 displays at least the joint score along with the user's physical information 521. Also, as shown in the posture feature information display unit 620 in Figure 4, the interpretation result output processing unit 109 outputs the information (posture feature information 212) generated by the interpretation unit 108.

[0054] Amidst the ongoing COVID-19 pandemic since 2020, home-based exercise has increased. This increase in home-based exercise has heightened the need for systems to monitor and support exercise. Existing exercise support technologies have the following limitations. Home-based exercises include forms such as Tai Chi and yoga. (A1) Depending on the movement pattern, it takes time to adjust the accuracy of the evaluation values ​​used to assess movement. (A2) The evaluation results are not easily visible. (A3) Regarding the interpretation of evaluation values, there is little information provided as feedback to the user (elderly person), making it difficult for the user to obtain specific methods for improving their exercise based on the evaluation values.

[0055] In the first embodiment, the evaluation model interpretation unit 105 and selection unit 106 form the first stage, and the search unit 107 and interpretation unit 108 form the second stage, resulting in a two-stage configuration. In the first stage, information is generated to visualize joint areas that exhibit desirable movements (angles) and joint areas that exhibit undesirable movements (angles) in the movement, based on the joint score information 206 for each joint. In the second stage, based on the joint score information 206 for each joint, areas for improvement in the movement are selected, and information for visualizing the areas for improvement is generated.

[0056] According to the first embodiment, first, the evaluation model interpretation unit 105 generates joint score information 206 for each joint. This allows for an evaluation of the user's joints. Subsequently, in a two-stage configuration with the evaluation model interpretation unit 105 and selection unit 106 as the first stage and the search unit 107 and interpretation unit 108 as the second stage, areas for improvement are selected, and the angles of the joints of the areas for improvement are calculated. This makes it possible to provide an exercise support system 1 that takes into account individual differences. Thus, according to the first embodiment, accurate exercise improvement information can be provided to the user.

[0057] Furthermore, in the first embodiment, the processing speed can be improved by having the evaluation model interpretation unit 105 perform processing using the feature map 205.

[0058] [Embodiment 1-2] (System Configuration) Figure 5 is a functional block diagram showing an example of the configuration of the movement support system 1a according to the first and second embodiments. In Figure 5, components similar to those in Figure 1 are denoted by the same reference numerals and their descriptions are omitted. The exercise support system 1a shown in Figure 5 differs from the exercise support system 1 shown in Figure 1 in that it has an elderly medical record 311, which is medical record information containing information about the user's physical condition. The elderly patient medical record 311 stores information about the elderly user's pre-existing medical conditions and other relevant details.

[0059] Figure 6 is a flowchart showing the processing procedure of the exercise support system 1a according to the first and second embodiments. In Figure 6, processes similar to those shown in Figure 2 are given the same step numbers and their explanations are omitted. Also, refer to Figure 5 as appropriate.

[0060] After step S141, the interpretation result output processing unit 109 performs processing for elderly individuals (S151). In step S201, the interpretation result output processing unit 109 extracts information about joint areas with impaired function and joint areas with movement difficulties from the user's (elderly person's) chronic illness information stored in the elderly person's medical record 311. Then, the interpretation result output processing unit 109 visualizes the joint areas with impaired function and joint areas with movement difficulties extracted from the elderly person's chronic illness information in the elderly person's medical record 311 against the posture characteristic information 212. In this way, the interpretation result output processing unit 109 outputs information about the movement limitations of the user's joint areas to the output unit 110. For example, the interpretation result output processing unit 109 indicates the joint areas with impaired function and joint areas with movement difficulties using color. Alternatively, the interpretation result output processing unit 109 suggests other joint areas that should be improved. Other areas that should be improved are the joint areas with the next lowest joint score after the joint areas with problems. In this case, the interpretation result output processing unit 109 should refer to the joint score information 206 for each joint and identify the joint with the next lowest joint score after the joint with the problem.

[0061] For example, suppose the shoulder is suggested as an area for improvement, but the user (an elderly person) has a pre-existing shoulder condition that makes improvement impossible. The exercise support system 1a takes this into account, highlights the shoulder area with a specific color, and then suggests other areas that should be improved.

[0062] Furthermore, in step S151, the interpretation result output processing unit 109 may perform a process to lower the movement priority of joints that are impaired or have movement difficulties.

[0063] The interpretation result output processing unit 109 then outputs the visualized joint area to the output unit 110.

[0064] According to the first and second embodiments, it becomes possible to suggest areas for improvement that take into account the individual physical needs of elderly people and others. This enables exercise support tailored to the individual's physical condition. In particular, it is possible to select areas for improvement that are not strenuous for elderly people. This prevents encouraging elderly people to engage in strenuous exercise. Although the first and second embodiments target elderly people, these embodiments can also be applied to younger people, such as those with knee injuries.

[0065] [Embodiments 1-3] Figure 7 is a functional block diagram showing an example of the configuration of the exercise support system 1b according to the first to third embodiments. In Figure 7, components similar to those in Figure 1 are denoted by the same reference numerals and their descriptions are omitted. The exercise support system 1b shown in Figure 7 differs from the exercise support system 1 shown in Figure 1 in that it has a user input unit 111. The user input unit 111 is a device that allows the user to input information via a touch panel display or the like.

[0066] (flowchart) Figure 8 is a flowchart showing the processing procedure of the exercise support system 1b according to the first to third embodiments. In Figure 8, processes similar to those shown in Figure 2 are given the same step numbers and their explanations are omitted. Also, refer to Figure 7 as appropriate. The interpretation result output processing unit 109 performs user selection processing after the processing in step S141 (S161). In step S161, the interpretation result output processing unit 109 displays joint areas that are moving well, for example, in green, and joint areas that are not moving well, for example, in red. Joint areas that are moving well are joint areas that have a high joint score (above a predetermined value), and joint areas that are not moving well are joint areas that have a low joint score (below a predetermined value). In this way, the interpretation result output processing unit 109 highlights the joint areas in the body image output to the output unit 110 according to the joint score. The body image output to the output unit 110 is the body image that the interpretation result output processing unit 109 outputs to the output unit 110 in step S141 based on the posture feature information 212.

[0067] Furthermore, there may be multiple joint parts output to the output unit 110. Then, in step S161, the user inputs information via the user input unit 111 to set the priority of movement for each joint area for the improvement area output by the selection unit 106. The interpretation result output processing unit 109 can also set the priority order (priority order) of the output joint areas based on the information input via the user input unit 111.

[0068] Alternatively, in step S161, the user inputs information via the user input unit 111 to select joint parts according to the user's preferences. The interpretation result output processing unit 109 selects the output joint parts based on the information input via the user input unit 111.

[0069] Furthermore, in step S161, the user can input information via the user input unit 111 to zoom in on a joint area of ​​interest. Based on this, the interpretation result output processing unit 109 enlarges the image output to the output unit 110 based on the information input via the user input unit 111.

[0070] According to the first to third embodiments, it becomes easier for users to understand the output content, thereby improving usability.

[0071] [Embodiments 1-4] (System Configuration) Figure 9 is a functional block diagram showing an example configuration of the exercise support system 1c according to the first to fourth embodiments. In Figure 9, components similar to those in Figure 1 are denoted by the same reference numerals and their descriptions are omitted. The exercise support system 1c shown in Figure 9 differs from the exercise support system 1 shown in Figure 1 in that it has an output adjustment processing unit 112 that adjusts the output by the interpretation result output processing unit 109.

[0072] (flowchart) Figure 10 is a flowchart showing the processing procedure of the exercise support system 1c according to the first to fourth embodiments. In Figure 10, processes similar to those shown in Figure 2 are given the same step numbers and their explanations are omitted. Refer to Figure 9 as appropriate. The output adjustment processing unit 112 adjusts the output of the improvement area according to the pre-set user information 221 (S171) and outputs it to the output unit 110. For example, if the user is elderly (if the user's age is above a predetermined age), the output adjustment processing unit 112 may enlarge the improvement area as a predetermined region or color the improvement area for emphasis. If the user is young, the output adjustment processing unit 112 may enlarge the area that the user particularly wants to improve (for example, an area to be made slimmer) as a predetermined region, or color the improvement area for emphasis. The areas to be enlarged or colored for emphasis are preferably pre-set. In this way, the output adjustment processing unit 112 emphasizes a predetermined region in the output image.

[0073] Incidentally, in the first to fourth embodiments, the output of the improvement area is adjusted based on the pre-set user information 221. In contrast, the first to third embodiments differ in that the output of the improvement area is adjusted based on the information input via the user input unit 111.

[0074] According to the first to fourth embodiments, even elderly users can easily recognize the areas that need improvement. Furthermore, according to the first to fourth embodiments, usability can be improved even when the user is not elderly.

[0075] [Implementation 2-1] (System Configuration) Figure 11 is a functional block diagram showing an example of the configuration of the exercise support system 1d according to the second-first embodiment. In Figure 11, components similar to those in Figure 1 are denoted by the same reference numerals and their descriptions are omitted. The exercise support system 1d shown in Figure 11 differs from the exercise support system 1 shown in Figure 1 in that it has an exercise recommendation processing unit 121 instead of an interpretation result output processing unit 109. In addition to the functions of the interpretation result output processing unit 109 shown in Figure 1, the exercise recommendation processing unit 121 outputs comparison results with skilled individuals, etc., to the output unit 110.

[0076] (flowchart) Figure 12 is a flowchart showing the processing procedure of the exercise support system 1d according to the second-first embodiment. In Figure 12, processes similar to those shown in Figure 2 are given the same step numbers and their explanations are omitted. Also, refer to Figure 11 as appropriate. The exercise recommendation processing unit 121 generates exercise recommendation information (S201) based on the joint score information 206 obtained from the selection unit 106 and the posture characteristic information 212 obtained from the interpretation unit 108. The exercise recommendation information stores information about methods for improving exercise. The exercise recommendation information is used to highlight areas for improvement with color and display explanatory text based on the joint score information 206 and the posture characteristic information 212. The exercise recommendation processing unit 121 generates recommendation information, which is information about improvement methods, based on the joint score information 206 and the posture characteristic information 212.

[0077] Specifically, the exercise recommendation information is used to color-code and display joint areas with joint scores below a predetermined value and joint areas with joint scores above a predetermined value, based on the 206 joint score data for each joint. Alternatively, the exercise recommendation information is used to output explanatory text indicating that the joint score is low for joint areas with joint scores below a predetermined value. Furthermore, the exercise recommendation information is used to output text praising the user for joint areas with joint scores above a predetermined value.

[0078] The exercise recommendation processing unit 121 outputs the generated exercise recommendation information to the output unit 110 (S202). The user then uses the shooting unit 101 to capture (re-shoot) an improved exercise based on the exercise recommendation information output to the output unit 110 (S101).

[0079] According to the second-first embodiment, the exercise support system 1d can show the user specific methods for improving their exercise. In other words, the exercise support system 1d can recommend exercises to strengthen the area to be improved, based on the area to be improved, the joint score information 206, etc. This allows the user to specifically recognize the area to be improved. The user then re-records the improved exercise for the area indicated by the output unit 110 using the recording unit 101. Based on the re-recorded video, the exercise support system 1d re-executes the process shown in Figure 12. By repeating this process, the user can efficiently improve their exercise.

[0080] [Second-2 Embodiment] (System Configuration) Figure 13 is a functional block diagram showing an example of the configuration of the exercise support system 1e according to the second-second embodiment. In Figure 13, components similar to those in Figure 11 are denoted by the same reference numerals and their descriptions are omitted. The exercise support system 1e shown in Figure 13 differs from the exercise support system 1d shown in Figure 11 in that it has an elderly patient medical record 311. Similar to Figure 5, the elderly patient medical record 311 stores information about the user's pre-existing medical conditions, etc.

[0081] (flowchart) Figure 14 is a flowchart showing the processing procedure of the exercise support system 1e according to the second-second embodiment. In Figure 14, processes similar to those shown in Figure 12 are given the same step numbers and their explanations are omitted. Refer to Figure 13 as appropriate. After processing in step S202, the exercise recommendation processing unit 121 performs an exercise recommendation re-examination process (regeneration) on the exercise recommendation information (S211). In step S601, the exercise recommendation processing unit 121 identifies joint areas that are difficult to move or joint areas that have difficulty moving, based on the elderly person's chronic illness information stored in the elderly person's medical record 311. Then, the exercise recommendation processing unit 121 considers the identified information and presents the risks of exercise or indicates the range of exercise that can be performed. In the first and second embodiments, joint areas that have difficulty moving are indicated by color, or other joint areas that should be improved are presented. In contrast, in the second and second embodiments, the risks of exercise are presented, or the range of exercise that can be performed is indicated. Thus, in the second and second embodiments, the recommendation information is regenerated (re-examined) based on the elderly person's medical record 311, which is medical record information that stores information about the user's physical condition.

[0082] According to the second embodiment, users with pre-existing medical conditions (such as the elderly) can improve their exercise without overexerting themselves.

[0083] [Second-Third Embodiments] (System Configuration) Figure 15 is a functional block diagram showing an example configuration of the exercise support system 1f according to the second-third embodiment. In Figure 15, components similar to those in Figure 11 are denoted by the same reference numerals and their descriptions are omitted. The exercise support system 1f shown in Figure 15 differs from the exercise support system 1d shown in Figure 11 in that it has a target value of 321. The target value 321 stores information about the angle of the joint area that the user is targeting.

[0084] (flowchart) Figure 16 is a flowchart showing the processing procedure of the exercise support system 1f according to the second-third embodiment. In Figure 16, processes similar to those shown in Figure 12 are given the same step numbers and their explanations are omitted. Refer to Figure 15 as appropriate. After processing in step S202, the exercise recommendation processing unit 121 performs target value output processing (S221). In step S701, the exercise recommendation processing unit 121 outputs the result of comparing the current posture (joint angles, etc.) with a preset target value 321. In the second and third embodiments, the target value 321 stores the target value of the joint angle. The current posture is stored in the posture feature information 212. For example, the current angle of the right elbow is 20 degrees, but the target value 321 is 30 degrees. In step S221, the exercise recommendation processing unit 121 outputs the difference between the target value 321 and the current joint value (angle, etc.) to the output unit 110. The exercise recommendation processing unit 121 may also output advice to the output unit 110, such as "raise your right arm by 10°".

[0085] The target value 321 may be output as a numerical value. Alternatively, the target image may be superimposed on the user's image. For example, the current angle of the user's right elbow may be output to the output unit 110, and the target angle of the right elbow may be output to the image of the user's right elbow. In this way, for the improvement area selected by the selection unit 106, both the posture (angle) of the target joint area and the posture (angle) of the user's joint area calculated by the interpretation unit 108 are output. This allows the user to easily compare the target value with the current state. In this manner, the exercise recommendation processing unit 121 outputs to the output unit 110 the comparison result of the joint area stored in the posture feature information 212 between the pre-set target value 321 of the joint area angle.

[0086] According to the second and third embodiments, specific examples of methods for improving exercise are shown to the user. This allows the user to efficiently improve their exercise.

[0087] [Second to Fourth Embodiments] (System Configuration) Figure 17 is a functional block diagram showing an example configuration of the exercise support system 1g according to the second-fourth embodiment. In Figure 17, components similar to those in Figure 11 are denoted by the same reference numerals and their descriptions are omitted. The exercise support system 1g shown in Figure 17 differs from the exercise support system 1d shown in Figure 11 in that it has a grouping processing unit 122 that divides users into groups based on the output of the exercise recommendation processing unit 121.

[0088] (flowchart) Figure 18 is a flowchart showing the processing procedure of the exercise support system 1g according to the second-fourth embodiment. In Figure 18, processes similar to those shown in Figure 12 are given the same step numbers and their explanations are omitted. Also, refer to Figure 17 as appropriate. After processing in step S202, the grouping processing unit 122 groups users based on the exercise recommendation information (S241). In step S241, for example, users who improve the same joint area (have common improvement areas) are grouped together. The exercise support system 1g makes recommendations to each user belonging to the common group, taking into account the common improvement area. In other words, the grouping processing unit 122 groups users with common improvement areas based on the posture characteristic information 212 of multiple users. As a result, exercises for the common improvement area are recommended to users belonging to the group.

[0089] Furthermore, in step S241, the grouping processing unit 122 provides each person divided into groups having the same area to be improved with an exercise program that focuses on improving that area. Alternatively, in step S241, the grouping processing unit 122 provides each person divided into groups having the same area to be improved with an exercise program that starts from that area. Alternatively, in step S241, the grouping processing unit 122 discloses to each person divided into groups having the same area to be improved the current exercise score 204 and joint score of all persons in the group.

[0090] According to the second to fourth embodiments, users with similar areas of improvement, or in other words, users with similar points of improvement, can be grouped together. This allows users belonging to the same group to become aware of each other, thereby promoting exercise improvement.

[0091] Furthermore, the grouping processing unit 122 may notify each user belonging to the group of their progress in improvement. The progress in improvement may include the joint score of the improved area. In this way, users belonging to the same group will become more aware of each other, which can further promote exercise improvement.

[0092] Incidentally, in the second to fourth embodiments, one user may belong to multiple groups.

[0093] [Third Embodiment] (System Configuration) Figure 19 is a functional block diagram showing an example configuration of the exercise support system 1h according to the third embodiment. In Figure 19, components similar to those in Figure 1 are denoted by the same reference numerals and their descriptions are omitted. The exercise support system 1h shown in Figure 19 differs from the exercise support system 1 shown in Figure 1 in that it has a database 331 and an effect analysis unit 131 instead of an interpretation result output processing unit 109. Database 331 stores information on each joint score 206 and postural characteristic information 212. The effectiveness analysis unit 131 analyzes the effects of exercise improvement provided by the exercise support system 1h based on the information stored in the database 331.

[0094] (flowchart) Figure 20 is a flowchart showing the processing procedure of the exercise support system 1h according to the third embodiment. In Figure 20, processes similar to those shown in Figure 2 are given the same step numbers and their explanations are omitted. Also, refer to Figure 19 as appropriate. First, in Figure 20, step S124 in Figure 2 is omitted, and steps S301 and S302 are performed instead of step S141 in Figure 2. In step S301, the joint score information 206 and posture characteristic information 212 are stored in the database 331. Then, the effect analysis unit 131 analyzes the effect of exercise improvement based on the joint score information 206 and posture characteristic information 212 stored in the database 331 (S302). In step S302, the effect analysis unit 131 analyzes, for example, the time it took for the joint score in each improved area to reach 80 points. The effect analysis unit 131 also analyzes, for example, how long it took and how much the arm could be raised, and how close it got to the target value 321 (see Figure 15) in terms of time.

[0095] Thus, in the third embodiment, the long-term improvement trend is analyzed by verifying the improvement effect based on the exercise information accumulated on a daily basis. Furthermore, according to the third embodiment, the effect of improvement can be verified using the joint score information 206 and posture characteristic information 212 stored in the database 331. The designer of the exercise support system 1h can then improve the exercise support system 1h based on the analyzed effect.

[0096] [Hardware configuration] Figure 21 shows the hardware configuration of the exercise support systems 1,1a to 1h. The exercise support systems 1,1a~1h consist of a PC (Personal Computer), smartphone, tablet, etc. Alternatively, the imaging unit 101 to output unit 110, user input unit 111, output adjustment processing unit 112, exercise recommendation processing unit 121, grouping processing unit 122, and effect analysis unit 131 shown in Figures 1, 5, 7, 9, 11, 13, 15, 17, and 19 may all be mounted on a single device. Alternatively, the joint location estimation unit 102 to interpretation result output processing unit 109, user input unit 111, output adjustment processing unit 112, exercise recommendation processing unit 121, grouping processing unit 122, and effect analysis unit 131 may be mounted on a company server, while only the imaging unit 101 and output unit 110 are mounted on a PC, smartphone, or tablet owned by the user.

[0097] Figure 21 shows an example of the hardware configuration of the exercise support systems 1,1a to 1h, including examples of PC and server hardware configurations. The motor support systems 1,1a to 1h include a memory 151, a computing device 152, a storage device 153, an input device 154, an output device 155, and a communication device 156. Memory 151 consists of RAM (Random Access Memory), etc. The computing unit 152 consists of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and the like. The storage device 153 consists of an HD (Hard Disk) or an SSD (Solid State Drive), etc. The storage device 153 stores the trained model 301, the elderly patient's medical record 311, the target value 321, etc., as shown in Figures 1, 5, 7, 9, 11, 13, 15, 17, and 19.

[0098] The input device 154 consists of a keyboard, mouse, etc., and corresponds to the user input unit 111 in Figure 7. The output device 155 consists of a display or the like, and corresponds to the output unit 110 in Figures 1, 5, 7, 9, 11, 13, 15, 17, and 19. The communication device 156 is a device for communicating with other devices.

[0099] Furthermore, the program stored in the storage device 153 is loaded into the memory 151. The loaded program is then executed by the processing unit 152. As a result, the imaging unit 101 to the output unit 110, the user input unit 111, the output adjustment processing unit 112, the exercise recommendation processing unit 121, the grouping processing unit 122, and the effect analysis unit 131 are appropriately implemented.

[0100] [Comparative Example] Next, a comparative example of this embodiment will be described with reference to Figures 22 and 23. Figure 22 shows the configuration of the exercise support system 4a in the comparative example. The exercise support system 4a shown in Figure 22 includes a two-dimensional posture extraction unit 401, an evaluation result output unit 402, and an exercise recommendation processing unit 403. The two-dimensional posture extraction unit 401 acquires video information 201, which is a video of the user exercising, and sample video information 411, which is a video of a sample exercise. The two-dimensional posture extraction unit 401 then compares the video information 201 and the sample video information 411 to generate and output an evaluation model 412 for the user's overall movement.

[0101] Next, the evaluation result output unit 402 outputs the evaluation model 412, and the exercise recommendation processing unit 403 presents the user with areas for improvement based on a comparison of the video information 201 and the sample video information 411. The user then provides feedback by recording an exercise video while paying attention to the areas for improvement presented by the exercise recommendation processing unit 403.

[0102] The exercise support system 4a shown in Figure 22 evaluates the overall movement and suggests areas for improvement, but it does not evaluate the movement of individual joints. Therefore, users (especially the elderly) can only make vague improvements to their movements.

[0103] Figure 23 shows the configuration of the exercise support system 4b in a comparative example different from Figure 22. The motor support system 4b shown in Figure 23 includes a feature extraction unit 451, an evaluation unit 452, an XAI 453, an evaluation result output unit 454, and a trained model 461. In the exercise support system 4b shown in Figure 23, first, the feature extraction unit 451 extracts the characteristics of the exercises performed by the user from the video information 201, which is a video of the user exercising.

[0104] On the other hand, the trained model 461 stores the results of training based on the example video information 411, which is a video of an example movement. Then, the evaluation unit 452 evaluates the motion based on the trained model 461 and the motion features extracted from the feature extraction unit 451, and outputs the evaluation result. Furthermore, XAI453 analyzes the evaluation performed by the evaluation unit 452, determining which joint areas were used as the basis for the evaluation, or in other words, which joint areas contributed most significantly to the evaluation.

[0105] Then, the evaluation result output unit 454 outputs the evaluation result from the evaluation unit 452 and the result from the XAI 453. In this way, the user can obtain an interpretation of the evaluation of the movement.

[0106] In the exercise support system 4b shown in Figure 23, the XAI453 allows the user to recognize which parts of the movement influenced the evaluation. However, the exercise support system 4b shown in Figure 23 only provides a general interpretation.

[0107] The exercise support systems 1,1a to 1h described in this embodiment are applicable to elderly care systems, rehabilitation support systems, home exercise service systems, remote exercise support systems, etc. Furthermore, the exercise support systems 1,1a to 1h described in this embodiment can utilize a big data analysis system.

[0108] The present invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above are described in detail to make the present invention easier to understand, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace parts of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add configurations from other embodiments to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations.

[0109] While the evaluation model interpretation unit 105 uses XAI technology, it is not limited to this. Other technologies such as SHAP (Shapley Additive exPlanation) or Shapley value estimation may also be used in the evaluation model interpretation unit 105.

[0110] Furthermore, each of the above-mentioned configurations and functions can be implemented in hardware, for example, by designing some or all of them using integrated circuits. In other words, the imaging unit 101 to the output unit 110, the user input unit 111, the output adjustment processing unit 112, the motion recommendation processing unit 121, the grouping processing unit 122, and the effect analysis unit 131 can all be implemented in hardware, for example, by designing some or all of them using integrated circuits. In addition, the database 331, etc., can also be implemented in hardware, for example, by designing some or all of them using integrated circuits. Moreover, as shown in Figure 21, each of the above-mentioned configurations and functions may also be implemented in software by having a processor such as a CPU interpret and execute a program that implements each function. Information such as programs, tables, and files that implement each function can be stored not only on the HD (Hard Disk), but also on memory 151, recording devices such as SSDs (Solid State Drives), or recording media such as IC (Integrated Circuit) cards, SD (Secure Digital) cards, and DVDs (Digital Versatile Discs).

[0111] Furthermore, in each embodiment, only those control lines and information lines deemed necessary for explanation are shown, and not all control lines and information lines are necessarily shown in the actual product. In practice, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0112] 1,1a~1h Exercise support system 101 Photography Department 102 Joint site estimation area 103 Exercise Evaluation Department 104. Motor Evaluation Result Output Processing Unit (Output Processing Unit) 105 Evaluation Model Interpretation Unit (Joint Score Calculation Unit) 106 Selection Department 107 Search Department 108 Interpretation Section 109 Interpretation Result Output Processing Unit (Output Processing Unit) 110 Output section 111 User Input Section (Input Section) 112 Output adjustment processing unit 121 Exercise Recommendation Processing Unit 122 Grouping Processing Unit 131 Effectiveness Analysis Department 201 Video Information 202 Joint Information 203 Weight Information 204. Exercise Score 205 Feature Map (Contribution) 206 Information on each joint score (joint score) 207 Joint Identification Information 210 Joint Time-Series Information (Time-Series Information) 211 Joint Location Information 212 Postural characteristic information (information on joint movement) 221 User Information 301 Pre-trained models 311 Elderly Medical Records (Medical Record Information) 321 Target Value 331 Database 500 Joint Score Information Output Screen 510 Joint Score Graph 520 Joint part number display section 521 Physical information 600 Exercise Score / Posture Characteristics Information Display Screen 610 Exercise score display unit 620 Posture characteristic information display unit 630 Analysis Report Display Section S111 Motor evaluation process (motor evaluation step) S121 Feature map generation (motor evaluation step) S122 Generation of joint score information (joint score calculation step) S123 Selection of areas to be improved (selection step) S131 Search for areas to improve (search step) S132 Interpretation of the improved area (interpretation step)

Claims

1. A movement evaluation unit calculates a movement score by evaluating the user's movement based on video information related to the user's movement, and outputs a feature map which is a contribution that shows which joint parts were given emphasis when calculating the movement score. Based on the aforementioned feature map, the joint score calculation unit calculates a joint score, which is a score of movement at the joint site. A selection unit that selects the joint site based on the joint score, A search unit extracts the frame at the moment of greatest movement in the selected joint area from the frames constituting the time-series information, based on time-series information consisting of multiple frames relating to joint movement and information about the selected joint area. An interpretation unit that generates information regarding the movement of joint parts in the frame extracted by the search unit, An output processing unit that outputs the aforementioned motor score, the aforementioned joint score, and the information generated by the interpretation unit to an output unit, It has, The output processing unit, Based on the medical record information containing information about the user's physical condition, the system indicates the joints that are impaired and / or have difficulty moving using different colors. The search unit, For the selected joint area, the angle of the joint area at the moment the movement begins is used as the initial value, and the frame at the moment when the angle of the joint area is the largest value from the initial value is extracted as the frame at the moment of the greatest movement. An exercise support system characterized by the following features.

2. An output processing unit that displays at least the joint score along with physical information and outputs the information generated by the interpretation unit. The exercise support system according to claim 1, characterized by having the following features.

3. The output processing unit, The user's body image is output to the output unit. In the body image output to the output unit, the joint area is assigned to the joint score. Emphasize accordingly. The exercise support system according to feature 1.

4. The output processing unit, The user's body image is output to the output unit. Multiple joint parts are output to the output unit. Based on the information input via the input unit, the priority order of the output joint parts is set. The exercise support system according to feature 3.

5. The output processing unit, The user's body image is output to the output unit. Multiple joint parts are output to the output unit. The output processing unit, Based on the information input via the input unit, the joint portion to be output is selected. The exercise support system according to feature 3.

6. The output processing unit Based on the information input via the input unit, the image output to the output unit is enlarged. The exercise support system according to feature 1.

7. An output adjustment processing unit enhances a predetermined area of ​​the output image. The exercise support system according to claim 2, characterized by having the following features.

8. Based on the joint score and the information regarding the movement of the joint area, the exercise recommendation processing unit generates recommendation information, which is information regarding improvement methods, and outputs the recommendation information to the output unit. The exercise support system according to claim 1, characterized by having the following features.

9. The exercise recommendation processing unit is: Based on the medical record information containing information about the user's physical condition, the recommendation information is regenerated. The exercise support system according to feature 8.

10. The exercise recommendation processing unit is: The output unit outputs a comparison result of the joint part stored in the information regarding the movement of the joint part, between the target value of the joint part's angle, which is set in advance, to the output unit. The exercise support system according to feature 8.

11. The exercise recommendation processing unit is: Based on the aforementioned recommendation information, the users are grouped. The exercise support system according to feature 8.

12. The aforementioned joint score and a database in which information regarding the movement of the aforementioned joint site is stored, Based on the information stored in the aforementioned database, an effectiveness analysis unit conducts an analysis on the improvement effects of exercise, The exercise support system according to claim 1, characterized by having the following features.

13. An exercise support system that assists users in their exercise, Based on the video information regarding the user's exercise, an exercise score is calculated by evaluating the exercise. Furthermore, the motor evaluation step outputs a feature map, which is a contribution that indicates which joint area was given emphasis when calculating the motor score. A joint score calculation step, which calculates a joint score, which is a score of movement at a joint site, based on the aforementioned feature map, A selection step in which the joint site is selected based on the joint score, A search step of extracting the frame at the moment when the movement of the selected joint is greatest from the frames constituting the time-series information, based on time-series information consisting of multiple frames relating to joint movement and information relating to the selected joint. An interpretation step that generates information regarding the movement of joint parts in the frame extracted in the search step, An output step that outputs the aforementioned motor score, the aforementioned joint score, and the information generated in the interpretation step to an output unit, Execute, In the output step described above, Based on the medical record information containing information about the user's physical condition, the joints that are impaired and / or have difficulty moving are indicated by color. In the aforementioned search step, For the selected joint area, the angle of the joint area at the moment the movement begins is used as the initial value, and the frame at the moment when the angle of the joint area is the largest value from the initial value is extracted as the frame at the moment of the greatest movement. A method for supporting exercise characterized by the following features.

Images