Physical Exercise Support System

Abstract

To easily evaluate the body movement.SOLUTION: A body movement support server for supporting the improvement of the effect of the body movement performed by a user using a tool comprises: a reference value storage unit which stores a reference value related to a position of at least one tool; a portion specification unit which specifies a portion of the tool by analyzing an image obtained by imaging the tool; and an evaluation unit which determines an evaluation value of the movement on the basis of the position of the portion in the image and the reference value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a physical exercise support system. [Background technology]

[0002] There is a wide range of physical exercises that use tools (including devices). For example, in ball games, which are one category of sports, the tool is the ball, and depending on the sport, the ball is handled with tools that are mainly held or attached to the hands, feet, or body. In addition, the use of tools used in sports, training, and rehabilitation in the practice of such sports, as well as in strength training and rehabilitation, is widespread.

[0003] Here, attention is focused on training machines. An invention has been disclosed that detects the behavior of a training machine and measures the amount of exercise performed when a user uses the machine to train (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-198867 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology of Patent Document 1 requires that a measuring device be attached to the training machine itself, which poses problems in terms of convenience and safety. Also, depending on the type of tool and the method of using the tool, it may be difficult to attach a measuring device to the tool.

[0006] The present invention has been made in consideration of the above background, and aims to provide technology that can support exercise by easily detecting and evaluating the movement of tools used in physical exercise, and giving advice aimed at further improving skills and improving the effectiveness of training, rehabilitation, etc. [Means for solving the problem]

[0007] The present invention is a physical exercise support system that supports a user in improving the effectiveness of physical exercise performed using a tool, and is characterized by comprising a reference value memory unit that stores a reference value relating to the position of at least one of the tools, a part identification unit that analyzes an image of the tool to identify a part of the tool, and an evaluation unit that compares the position of the part in the image with the reference value to determine an evaluation value of the exercise.

[0008] Other problems and solutions disclosed in this application will be made clear in the section on preferred embodiments of the invention and the drawings. Effect of the Invention

[0009] According to the present invention, it is possible to easily evaluate the skills and effects of physical exercise using tools and provide support that contributes to improving skills and effects. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of a physical exercise support system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal 10. [Diagram 3] FIG. 2 is a diagram illustrating an example of the software configuration of the user terminal 10. [Figure 4] 4 is a diagram showing an example of the configuration of physical information stored in a physical information storage unit 130. FIG. [Diagram 5] 13 is a diagram showing an example of the configuration of a rating request that a rating request sending unit 112 sends to a server device 20. FIG. [Figure 6]10 is a diagram showing an example of the structure of evaluation information received by an evaluation information receiving unit 113 from a server device 20. FIG. [Figure 7] 13 is a diagram showing an example of the configuration of a remedy request that a remedy request sending unit 116 sends to a server device 20. FIG. [Figure 8] FIG. 13 is a diagram illustrating an example of a configuration of improvement measure information. [Figure 9] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server device 20. [Figure 10] FIG. 2 is a diagram illustrating an example of the software configuration of the server device 20. [Figure 11] 13 is a diagram showing an example of the configuration of image information stored in an image data storage unit 231. FIG. [Figure 12] 13 is a diagram showing an example of the configuration of reference information stored in a reference information storage unit 232. FIG. [Figure 13] 13 is a diagram showing an example of the configuration of evaluation condition information stored in an evaluation condition information storage unit 233. FIG. [Figure 14] 13 is a diagram showing an example of the configuration of remediation information stored in a remediation information storage unit 234. FIG. [Figure 15] 13 is a diagram showing an example of the configuration of group analysis information stored in a group analysis information storage unit 235. FIG. [Figure 16] FIG. 2 is a diagram showing a flow of processing executed in the physical exercise support server of the present embodiment. [Figure 17] FIG. 13 is a diagram showing an example of a video evaluation screen in a weightlifting mode. [Figure 18] FIG. 13 is another diagram showing an example of the video evaluation screen in the weightlifting mode. [Figure 19] FIG. 13 is another diagram showing an example of the video evaluation screen in the weightlifting mode. [Figure 20] FIG. 13 is another diagram showing an example of the video evaluation screen in the weightlifting mode. [Figure 21] FIG. 13 is another diagram showing an example of the video evaluation screen in the weightlifting mode. [Figure 22]13 is another diagram showing an example of the video evaluation screen in the squat mode presented on the user terminal 10. FIG. [Figure 23] 13 is a diagram showing an example of an evaluation screen for a video in group training, which is presented on a supporter terminal 50. FIG. [Figure 24] 13 is another diagram showing an example of the evaluation screen for a video in group training, which is presented on supporter terminal 50. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The contents of the embodiment of the present invention will be described below. A physical exercise support server according to the embodiment of the present invention has the following configuration.

[0012] [Item 1] A physical exercise support system that supports a user in improving the effectiveness of physical exercise performed using a tool, A reference value storage function for storing a reference value relating to at least one of the positions of the tool; A part identification function that identifies a part of the tool by analyzing an image of the tool; an evaluation function for determining an evaluation value of the movement based on the position of the portion in the image and the reference value; A physical exercise support system comprising: [Item 2] The evaluation function determines the evaluation value according to a distance from the reference value to the position; 2. The physical exercise support system according to claim 1, [Item 3] The evaluation function determines an evaluation value of the exercise based on a relationship between a part of the tool and a part of the body; 3. The physical exercise support system according to claim 1 or 2, [Item 4] a comment storage function for storing a comment on a position of the part in association with the part and a condition for the part; a comment output function for outputting the comment corresponding to the condition satisfied by the position; 4. The physical exercise support system according to any one of items 1 to 3, [Item 5] an improvement information storage function that stores advice on an improvement measure related to the physical exercise in association with a goal of improvement related to the exercise and a condition for the position; an advice output function for outputting the advice corresponding to the condition satisfied by the position; 5. The physical exercise support system according to any one of items 1 to 4, [Item 6] A physical exercise support server that supports a user in improving the effectiveness of physical exercise performed using a tool, A reference value storage unit that stores a reference value related to at least one of the tool positions; A part identification unit that analyzes an image of the tool and identifies a part of the tool; an evaluation unit for determining an evaluation value of the movement by comparing a position of the part in the image with the reference value; A physical exercise support server comprising: [Item 7] A physical exercise support method for supporting a user in improving the effectiveness of physical exercise performed using a tool, comprising: A reference value storage step of storing a reference value relating to at least one of the tool positions; A part identification step of analyzing an image of the tool to identify a part of the tool; an evaluation step for determining an evaluation value of the movement by comparing the position of the portion in the image and the reference value; A physical exercise support method comprising:

[0013] A physical exercise support server according to one embodiment of the present invention is intended to evaluate the movement of a tool when a user performs physical exercise using the tool.

[0014] Physical exercises using the user's tools include, but are not limited to, sports such as soccer, volleyball, and basketball, which use only a ball as the tool; sports such as baseball, tennis, table tennis, and golf, which use a ball and other tools; sports such as kendo and fencing, which use tools other than a ball; physical exercises using tools such as juggling; physical exercises using barbells or training equipment or machines; various types of rehabilitation using tools; movements of the person being cared for using tools such as a cane; and playing musical instruments.

[0015] The physical exercise support server of this embodiment identifies the tool and parts of the tool from an image (which may be a still image or a moving image, but in this embodiment it is a moving image) captured of a user exercising with a tool, and evaluates the tool's movement based on the absolute position of the part and the relative positional relationship between a plurality of different parts, etc. Note that a part of the body may be identified, and the tool's movement may be evaluated based on the relative positional relationship between the tool part and the body part, etc.

[0016] 1 is a diagram showing an example of the overall configuration of a physical exercise support server according to this embodiment. As shown in the figure, the physical exercise support server of this embodiment is configured to include a user terminal 10 and a server device 20. It may further include an imaging terminal 30. The user terminal 10, the server device 20, and the imaging terminal 30 are communicably connected to each other via a communication network 40. The communication network 40 is, for example, the Internet or a LAN (Local Area Network), and is constructed by a public telephone line network, a dedicated telephone line network, a mobile phone line network, Ethernet (registered trademark), a wireless communication path, etc.

[0017] ==User terminal 10== The user terminal 10 is a computer operated by a user performing physical exercise or by his / her supporter. The user terminal 10 is, for example, a smartphone, a tablet computer, a personal computer, or the like. The user terminal 10 is equipped with an imaging device such as a camera, which can capture images of the user's body during exercise. In this embodiment, a video image of the user's body during exercise is transmitted from the user terminal 10 to the server device 20. The user terminal 10 may also function as the imaging terminal 30. Although only one user terminal 10 is shown in FIG. 1, it goes without saying that there may be multiple user terminals 10.

[0018] ==Server Apparatus 20== The server device 20 is a computer that evaluates the movement of the tool. The server device 20 is, for example, a workstation, a personal computer, or a virtual computer logically realized by cloud computing. The server device 20 receives the video captured by the user terminal 10, analyzes the received video, and evaluates the physical movement. The server device 20 also makes suggestions regarding measures to improve the physical movement. The details of the evaluation of the physical movement and the suggestions of measures to improve the physical movement will be described later.

[0019] ==Imaging terminal 30== The imaging terminal 30 is a computer that captures an image of a user, or a camera with a communication function, which is installed or attached at a place where the user performs physical exercise. The imaging terminal 30 is, for example, a smartphone, a tablet computer, a personal computer, or the like. The imaging terminal 30 is equipped with an imaging device such as a camera, and can capture an image of the user's body during exercise. In this embodiment, a moving image of the user's body during exercise is transmitted from the imaging terminal 30 to the server device 20. Note that the data of the image of the user stored in the imaging terminal 30 may be input directly to the server device 20 by the user or his / her supporter, or a business operator who uses a physical exercise support server to perform a business. Alternatively, the data may be input via a network.

[0020] ==Supporter Terminal 50== The supporter terminal 50 is a computer operated by a person (supporter) who is responsible for providing guidance, instruction, explanation, support, etc. to a user performing physical exercise, such as a trainer, a physical therapist, a caregiver, etc. The supporter terminal 50 is, for example, a smartphone, a tablet computer, a personal computer, etc.

[0021] FIG. 2 is a diagram showing an example of a hardware configuration of the user terminal 10. The user terminal 10 includes a CPU 101, a memory 102, a storage device 103, a communication interface 104, a touch panel display 105, and a camera 106. The storage device 103 is, for example, a hard disk drive, a solid state drive, or a flash memory that stores various data and programs. The communication interface 104 is an interface for connecting to the communication network 40, and is, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone line network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector or an RS232C connector for serial communication, or the like. The touch panel display 105 is a device for inputting and outputting data. The user terminal 10 may further include input devices such as a keyboard, a mouse, buttons, and a microphone, and output devices such as a speaker and a printer.

[0022] 3 is a diagram showing an example of the software configuration of the user terminal 10. The user terminal 10 includes functional units, namely, an imaging unit 111, an evaluation request transmitting unit 112, an evaluation information receiving unit 113, an evaluation display unit 114, a checkpoint display unit 115, an improvement request transmitting unit 116, an improvement information receiving unit 117, and an improvement information display unit 118, as well as storage units, namely, a physical information storage unit 130, an image storage unit 131, an evaluation information storage unit 132, and an improvement storage unit 133.

[0023] Each of the above functional units is realized by the CPU 101 of the user terminal 10 reading out a program stored in the storage device 103 into the memory 102 and executing it, and each of the above storage units is realized as part of the storage area provided by the memory 102 and storage device 103 of the user terminal 10.

[0024] The imaging unit 111 captures images, including video, of the user performing physical exercise using the tool. The imaging unit 111 can obtain video of the physical exercise using the tool by controlling the camera 106. The user or the user's supporter installs the user terminal 10 on a flat surface or a wall, points the optical axis of the camera 106 toward the place where the user performs exercise, and issues an instruction to start video shooting. In response to this, the imaging unit 111 operates the camera 106 to obtain the video. The imaging unit 111 stores the obtained video in the image storage unit 131.

[0025] The image storage unit 131 stores images captured by the imaging unit 111. In this embodiment, the images are moving images, but are not limited to this. The image storage unit 131 can store the moving images as files, for example.

[0026] The physical information storage unit 130 stores information (hereinafter referred to as physical information) related to the user's body, physical ability, factors that affect training effects, etc. Fig. 4 is a diagram showing an example of the configuration of physical information stored in the physical information storage unit 130. As shown in the figure, the physical information includes height, weight, sex, dominant hand, arm length, leg length, hand size, finger length, grip strength, muscle strength, flexibility, shoulder strength, genome, epigenome, gene polymorphism, intestinal flora, diet, etc.

[0027] The evaluation request sending unit 112 sends to the server device 20 a request to evaluate the physical exercise using the tool (hereinafter, referred to as an evaluation request) based on the image captured by the imaging unit 111.

[0028] FIG. 5 is a diagram showing an example of the configuration of an evaluation request transmitted by evaluation request transmission unit 112 to server device 20. As shown in the figure, the evaluation request includes a user ID, a mode, physical information, and image data. The user ID is information for identifying the user. The mode is information indicating the exercise performed by the user. The mode can be, for example, "bench press", "tennis serve", "walking rehabilitation", etc. It should be noted that the mode is selected from a predetermined selection. The physical information is physical information stored in physical information storage unit 130. The image data is data of a moving image acquired by imaging unit 111.

[0029] The evaluation information receiving unit 113 receives information regarding the evaluation of the physical exercise using the tool (hereinafter referred to as evaluation information) that is returned from the server device 20 in response to the evaluation request. The evaluation information receiving unit 113 registers the received evaluation information in the evaluation information storage unit 132.

[0030] 6 is a diagram showing an example of the configuration of evaluation information received by the evaluation information receiving unit 113 from the server device 20. As shown in the figure, the evaluation information includes a mode, a user ID, tool position information, body part position information, posture information, movement information, and checkpoint information.

[0031] The user ID and mode are the user ID and mode included in the evaluation request. The captured image shows the user's body performing the exercise indicated by the mode.

[0032] The tool position information indicates the position of each part of the tool (for example, the entire bat used in baseball, both ends, the grip, the place where the ball is met, the center of gravity, any other part, etc., as well as the entire barbell, the shaft, the plate (weight part), the gripped part, the center of the shaft, the center of gravity, any other part, etc., including the entire tool) in the image. The tool position information includes each part of the tool and the position of the part in association with a time point on the time axis of the video. Based on the tool position information, the movement of the tool and the relationship with the body part can be displayed. That is, for example, a figure indicating a part (for example, a circle, etc.) can be displayed superimposed on the image at the position indicated by the tool position information. Note that the positions of multiple parts may be included for one time point. Furthermore, the tool position information may not be included for a part connecting two parts (for example, the midpoint between the parts where the barbell is gripped by the right hand and the left hand). In this case, the part connecting these two parts can be expressed by connecting a pair of marks (for example, a circle, etc.) indicating two predetermined parts with a line. The tool part position information may be included for each frame constituting the video, for each key frame (including frames related to checkpoints, which will be explained later), for every arbitrary number of frames, or for a random point in time. If My Frame does not include position information, the figure can be displayed based on the position information of the closest past point in time.

[0033] The body position information indicates the position of each part of the body (for example, head, shoulders, elbows, waist, knees, ankles, etc.) in the image. The body position information includes parts and the positions of the parts in association with the time points on the time axis of the video. The state of the body skeleton (bones) can be displayed based on the body position information. That is, for example, a figure (for example, a circle, etc.) indicating the part can be displayed by superimposing it on the image at the position indicated by the body position information. Note that the positions of multiple parts can be included for one time point. Note that position information does not need to be included for parts connecting two parts (for example, the forearm connecting the wrist and elbow, or the thigh connecting the waist and knee, etc.). In this case, the part connecting these two parts can be expressed by connecting a pair of marks (for example, a circle, etc.) indicating two predetermined parts with a line. The position information may be included for each frame constituting the video, may be included for each key frame (including frames related to checkpoints described later), may be included for every arbitrary number of frames, or may be included for a random time point. If My Frame does not include position information, bones can be displayed based on position information of the most recent past time point.

[0034] The tool orientation information is information related to the orientation of the tool used by the user and the direction in which a part of the tool is facing. The tool orientation information includes the part of the tool to be evaluated, the tool movement value, the evaluation rank, the evaluation comment, etc., in association with a time point on the time axis of the video. The tool orientation value is a value that indicates the orientation of the tool. The tool orientation value is, for example, the distance from the ground to a certain part of the tool, the distance between two parts of the tool, the angle of the part (the angle formed by a first part of the tool and a straight line to, for example, the part where the user is holding the tool, and a second part of the tool and a straight line to, for example, the part where the user is holding the tool), the movement of a certain part of the tool, etc. The evaluation rank is a value that expresses the evaluation value by a rank. The evaluation rank is expressed, for example, from 1 to 5 on a 5-point scale, ABC, etc. The evaluation comment is a comment related to the evaluation of the posture. For example, when the mode is "upright row" and the distance from the ground to the right end and the left end of the barbell is different, an evaluation comment such as "Different forces are being applied to the left and right" may be included.

[0035] The tool movement information is information related to the movement of the tool used by the user. The tool movement information includes, in association with a period on the time axis of the video, a part of the tool to be evaluated, a list of the tool orientation values, an evaluation rank, and an evaluation comment. The list of tool orientation values ​​is a chronological series of tool orientation values ​​within the period. The evaluation comment is a comment related to the evaluation of the tool movement. For example, when the mode is "upright row" and the up and down movement of the barbell is insufficient, an evaluation comment such as "the barbell is not lifted sufficiently" may be included.

[0036] The posture information is information related to the posture of the user's body. The posture information includes parts to be evaluated, posture values, evaluation ranks, and evaluation comments, which are associated with points on the time axis of the video. The posture value is a value that represents the posture. The posture value is, for example, the distance from the ground to a part, the distance between two parts, the angle of the joint part (the angle formed by the straight line from the first end part to the joint part and the straight line from the second end part to the joint part), etc. The evaluation rank is a value that expresses the evaluation value by rank. The evaluation rank is expressed, for example, from 1 to 5 on a 5-point scale, or A, B, C, etc. The evaluation comment is a comment related to the evaluation of the posture. For example, if the mode is "lifting" and bending is insufficient, an evaluation comment such as "your hips are not lowered" may be included. (Flexibility such as the range of motion of the joints, numerical values ​​(expressed as angles and distances) → also affect (confirmation) advice)

[0037] The body movement information is information related to the user's body movement. The body movement information includes parts to be evaluated, a list of posture values, an evaluation rank, and an evaluation comment, all of which are associated with a period on the time axis of the video. The list of posture values ​​is a time series of posture values ​​within a period. The evaluation comment is a comment related to the evaluation of the movement. For example, when the mode is "lifting" and the knee extension is not smooth, an evaluation comment such as "the knee movement is not smooth" may be included.

[0038] The relationship information indicates one or more pieces of positional relationship information between the tool and the body. The relationship information includes information on the position, orientation, and movement of the tool, etc., and information on the relationship between the body part, posture, and movement, etc., in association with a time point on the time axis of the video. For example, based on the tool position information and the body position information, it is possible to display what kind of positional relationship there is between the two. That is, for example, a figure indicating a part (e.g., a circle, etc.) can be superimposed on the image at the position indicated by the tool position information, and a figure indicating a part (e.g., a circle, etc.) can be displayed at the position indicated by the body position information. Note that the positions of multiple parts and parts can be included for one time point. Note that position information may not be included for a point connecting parts and parts (e.g., the tip of the bat and the center point of the part where the user is holding the bat, etc.). In this case, the point connecting these two parts and parts can be expressed by connecting a pair of figures (e.g., a circle, etc.) indicating two specified parts and parts with a line. The relationship information may be included for each frame constituting the video, for each key frame, for each check point (details will be described later), for every arbitrary number of frames, or for a random point in time. If My Frame does not include location information, it is possible to display figures showing parts or regions and lines connecting them based on location information from the most recent past point in time.

[0039] The checkpoint information is information indicating a point (hereinafter referred to as a checkpoint) where the orientation of the tool or the posture of the body should be checked in the movement of the tool used by the user or in a series of movements of the user's body. For example, in the case of "weightlifting" mode, checkpoints include the point where the barbell reaches its highest position, the point where it reaches its lowest position, and the moment it is lifted. In the case of "pitching" mode, checkpoints include the point where the foot is raised, the point where the raised foot is lowered and the weight is transferred, and the point where the ball is released. The checkpoint information stores information indicating the checkpoint (hereinafter referred to as a checkpoint ID) in association with a time point on the time axis of the video. In other words, it is possible to identify the frame (still image) in the video in which the checkpoint indicated by the checkpoint ID is displayed.

[0040] The evaluation display unit 114 displays the evaluation information. For example, the evaluation display unit 114 can display a figure representing a part of the tool or a part of the body (for example, a circle representing the end of the tool or the center of gravity of the body, and a line connecting them, etc.) superimposed on the video based on the tool position, tool orientation, tool movement information, body position, posture, body movement information, etc. included in the evaluation information, thereby superimposing the part of the tool or the movement of the part of the body on the video. In addition, the evaluation display unit 114 can display, for example, a graph of the position of the part of the tool, the orientation of the tool, the movement of the tool, and the position, posture, movement of the body, etc. over time.

[0041] Furthermore, the evaluation display unit 114 can display an evaluation rank and an evaluation comment together with the display of the video based on the tool orientation information and the tool movement information included in the evaluation information. For example, the evaluation display unit 114 can display an evaluation rank and an evaluation comment included in the tool orientation information when the playback time of the video reaches a point around the time point included in the tool orientation information (for example, it can be any length such as around 5 seconds). Furthermore, the evaluation display unit 114 can display an evaluation rank and an evaluation comment included in the tool movement information when the playback time of the video reaches a period included in the tool movement information. Furthermore, the evaluation display unit 114 can display an orientation value included in the orientation information. Furthermore, the evaluation display unit 114 can display a graph of the time-series conversion of the tool orientation value based on a list of the tool orientation values ​​included in the tool movement information.

[0042] Furthermore, the evaluation display unit 114 can display an evaluation rank and an evaluation comment in accordance with the display of the video based on the posture information and the movement information included in the evaluation information. For example, the evaluation display unit 114 can display an evaluation rank and an evaluation comment included in the posture information when the playback time of the video reaches a point around the time point included in the posture information (for example, it can be an arbitrary length such as around 5 seconds). Furthermore, the evaluation display unit 114 can display an evaluation rank and an evaluation comment included in the movement information when the playback time of the video reaches a period included in the movement information. Furthermore, the evaluation display unit 114 can display a posture value included in the posture information. Furthermore, the evaluation display unit 114 can display a graph of the time-series conversion of the posture value based on a list of posture values ​​included in the movement information. The evaluation rank and the evaluation comment may be displayed together with the evaluation rank and the evaluation comment displayed in accordance with the display of the video based on the tool orientation information and the tool movement information described in the previous paragraph.

[0043] The checkpoint display unit 115 can extract and display images of checkpoints from the video. The checkpoint display unit 115 can read frames corresponding to time points included in the checkpoint information from the image data of the video stored in the image storage unit 131 and display them as still images. Furthermore, the checkpoint display unit 115 may, for example, extract and display only body parts from the read frames.

[0044] The improvement measure request sending unit 116 sends a request for acquiring improvement measures related to tool handling and physical exercise (hereinafter referred to as an improvement measure request) to the server device 20. FIG. 7 is a diagram showing an example of the configuration of an improvement measure request. As shown in the figure, the improvement measure request includes a user ID, a mode, and a purpose. The purpose is the purpose for which the user makes an improvement. Examples of the purpose may be "increase the ball speed," "increase muscle strength," "stabilize the lower body," and the like. The purpose is also selected from a predetermined selection.

[0045] The improvement measure information receiving unit 117 receives information on the improvement measure transmitted from the server device 20 in response to the improvement measure request (hereinafter, referred to as improvement measure information). The improvement measure information receiving unit 117 stores the received improvement measure information in the improvement measure storage unit 133. An example of the configuration of the improvement measure information is shown in FIG. 8. As shown in the figure, the improvement measure information includes a purpose, advice, and standard information. In this embodiment, the advice is assumed to be a character string representing the improvement measure, but it may be content that presents the improvement measure using an image or video. The standard information is a suitable orientation and movement of the tool (position, orientation, movement, speed, etc. of each part) and a body posture (position, angle, etc. of each part). The improvement measure information receiving unit 117 may receive the improvement measure information transmitted by the improvement measure information transmitting unit 216 based on the evaluation result and the standard value even without an improvement measure request.

[0046] The improvement information display unit 118 displays the improvement measures. The improvement information display unit 118 displays advice included in the improvement information. In addition, when the improvement information includes a suitable position or angle for a part,

[0047] 9 is a diagram showing an example of the hardware configuration of the server device 20. The server device 20 includes a CPU 201, a memory 202, a storage device 203, a communication interface 204, an input device 205, and an output device 206. The storage device 203 is, for example, a hard disk drive, a solid state drive, or a flash memory that stores various data and programs. The communication interface 204 is an interface for connecting to the communication network 40, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone line network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector or an RS232C connector for serial communication, etc. The input device 205 is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, etc. that input data. The output device 206 is, for example, a display, a printer, a speaker, etc. that output data.

[0048] 10 is a diagram showing an example of the software configuration of the server device 20. As shown in the diagram, the server device 20 includes functional units, namely, an evaluation request receiving unit 211, an image analysis unit 212, an evaluation unit 213, an evaluation information transmitting unit 214, an improvement request receiving unit 215, and an improvement information transmitting unit 216, and storage units, namely, an image data storage unit 231, a reference information storage unit 232, an evaluation condition information storage unit 233, and an improvement information storage unit 234.

[0049] Each of the above-mentioned functional units is realized by the CPU 201 of the server device 20 reading out a program stored in the storage device 203 into the memory 202 and executing it, and each of the above-mentioned storage units is realized as part of the storage area provided by the memory 202 and storage device 203 of the server device 20.

[0050] The rating request receiving unit 211 receives a rating request transmitted from the user terminal 10. The rating request receiving unit 211 registers information including image data included in the received rating request (hereinafter referred to as image information) in the image data storage unit 231. FIG. 11 is a diagram showing an example of the configuration of image information stored in the image data storage unit 231. As shown in the figure, the image information includes image data associated with a user ID indicating the user who captured the image. The image data was included in the rating request.

[0051] The reference information storage unit 232 stores information (hereinafter referred to as reference information) including reference values ​​related to the relationship derived from the relationship between the tool and the body, such as the tool position, tool movement (the direction and movement of the tool, etc.), and posture (the position and angle of the part), which are related to the body exercise using the tool. FIG. 12 is a diagram showing an example of the configuration of reference information stored in the reference information storage unit 232. As shown in the figure, the reference information includes, but is not limited to, the absolute position of the tool part, information on how the tool part moved (movement speed, movement distance, movement direction, etc.), the absolute position of the part, or the relative position with respect to other parts or other reference objects (hereinafter referred to as position reference information), and, for three parts including a joint part, reference information on the angle formed by the straight line connecting each of two parts and the joint part (hereinafter referred to as angle reference information), and further information on the relationship between the tool part and the body part.

[0052] The tool position reference information includes a part of the tool and a reference position of the part in association with the mode and the checkpoint ID. There may be a plurality of parts. The vertical position of the part may be, for example, the height from the ground or the distance from one of the toes. For example, when the mode is "weightlifting", the vertical position may be the distance from a body part or a line connecting parts, such as the distance between the line connecting both shoulders and the shaft. The horizontal position of the part may be the distance from a predetermined reference object (for example, a mound plate or a mark on the floor) or the distance from a reference part such as the shoulders, chest, or foot. The position reference information is assumed to be registered in advance.

[0053] The tool movement reference information includes reference values ​​for information such as the moving speed and moving distance of tool parts, the direction of movement at a certain point in time, and the trajectory of movement over a certain period of time, in association with the mode and checkpoint ID.

[0054] The position reference information includes a part and a reference position of the part in association with a mode and a checkpoint ID. There may be a plurality of parts. Regarding the position, the vertical position may be, for example, the height from the ground or the distance from one of the toes. The horizontal position may be the distance from a predetermined reference object (for example, a mound plate or a mark on the floor) or the distance from a reference part such as the shoulder, chest, or foot. The position reference information is assumed to be registered in advance.

[0055] The angle reference information corresponds to a mode and a checkpoint ID and includes two parts (part 1 and part 2), one joint part, and a reference value of the angle between a line connecting part 1 and the joint part, and a line connecting part 2 and the joint part.

[0056] The related reference information includes information on the reference expressed in the relationship between the tool part and the body part in association with the mode and the checkpoint ID. The related reference information includes information obtained from the moving speed, moving distance, angle, etc., for one or more parts and parts in association with the mode and the checkpoint ID. For example, when the mode is batting, the related reference information includes the moving speed of the tip of the bat at the time when the ball is met, and the angle formed by the bat and the dominant arm holding the bat, as reference information.

[0057] The evaluation condition information storage unit 233 stores information for performing evaluation (hereinafter, referred to as evaluation condition information). FIG. 13 is a diagram showing an example of the configuration of the evaluation condition information stored in the evaluation condition information storage unit 233. The evaluation condition information includes a category, a condition, an evaluation rank, and a comment. The category is an evaluation category. Examples of categories include "muscular strength," "ball speed," and "control." The conditions are conditions for the position, direction, or movement (change in position in time series) of each part of the tool in the image, and also conditions for the position or movement (change in position in time series) of each part of the body. For example, when analyzing weightlifting movements, conditions for the elbow angle, the arm extension speed, and the like, and conditions for the shaft movement and up and down speed during the period when the barbell is lifted and lowered, and the like, can be set in the evaluation condition information. Also, when analyzing pitching form, conditions for the elbow angle, the arm line speed, and the like, can be set in the evaluation condition information for the checkpoint at which the ball is released. The evaluation rank is an evaluation value when the above conditions are satisfied. The comments are explanations of the body posture and movements when the above conditions are met.

[0058] The image analysis unit 212 (part / site identification unit) analyzes the image data. The image analysis unit 212 analyzes the image data to extract the feature amount of each part of the tool and each part of the body, and identifies the position of each part and each site in the image. The image analysis unit 212 also analyzes the image data to extract the feature amount of each part of the tool, and identifies the direction in which each part is facing. Note that a general method of image analysis is adopted by the image analysis unit 212, and detailed explanations are omitted here. The image analysis unit 212 may analyze the image data for each frame or key frame, may analyze the image data for each checkpoint, or may analyze at random timing.

[0059] The image analysis unit 212 also compares, for each checkpoint ID, the position of each part extracted from the image data with the position reference information stored in the reference information storage unit 232, and identifies the closest time as the checkpoint time.

[0060] The evaluation unit 213 evaluates the movement of the tool used by the user based on the image data. In this embodiment, the evaluation unit 213 searches the evaluation condition information storage unit 233 for evaluation condition information including conditions satisfied by the positions of each part of the tool and the movement of the parts identified from the image data, and if there is evaluation condition information for which the conditions are satisfied, acquires the evaluation rank and comments included therein. The evaluation unit 213 may evaluate the movement of the tool and count the number of physical movements.

[0061] The evaluation unit 213 evaluates the user's body movements based on the image data. In this embodiment, the evaluation unit 213 searches the evaluation condition information storage unit 233 for evaluation condition information including conditions satisfied by the positions of each part and the movements of the parts identified from the image data, and if there is evaluation condition information for which the conditions are satisfied, acquires the evaluation rank and comments included therein. The evaluation unit 213 may evaluate the body movements and count the number of body movements.

[0062] The evaluation unit 213 evaluates the tool used by the user and the body movement based on the image data. In this embodiment, the evaluation unit 213 searches the evaluation condition information storage unit 233 for evaluation condition information including the positions of each part of the tool and each part of the body specified from the image data and the conditions satisfied by the movements or relationships of the parts and the parts, and if there is evaluation condition information for which the conditions are satisfied, obtains the evaluation rank and comments included therein. The evaluation unit 213 may evaluate the tool and the body movement and count the number of body movements.

[0063] The evaluation information transmission unit 214 transmits the evaluation information to the user terminal 10. The evaluation information transmission unit 214 generates tool position information including the time on the time axis of the video identified by the image analysis unit 212 and the position of each part of the tool. For the evaluation rank and comment acquired by the evaluation unit 213, if the position of the part of the tool satisfies the condition, the evaluation unit 214 generates posture information including the time, part, and tool orientation value, and the evaluation rank and comment, and if the movement of the part (change in position in time series) satisfies the condition, the evaluation information transmission unit 214 generates tool movement information including a list of the time, part, and tool orientation value, and the evaluation rank and comment. In addition, the evaluation information transmission unit 214 generates checkpoint information including the time corresponding to each checkpoint analyzed by the image analysis unit 212 and a checkpoint ID indicating the checkpoint. The evaluation information transmission unit 214 creates evaluation information including the generated tool position information, tool orientation information, tool movement information, and checkpoint information, and transmits it to the user terminal 10. Note that the evaluation unit 213 and the evaluation information transmission unit 214 may correspond to the comment output unit of the present invention.

[0064] The evaluation information transmission unit 214 transmits the evaluation information to the user terminal 10. The evaluation information transmission unit 214 generates position information including the time point on the time axis of the video identified by the image analysis unit 212 and the position of each part. For the evaluation rank and comment acquired by the evaluation unit 213, if the position of the part satisfies the condition, the evaluation unit 214 generates posture information including the time point, part, and posture value, and the evaluation rank and comment, and if the movement of the part (change in position in time series) satisfies the condition, the evaluation information transmission unit 214 generates movement information including a list of the time point, part, and posture value, and the evaluation rank and comment. In addition, the evaluation information transmission unit 214 generates checkpoint information including the time point corresponding to each checkpoint analyzed by the image analysis unit 212 and a checkpoint ID indicating the checkpoint. The evaluation information transmission unit 214 creates evaluation information including the generated position information, posture information, movement information, and checkpoint information, and transmits it to the user terminal 10. The evaluation unit 213 and the evaluation information transmission unit 214 may correspond to the comment output unit of the present invention.

[0065] The improvement measure information storage unit 234 stores information related to improvement measures (hereinafter referred to as improvement measure information). Fig. 14 is a diagram showing an example of the configuration of improvement measure information stored in the improvement measure information storage unit 234. As shown in the figure, the improvement measure information includes advice associated with the purpose, category, and condition. The condition may be a condition on the tool itself (such as the weight of a barbell), a condition on how to use the tool, a condition on the body (such as flexibility), a condition on the position, orientation, or movement of a part of the tool, or a condition on the position or movement of a part of the body.

[0066] The remedy request receiving unit 215 receives a remedy request sent from the user terminal 10 .

[0067] The improvement information transmitting unit 216 searches for improvement information corresponding to the mode and purpose included in the improvement request, which satisfies the conditions of the user's body information included in the evaluation request and the positions, orientations, movements, etc. of each part or each part specified by the image analysis unit 212. The improvement information transmitting unit 216 acquires advice from the searched improvement information, creates improvement information in which the purpose and advice are set, and responds to the created improvement information to the user terminal 10. The improvement information transmitting unit 216 also transmits the positions, orientations, speeds, angles, etc. of each part or each part included in the reference information, including them in the improvement information. Note that the improvement information transmitting unit 216 may search for an improvement based on the evaluation information and the reference value even without an improvement request, and the improvement information transmitting unit 216 may transmit the improvement to the user terminal 10.

[0068] FIG. 15 is a diagram showing an example of the flow of processing executed in the physical exercise support server of this embodiment.

[0069] In the user terminal 10, the imaging unit 111 accepts the input of the mode, images the user's body during exercise, and acquires video data (S321). The evaluation request sending unit 112 sends an evaluation request including the user ID indicating the user, the accepted mode, physical information, and video data to the server device 20 (S322).

[0070] When the evaluation request receiving unit 211 in the server device 20 receives an evaluation request, the image analysis unit 212 analyzes the video data to extract features (S323) and identifies the position of each part and each part (S324). Here, the image analysis unit 212 may identify the position on the image, or may identify the actual position (height from the ground, distance from a reference point such as the center of gravity of the body, etc.) using body information. The evaluation unit 213 obtains an evaluation rank and a comment from evaluation condition information that satisfies the conditions of the position of each part and each part and the movement of the part and part (time-series change in position) (S325). The evaluation information transmission unit 214 creates evaluation information and transmits it to the user terminal 10 (S326).

[0071] The evaluation display unit 114 in the user terminal 10 displays the position, orientation, movement, etc. of the tool on the video data based on the received evaluation information (S327). Furthermore, the evaluation display unit 114 in the user terminal 10 may display the position (bones) of each part indicating the posture of the body, as well as an evaluation rank and comments (S327). Here, the evaluation display unit 114 may display a graph of the position, orientation, movement, etc. of the part, or a time-series change in the position and movement of the part. Furthermore, the checkpoint display unit 115 may extract and display an image of the checkpoint from the video. The improvement measure request sending unit 116 sends an improvement measure request to the server device 20 in response to an instruction from the user (S328).

[0072] In the server device 20, when the improvement request receiving unit 215 receives an improvement request sent from the user terminal 10, the improvement information sending unit 216 searches for improvement information that satisfies the conditions, acquires the advice contained in the searched improvement information (S329), creates improvement information including the acquired advice, and sends it to the user terminal 10 (S330).

[0073] When the improvement information receiving unit 117 in the user terminal 10 receives improvement information, the improvement information display unit 118 displays the advice contained in the received improvement information and can also display a suitable way to use the tool by superimposing it on the video data (S331).

[0074] When the improvement information receiving unit 117 in the user terminal 10 receives improvement information, the improvement information display unit 118 displays the advice contained in the received improvement information and can also display an appropriate body posture in the form of bones superimposed on the video data (S331).

[0075] In this way, the physical exercise support server of this embodiment can easily evaluate physical exercise. In particular, for physical exercise related to sports, the positional relationship and movement of each part of the equipment and each part of the body can be evaluated, which is likely to lead to concrete efforts to improve and improve performance. In addition, the physical exercise support server of this embodiment also provides comments and advice, allowing the user to easily understand the current situation and improvement measures.

[0076] Fig. 17 is a diagram showing an example of a screen displaying an evaluation of physical exercise using a tool. Fig. 17 illustrates a case where a video is captured in weightlifting mode. As shown in Fig. 17, a mark 411 indicating the position of the barbell shaft is displayed on a screen 41. The movement of the barbell shaft is displayed by a line 412.

[0077] FIG. 18 is another diagram showing an example of a screen displaying an evaluation of physical exercise using a tool. FIG. 18 illustrates a case where a video is captured in weightlifting mode. As a result of evaluation performed by evaluation unit 213 in weightlifting mode, for example, the inclination of the barbell shaft, the moving distance of the shaft, and the moving speed (line 421) are displayed. In FIG. 18, a reference value (line 422) is displayed. The actual measurement result (line 423) may be displayed, and the difference from the reference value may be displayed numerically or in a graph. The user can use this as a reference to consider the movement or posture that should be corrected.

[0078] FIG. 19 is another diagram showing an example of a screen displaying an evaluation of physical exercise using a tool. FIG. 19 illustrates a case where a video is captured in weightlifting mode. As a result of evaluation by the evaluation unit 213 in weightlifting mode, for example, an evaluation result (line 432) is displayed together with a reference value (line 431) such as the angle of the body joints at the lowest point of the barbell. The difference from the reference value may be displayed numerically or as a graph. Furthermore, the evaluation result based on the relationship between the body and the tool may be displayed. The user can use this as a reference to consider the movement or posture that should be corrected.

[0079] FIG. 20 is another diagram showing an example of a screen displaying an evaluation of physical exercise using a tool. FIG. 20 illustrates a case where a video is captured in weightlifting mode. As a result of evaluation by the evaluation unit 213 in weightlifting mode, a line 441 shows bones that are lines connecting parts of the tool identified from the image and predetermined positions of body parts. The bones may be displayed superimposed on the captured image. A line 442 shows the acceleration of each body part. Furthermore, a count result such as how many times the weight was lifted may be displayed. Also, as shown by a line 443, the evaluation result and the next training result according to the purpose may be displayed.

[0080] Fig. 21 is another diagram showing an example of a screen displaying an evaluation of physical exercise using a tool. Fig. 21 illustrates a case where a video is captured in weightlifting mode. As a result of evaluation performed by evaluation unit 213 in weightlifting mode, an evaluation (line 441) obtained by comparison with various reference values ​​is displayed.

[0081] The group analysis unit 217 is assumed to be used in a scene where a plurality of users use the system at the same time and a small number of supporters provide guidance remotely (for example, training guidance using an online meeting tool or an online communication tool, remote rehabilitation support, etc.). In such a scene, there may be a plurality of users for one supporter. Although image information of a plurality of users is displayed in real time on the supporter terminal, it is difficult to grasp the detailed state of the users, such as which user is training effectively and which user is tired, and as a result, the effect is reduced. In addition, there is also a problem that it is difficult to communicate between the supporter and the user based on the content of the training that has been performed, resulting in low continuity. The group analysis unit 217 is intended to solve such a problem.

[0082] The group analysis unit 217 analyzes the evaluation information performed by the evaluation unit 213 for the image of each user, and stores the analyzed information in the group analysis information storage unit 235. Based on the evaluation information evaluated by the evaluation unit 213 for the physical exercise (including exercise using tools) performed by the user, the group analysis unit 217 creates a ranking of users whose physical exercise (which may be analyzed in real time) was effective (or ineffective), such as the number of repetitions of the training performed and the order close to (or far from) the reference value. In addition, each user may be categorized into a high ranking group, a low ranking group, etc. In addition, as the physical exercise is continued, the position, moving speed, moving distance, joint angle, etc. of the tool part or body part may be determined to be deviating from the reference value, and the degree of fatigue may be estimated based on the deviation from the reference value. Furthermore, the time to complete a predetermined number of repetitions may be measured for each set, and the degree of fatigue may be determined based on the increase in the completion time, or the degree of fatigue may be estimated based on the increase in the time or the completion time of the previous set.

[0083] Furthermore, the group analysis unit 217 may estimate the concentration level of the user by evaluating the time until the actual start of the physical exercise, such as the supporter's signal to start or end the exercise. Furthermore, the group analysis unit 217 may calculate a deviation value from the evaluation information between users who are participating in training at the same time, or may create a ranking using the deviation value. Furthermore, the group analysis unit 217 may calculate a deviation value from the evaluation information between users who performed the same physical exercise at different times, or may create a ranking using the deviation value. Furthermore, from the ranking, fatigue level, concentration level, etc., users who need guidance or users for whom direct comments are considered to be effective in terms of continuity, etc. may be identified. Furthermore, the group analysis unit 217 may compare evaluation information from when the user previously performed a similar physical exercise with evaluation information evaluated from an image of a newly performed physical exercise.

[0084] The group analysis information storage unit 235 stores the group analysis information performed by the group analysis unit 217. Fig. 15 is a diagram showing an example of the configuration of group analysis information stored in the group analysis information storage unit 235. The group analysis information mainly includes ranking information, status information, and warning information. The ranking information includes rankings based on the evaluation information, and category information such as whether the ranking is included in a higher or lower group. The status information includes fatigue levels and concentration levels. The warning information includes information on users who particularly need to be given guidance.

[0085] The group analysis information presenting unit 218 presents the group analysis information stored in the group analysis information storage unit 235 to the user terminal 10 and the supporter terminal 50. The group analysis information presenting unit 218 presents to the user terminal 10 the ranking, deviation value, categories such as high and low, etc., of the user among users who are performing physical exercise at the same time, and among users including users who performed similar physical exercise at different times. Furthermore, the group analysis information presenting unit 218 may also present information stored in the evaluation information storage unit 132. Furthermore, the group analysis information presenting unit 218 may present the ranking and deviation value information of each user to the supporter terminal 50, or may color the users who are ranked high (or low) so that they are easily visible on the screen (such as by displaying the screen showing the user large or at the top).

[0086] Fig. 22 is a diagram showing an example of a screen created by the group analysis information presenting unit 218 and presented to the user terminal 10. Fig. 22 illustrates a case where a video is captured in squat mode. As shown in Fig. 22, a screen 46 displays a line 461 indicating the rank of the user within the group. Also, a line 462 indicates the number of exercises performed. A line 463 displays a real-time video (or a recorded video) of a model trainer.

[0087] 23 is a diagram showing an example of a screen created by the group analysis information presenting unit 218 and presented to the supporter terminal 50. In the supporter terminal 50, the evaluation results and group analysis information of each user are displayed together with the video of each user.

[0088] FIG. 24 is a diagram showing an example of a screen that is presented to the supporter terminal 50 by the group analysis information presenting unit 218. The screen 47 shows images of users lined up, and displays users in the group who need more guidance (including support, etc.) based on the evaluation and group analysis so that the supporter can easily understand them. As an example, the screen 47 shows a mark in the upper right corner of the screen showing the user who needs guidance. The screen 48 shows details of each user, and displays the evaluation results (bones and posture values, tool orientation values, number of repetitions of physical exercise, etc.), the group analysis results (ranking within the group and its progress, etc.), advice sent to the user by the improvement information sending unit 216, a function to view recorded images (or a part of a video such as a few seconds before and after the point when the evaluation was lowered), and the like. The screen 47 and the screen 48 may be combined.

[0089] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention, and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and equivalents thereof are also included in the present invention.

[0090] For example, in this embodiment, the image analysis is performed in the server device 20, but this is not limiting. The image analysis may be performed in the user terminal 10 to identify the positional relationship of each part and each portion.

[0091] In addition, in this embodiment, the positions of the parts and regions are assumed to be positions on a two-dimensional image, but the present invention is not limited to this, and may be three-dimensional positions. For example, when the user terminal 10 is equipped with a depth camera in addition to the camera 106, the three-dimensional positions of the parts and regions can be specified based on the image from the camera 106 and the depth map from the depth camera. Also, for example, the three-dimensional position of the parts and regions may be specified by estimating the three-dimensionality from the two-dimensional image. It is also possible to provide a depth camera instead of the camera 106 and specify the three-dimensional position only from the depth map from the depth camera. In this case, the user terminal 10 can transmit the depth map to the server device 20 together with the image data or instead of the image data, and the image analysis unit 212 of the server device 20 can analyze the three-dimensional position.

[0092] In addition, in this embodiment, an image of the user's body while exercising using a tool is transmitted from the user terminal 10 to the server device 20, but this is not limited to the above. The user terminal 10 may extract features from the image and transmit the features to the server device 20. Alternatively, the user terminal 10 may estimate the part of the tool or the part of the body based on the features, obtain the absolute position of the part or part (which may be the position on the XY coordinate system of the image, or the actual distance from a reference position (for example, the ground, the toes, the head, the center of gravity of the body, etc.), or the position in any other coordinate system) or the relative positional relationships between multiple parts, multiple parts, or multiple part parts, and transmit these absolute positions and relative positional relationships to the server device 20.

[0093] In addition, in this embodiment, the improvement information is provided with contents prepared by the server device 20, but this is not limiting. For example, it is also possible to include reference values, and superimpose marks or bones that represent correct movements and postures (positions and orientations of each part, positions and angles of each part, etc.) based on the reference values ​​on a video or a still image extracted from the video. This makes it easy to understand what movements and postures should be used.

[0094] In addition, in this embodiment, the evaluation is performed with respect to the part or orientation of the tool, the position or movement of a body part, etc. (position over time), but this is not limited to this, and the evaluation may be performed by identifying the position of the tool worn by the user.

[0095] In addition, in this embodiment, the content such as advice is provided as a remedy, but for example, a tool recommendation may be performed. In this case, the server device 20 stores the tool and the standard value of the size (length, etc.) of the tool in association with the user's physical information (height, weight, etc.), extracts the feature amount of the tool used by the user from the image data to identify the shape of the tool, estimates the size of the tool based on the shape and the size of the user (e.g. height, etc.) included in the physical information, and if the difference between the estimated tool size and the standard value is equal to or greater than a predetermined threshold, recommends a tool of the standard size. Furthermore, a tool according to the purpose may be recommended based on information such as the conditions for the tool itself (weight of a barbell, etc.), how to use the tool, physical conditions (flexibility, etc.), the position, orientation, and movement of the tool parts, etc.

[0096] In this embodiment, the content such as advice is provided as a remedy, but for example, the physical exercise being performed may be interrupted. In this case, the server device 20 stores a reference value for interrupting the physical exercise in association with the user's physical information (purpose, height, weight, etc.), and interrupts the physical exercise when the number of times or the speed of the physical exercise being performed by the user from the image data (for example, the speed at which the barbell is lifted drops drastically, or too many times are performed at once) deviates from the reference value. In this case, a comment to stop the physical exercise may be issued to the user terminal 10, or the user may be notified by changing the display such as turning off the screen, or an alert sound may be issued, or the user may be notified by vibration.

[0097] In addition, in this embodiment, the content such as advice is provided as a remedy, but for example, a diagnosis of a disease and physical exercises for remedying the disease may be presented. In this case, the server device 20 extracts candidates of diseases that the user is assumed to have developed from the symptoms and evaluation information input by the user in the physical information, and presents a screening test for narrowing down the candidates. When the user performs the screening test and narrows down the name of the disease, the server device 20 may recommend a doctor's examination, physical exercises for remedy, or items such as tools and food for performing physical exercises.

[0098] Furthermore, by estimating the positions of the tool parts, the server device 20 can estimate the speed, acceleration, movement distance, trajectory, etc. of the tool. Furthermore, by extracting a pattern of change in the tool position over time, the server device 20 can estimate the number of times the pattern occurs as the number of actions using the tool.

[0099] In addition, in this embodiment, the exercise is evaluated, but the present invention is not limited to this. When a certain posture or movement is detected, a task for that movement may be proposed. In this case, the server device 20 may store a task in association with one or a series of postures or movements instead of an evaluation comment, and output the task.

[0100] In addition, in this embodiment, the exercise is evaluated, but the present invention is not limited to this, and when a certain tool movement, tool orientation, posture, or body part movement is detected, the content for improving the physical exercise according to the purpose, etc., such as training to be performed, rehabilitation, musical performance, or preparatory steps such as stretching, strength training, posture, etc., may be presented. In this case, the server device 20 may store the contents of the training, etc., in place of the evaluation comment, in association with one or a series of tool part movements, tool part orientations, body postures, or body part movements, and output the contents.

[0101] In addition, in this embodiment, the motion is evaluated, but the motion performed by the user can be automatically detected. In this case, the server device 20 stores the positions and postures of each part of the tool (positions of each body part) used to perform a specific motion such as shooting or passing as reference information, and can identify the motion performed by the user in the image by comparing the positions of the tool parts and body parts analyzed from the image with the reference information.

[0102] In addition, in this embodiment, the image captured in the past is analyzed to evaluate the movement, but the present invention is not limited to this. The analysis process may be performed in real time, and when a predetermined movement is detected, the next tactic to be taken may be recommended. In this case, the tactic may be stored in place of the evaluation comment in association with the posture or movement, and the tactic may be output in real time.

[0103] In addition, in this embodiment, the execution of a predetermined function and the storage of information are performed by the user terminal 10 or the server device 20, but this is not limiting, and the execution of the function and the storage of information may be performed by either one of the devices. Alternatively, the functional unit and the storage unit may be provided in a distributed manner in a form different from that of this embodiment. [Explanation of symbols]

[0104] 10 User terminal 20 Server device 30 Imaging terminal 40 Communication Network 50 Supporter terminal 111 Imaging unit 112 Evaluation request sending unit 113 Evaluation information receiving unit 114 Evaluation display section 115 Checkpoint display section 116 Improvement Request Submission Unit 117 Improvement Measures Information Receiving Department 118 Improvement information display section 130 Physical information storage unit 131 Image storage unit 132 Evaluation information storage unit 133 Improvement plan storage section 211 Evaluation request receiving unit 212 Image Analysis Unit 213 Evaluation Department 214 Evaluation information transmission unit 215 Remediation Request Reception Unit 216 Improvement Measures Information Transmission Department 217 Group Analysis Department 218 Group Analysis Presentation Section 231 Image data storage unit 232 Standard information storage section 233 Evaluation condition information storage unit 234 Improvement information storage unit 235 Group Analysis Information Storage Unit

Claims

1. A physical exercise support system that supports a user in improving the effectiveness of physical exercise performed using a tool, A reference value storage function that stores a reference value related to a positional relationship between a part of the tool and a part of the user's body; a part identification function that analyzes an image of the tool to identify a part of the tool and a part of the body and acquires a positional relationship between the part and the part; an evaluation function that determines an evaluation value of the exercise using the tool based on a comparison between the positional relationship acquired by the part identification function and the reference value; A physical exercise support system comprising:

2. a comment storage function for storing a comment on the relative positional relationship in association with the part and a condition for the relative positional relationship between the part and the body part; a comment output function that outputs the comment corresponding to the condition satisfied by the positional relationship acquired by the part identification function; The physical exercise support system according to claim 1 .

3. an improvement information storage function for storing advice on an improvement measure for the physical exercise in association with a goal of improvement for the exercise and a condition for a relative positional relationship between a part of the tool and a part of the body; an advice output function that outputs the advice corresponding to the condition satisfied by the positional relationship acquired by the part identification function; The physical exercise support system according to claim 1 or 2,

4. A physical exercise support server that supports a user in improving the effectiveness of physical exercise performed using a tool, a reference value storage unit that stores a reference value related to a positional relationship between a part of the tool and a part of the user's body; a part identification unit that analyzes an image of the tool to identify a part of the tool and a part of the body and acquires a positional relationship between the part and the part; an evaluation unit that determines an evaluation value of the exercise using the tool based on a comparison between the positional relationship acquired by the part identification unit and the reference value; A physical exercise support server comprising:

5. A physical exercise support method for supporting a user in improving the effectiveness of physical exercise performed using a tool, comprising: The computer A reference value storage step of storing a reference value related to a positional relationship between a part of the tool and a part of the user's body; a part identification step of identifying a part of the tool and a part of the body by analyzing an image of the tool and acquiring a positional relationship between the part and the part; an evaluation step of determining an evaluation value of the exercise using the tool based on a comparison between the positional relationship acquired in the part identification step and the reference value; A physical exercise support method comprising the steps of:

Images