Exercise promotion device, exercise promotion method, and recording medium

Abstract

This exercise promotion device is configured so that an avatar-data acquisition means acquires avatar data related to an avatar to be used for model video data for exercise. A frame video acquisition means acquires frame video data which is a video related to the movement of the user's frame in an exercise performed by a user. A generation means composites an avatar generated using the avatar data and the frame video data to generate model video data in which the avatar performs an exercise. A model video output means outputs the model video data. An activity-video acquisition means acquires activity video data which is a video obtained by imaging the movement of the user moving in accordance with the model video data. A frame extraction means extracts user-frame data representing the movement of the frame of the user from the activity video data. A score calculation means calculates a score on the basis of model-frame data representing the movement of the frame included in the frame video data, and the user-frame data. A result output means outputs the score as a result.

Description

Exercise promotion device, exercise promotion method, and recording medium

[0001] The present invention relates to a device that encourages exercise in a user.

[0002] As society ages, the number of elderly people who are not getting enough exercise is increasing. When people become physically disabled due to illnesses or other reasons caused by lack of exercise, they become unable to do many things, lose confidence, and become mentally weak. Therefore, in order for elderly people to maintain a healthy mind and body, it is desirable for them to pay attention to managing their physical condition and exercise moderately. Patent Document 1 describes a device that supports the physical condition management of a person to be evaluated who is performing exercises.

[0003] Japanese Patent Application Laid-Open No. 2023-55313

[0004] In recent years, there are a variety of games that promote exercise. However, the game settings and controls are complex and difficult. Furthermore, for elderly people who are not familiar with game consoles, even owning a controller can be a major hurdle.

[0005] One of the objects of the present invention is to motivate users and encourage them to exercise by displaying videos of exercises optimized for each user with simple operations.

[0006] In order to solve the above problem, in one aspect of the present invention, an exercise promotion device comprises: avatar data acquisition means for acquiring avatar data related to an avatar to be used in exercise model video data; skeletal video acquisition means for acquiring skeletal video data, which is a video of skeletal movements of an exercise performed by a user; generation means for generating model video data of the avatar performing the exercise by combining an avatar generated from the avatar data with the skeletal video data; model video output means for outputting the model video data; play video acquisition means for acquiring gameplay video data, which is a video capturing the movements of a user moving in accordance with the model video data; skeleton extraction means for extracting user skeletal data representing the skeletal movements of the user from the gameplay video data; score calculation means for calculating a score based on the model skeletal data representing the skeletal movements included in the skeletal video data and the user skeletal data; and result output means for outputting the score as a result.

[0007] In another aspect of the present invention, an exercise promotion method includes: acquiring avatar data relating to an avatar to be used in exercise model video data; acquiring skeletal video data, which is video of skeletal movements of an exercise performed by a user; generating model video data of the avatar performing the exercise by combining an avatar generated from the avatar data with the skeletal video data; outputting the model video data; acquiring gameplay video data, which is video capturing the movements of the user moving in accordance with the model video data; extracting user skeletal data representing the skeletal movements of the user from the gameplay video data; calculating a score based on the user skeletal data and the model skeletal data, which is included in the skeletal video data and represents the skeletal movements; and outputting the score as a result.

[0008] In another aspect of the present invention, a recording medium records a program that causes a computer to execute the following processes: acquire avatar data related to an avatar to be used in exercise model video data; acquire skeletal video data that is video of skeletal movements of an exercise performed by a user; generate model video data of the avatar performing the exercise by combining an avatar generated from the avatar data with the skeletal video data; output the model video data; acquire gameplay video data that is video of the user's movements in accordance with the model video data; extract user skeletal data that represents the skeletal movements of the user from the gameplay video data; calculate a score based on the user skeletal data and the model skeletal data that represents the skeletal movements included in the skeletal video data; and output the score as a result.

[0009] In yet another aspect of the present invention, the server comprises: an avatar data acquisition means for acquiring avatar data relating to an avatar to be used in exercise model video data; a skeletal video acquisition means for acquiring skeletal video data, which is a video relating to the skeletal movements of the exercise performed by the user; a generation means for generating model video data in which the avatar performs the exercise by combining an avatar generated from the avatar data with the skeletal video data; and a model video output means for outputting the model video data.

[0010] According to the present invention, by displaying an exercise video optimized for each user with a simple operation, it is possible to stimulate the user's motivation and encourage them to exercise.

[0011] 1 shows the configuration of an exercise promotion system; 2 shows the hardware configuration of an exercise promotion device and a server; 3 shows the functional configuration of an exercise promotion device in a first embodiment; 4 shows a diagram explaining a skeletal model and the coordinates of each part; 5 is a flowchart of exercise promotion processing in a first embodiment; 6 shows the configuration of an exercise promotion system in a sixth modified example; 7 shows the functional configuration of a server in a second embodiment; 8 is a flowchart of exercise promotion processing in a second embodiment; 9 shows the functional configuration of an exercise promotion device and a server in a third embodiment; 10 is a flowchart of exercise promotion processing in a third embodiment.

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. [First Embodiment] (Configuration) FIG. 1 shows the configuration of an exercise promotion system incorporating the exercise promotion device of the present invention. The exercise promotion system 100 is a system that promotes exercise for users by using videos featuring the user's "favorite" avatar. Here, "favorite" refers to a person or character that the user likes so much because of their attractive presence that the user wants to recommend them to others. In this invention, this also includes children, grandchildren, deceased spouses, pets, and other people who are close to the user. The videos are videos of specific avatars performing exercises such as gymnastics, fitness, and dancing. The exercise promotion system 100 allows users to play an exercise game by watching the videos and imitating the avatar's movements.

[0013] In this embodiment, the user is an elderly person, but the present invention is not limited to this and can be applied to people of all ages and genders.

[0014] In the exercise promotion system 100, an exercise promotion device 1 connected to a display device 3 wirelessly or by wire, and a server 20 are connected so as to be able to communicate with each other via a network 5 such as the Internet.

[0015] The exercise promotion device 1 is an information processing device that processes, stores, and transmits and receives various data. Specifically, the exercise promotion device 1 acquires skeletal video data relating to the skeletal movements of the exercise performed by the user from the server 20, and generates model video data by combining it with avatar data of the user's favorite idol. The exercise promotion device 1 then transmits the generated model video data to the display device 3 for display. This allows the user to play an exercise game in which they imitate the movements of the user's favorite idol's avatar while watching the model video played on the display device 3.

[0016] The exercise promotion device 1 also acquires play video data, which is a video of the user's movements in accordance with the model video. The exercise promotion device 1 then calculates a score for the user's movements based on the play video data and the skeletal video data, and displays the score on the display device 3.

[0017] In this embodiment, a suitable example of the display device 3 is a television, but it may also be a smartphone, a personal computer, or a general-purpose tablet PC (personal computer) as long as it can display the model video and results.

[0018] The server 20 is comprised of one or more information processing devices, and processes, stores, transmits and receives various data relating to exercise performed by the user. As will be described in detail later, the server 20 has a user database (hereinafter, the database may also be referred to as a "DB") 27, a skeletal animation DB 28, and a play history DB 29.

[0019] 2(a) is a block diagram showing the hardware configuration of the exercise promotion device 1. As shown in the figure, the exercise promotion device 1 includes an interface 11, a processor 12, a memory 13, a recording medium 14, an input unit 15, and an imaging unit 16.

[0020] The interface 11 exchanges data with the display device 3 and the server 20 via the network 5. The interface 11 is used to send various data to the display device 3 for display, and to receive various data from the server 20.

[0021] The processor 12 is a computer such as a CPU (Central Processing Unit) and executes a pre-prepared program to control the entire exercise promotion device 1. The memory 13 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 13 stores the programs executed by the processor 12. The memory 13 is also used as a working memory while the processor 12 is executing various processes.

[0022] The recording medium 14 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or semiconductor memory, and is configured to be detachable from the exercise promotion device 1. The recording medium 14 stores various programs executed by the processor 12. When the exercise promotion device 1 executes the exercise promotion process, the programs stored on the recording medium 14 are loaded into the memory 13 and executed by the processor 12. The input unit 15 includes a microphone and recognizes user instructions based on user voice data input by the user. In this embodiment, the input unit 15 recognizes user instructions based on the user voice data, but the present invention is not limited to this. The input unit 15 may also recognize user instructions based on data input via a keyboard, mouse, touch panel, or the like. The imaging unit 16 includes a camera and acquires captured still image data and video data.

[0023] 2(b) is a block diagram showing the hardware configuration of the server 20. As shown in the figure, the server 20 includes an interface 21, a processor 22, a memory 23, a recording medium 24, a display unit 25, and an input unit 26. These components, a user DB 27, a skeleton video DB 28, and a play history DB 29 are interconnected via a bus.

[0024] The interface 21 exchanges data with the exercise promotion device 1 via the network 5. The processor 22 is a computer such as a CPU, and controls the entire server 20 by executing a pre-prepared program. The memory 23 is composed of a ROM, a RAM, etc. The memory 23 stores the program executed by the processor 22. The memory 23 is also used as a working memory while the processor 22 is executing various processes.

[0025] The recording medium 24 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the exercise promotion device 1. The recording medium 24 records various programs executed by the processor 22.

[0026] The display unit 25 displays a predetermined image on, for example, an LCD (Liquid Crystal Display), etc. The input unit 26 includes a keyboard, a mouse, a touch panel, etc., and is used by an operator who manages the server 20.

[0027] The user DB 27 stores information about a user in association with a user ID that identifies the user. Specifically, the user DB 27 stores the user ID, range of motion data indicating the range of motion of the user's body, avatar data generated in response to a user instruction, contact information of a recipient to whom information about the user is to be transmitted, relationships between the recipients, and the like, in association with each other.

[0028] The range of motion data indicates the range of motion of the user's body, which is input as a pre-setting before the user exercises. Specifically, the range of motion data can be set in any format as long as it can recognize the range of motion of the user's body, such as "upper body," "lower body," "excluding right shoulder 90 degrees or more," etc. The avatar data is data that can be used to generate an avatar of a favorite character.

[0029] The transmission target is a target to which a link related to avatar selection, the degree of physical deterioration or risk of the user recognized by the exercise promotion device 1, etc. are transmitted, and specifically, the user himself / herself, family, care manager, family doctor, etc. The contact information is the telephone number, email address, etc. of the transmission target, and the relationship is "self," "child," "grandchild," "care manager," "family doctor," etc.

[0030] The skeletal video DB 28 stores information about skeletal video data in association with video IDs that identify the skeletal video data, which is video about skeletal movements during exercise performed by the user. Specifically, the skeletal video DB 28 stores, in association with each other, video IDs, range of motion data indicating the range of motion of the exercise performed by the skeletal video data, strengthening areas that are body parts trained by the skeletal video data, exercise intensity indicating the load of the exercise performed by the skeletal video data, etc.

[0031] The play history DB 29 stores information about the exercises (hereinafter also referred to as "play") performed by the user in accordance with the model video as a play history. Specifically, the play history DB 29 stores the play date and time, play video data which is a video of the play, user skeletal data which represents the user's skeletal movements extracted from the play video data, scores, etc. in association with each other.

[0032] For convenience of explanation, the server 20 includes a user DB 27, a skeletal animation DB 28, and a play history DB 29. However, the present invention is not limited to this, and the type of DB and the data structure of the DB are arbitrary as long as the necessary data can be acquired. For example, the user DB 27 and the play history DB 29 may be included in the exercise promotion device 1.

[0033] 3 is a block diagram showing the functional configuration of the exercise promotion device 1. Functionally, the exercise promotion device 1 includes a voice instruction acquisition unit 30, a presetting acceptance unit 31, an avatar data acquisition unit 32, a skeleton video acquisition unit 33, a generation unit 34, a model video output unit 35, a gameplay video acquisition unit 36, a skeleton extraction unit 37, a score calculation unit 38, a reward calculation unit 39, a result output unit 40, and a transmission unit 41, and is realized by the processor 12 executing a program.

[0034] In response to a user's voice instruction, the exercise promotion device 1 generates model video data by combining skeletal video data acquired from the server 20 with avatar data of the user's favorite character, and displays the generated model video data on the display device 3. The exercise promotion device 1 also calculates a score for the user's movements and a reward related to the play based on the captured gameplay video data and skeletal video data, and displays the results on the display device 3. The exercise promotion device 1 also transmits to the transmission target the user's level of decline and dangerous condition, which have been recognized based on the gameplay video data and the user's skeletal data.

[0035] The voice instruction acquisition unit 30 recognizes and acquires user instructions based on user voice data, which is the user's voice. Here, user instructions specifically include starting and stopping the exercise promotion device 1, inputting the user's body range of motion, selecting an avatar, selecting a model video, specifying an exercise time, playing and stopping the model video, etc.

[0036] The preset setting accepting unit 31 accepts, as a preset, motion range data indicating the range of motion of the user's body. The motion range data can be set by any method, such as a method in which the user inputs the motion range by voice, a method in which the user selects a model video for each motion range, or a method in which the motion range is identified based on video data capturing basic movements by the user. The user's motion range data is tagged with a user ID by the preset setting accepting unit 31 and transmitted to the server 20, where it is stored in the user DB 27.

[0037] The avatar data acquisition unit 32 acquires avatar data for generating an avatar of the favorite idol. Specifically, the avatar data acquisition unit 32 acquires facial image data of the favorite idol's face from multiple viewpoints (hereinafter also referred to as "avatar facial image data") and multiple pieces of audio data of the favorite idol's voice (hereinafter also referred to as "avatar audio data") via a link previously transmitted to the transmission target. The avatar data acquisition unit 32 then acquires avatar data for generating the favorite idol's avatar by generating a 3D model based on the avatar facial image data and a synthesized voice based on the avatar audio data. The avatar data is tagged with a user ID and transmitted to the server 20 by the avatar data acquisition unit 32, and is stored in the user DB 27.

[0038] Since it is difficult for an elderly user to input avatar face image data and avatar voice data, a transmitting unit 41 (described later) transmits a link for selecting an avatar to the contacts of a smartphone or the like used by the user's children or grandchildren, who have been set as recipients in advance. The user's children or grandchildren register the avatar face image data and avatar voice data via the link, and the exercise promotion device 1 can acquire the avatar face image data and avatar voice data.

[0039] The skeletal video acquisition unit 33 acquires skeletal video data stored in the skeletal video DB 28 from the server 20. When the preset setting acceptance unit 31 has accepted the user's range of motion data, the skeletal video acquisition unit 33 acquires skeletal video data corresponding to exercise within the user's range of motion from the skeletal video DB 28 based on the user's range of motion data. Note that the priority of the skeletal video data acquired by the skeletal video acquisition unit 33 may be set in advance based on strengthened body parts and exercise intensity. In this case, the skeletal video acquisition unit 33 acquires predetermined skeletal video data from the skeletal video DB 28 in consideration of the priority and the user's range of motion data.

[0040] The generation unit 34 generates model video data in which the avatar of the favorite character performs the exercise movements of the user by combining the avatar of the favorite character generated from the avatar data with the skeletal video data.

[0041] The model video output unit 35 transmits the model video data to the display device 3 and plays it back, thereby displaying the model video.

[0042] The play video acquisition unit 36 ​​acquires play video data capturing images of the user's movements in accordance with the model video displayed on the display device 3. The user moves in accordance with the movements of the avatar of their favorite character appearing in the model video while watching the display device 3, and the camera of the exercise promotion device 1 is installed in a position where it can capture images of the user's movements. The play video data is tagged with the user ID by the play video acquisition unit 36 ​​and transmitted to the server 20, and is stored in the play history DB 29 in association with the user ID and the play date and time.

[0043] The skeleton extraction unit 37 extracts user skeleton data representing the movement of the user's skeleton from the gameplay video data. Specifically, the skeleton extraction unit 37 uses a known skeleton detection method to extract the coordinates of each part of the user's skeleton from the image of each frame of the gameplay video data, and extracts the user skeleton data based on the time-series data of the coordinates of each part. FIG. 4( a) shows an example in which the user's skeleton is modeled by recognizing the coordinates of parts of the user's skeleton, such as the head, neck, and hands, and connecting them together. In this embodiment, body parts are classified into 15 locations and the coordinates of each part are recognized. Furthermore, the coordinates of each part of the modeled skeleton are composed of numerical values ​​on the X, Y, and Z coordinates, as shown in FIG. 4( b). In this case, the center of gravity of the user's body may be used as the origin.

[0044] The user skeleton data is sent to the server 20 by the skeleton extraction unit 37 in a state where it is tagged with the user ID and identification information of the play video data, and is stored in the play history DB 29 in association with the user ID and the play video data.

[0045] The score calculation unit 38 calculates a similarity based on the model skeletal data and the user skeletal data, and calculates a play score according to the calculated similarity. The model skeletal data is skeletal data that represents skeletal movements included in the skeletal video data used to generate the model video data. Specifically, the score calculation unit 38 calculates the similarity by comparing the coordinates of each body part of the model skeletal data with the coordinates of each body part of the user skeletal data based on time-series data. The score calculation unit 38 determines that the higher the similarity, the more the user moves in the same way as their favorite avatar in the model video, and calculates a higher score.

[0046] The model skeleton data may be stored in the skeleton video DB 28 in association with the skeleton video data, or may be extracted from the skeleton video data or the model video data using a known skeleton detection method. In addition, in this embodiment, the similarity is calculated by comparing the coordinates of each part, but the present invention is not limited to this, and any method can be applied, such as calculation based on the feature amount of the coordinates of each part using machine learning.

[0047] The score is sent to the server 20 by the score calculation unit 38 with the user ID and identification information of the play video data tagged therewith, and is stored in the play history DB 29 in association with the user ID and the play video data.

[0048] The reward calculation unit 39 acquires the play history stored in the play history DB 29 from the server 20, calculates the user's growth level based on the play history, and calculates a reward from the growth level. Specifically, the reward calculation unit 39 calculates the growth level, which indicates the growth of exercise intensity, etc., from the difference in the user's skeletal data based on a time series. The reward calculation unit 39 calculates a higher reward as the growth level increases. Here, the reward refers to, for example, points that can be exchanged for a benefit such as the ability to dress up the avatar of a favorite character. The reward is tagged with the user ID and identification information of the play video data and transmitted to the server 20 by the reward calculation unit 39, and is stored in the play history DB 29 in association with the user ID and the play video data.

[0049] The result output unit 40 transmits and displays data relating to the score and reward to the display device 3 at any timing, such as when the playback of the model video ends, i.e., when the user's exercise ends. The display device 3 displays the score and reward as results based on the received data.

[0050] The transmitting unit 41 acquires, in advance, contact information of the transmission target stored in the user DB 27 from the server 20. The transmitting unit 41 then transmits a link related to avatar selection to the transmission target at a predetermined timing. The transmitting unit 41 also acquires the play history stored in the play history DB 29 from the server 20 and transmits, to the transmission target, the user's decline level calculated based on the play history at a predetermined timing. Specifically, the transmitting unit 41 calculates a decline level indicating a decline in physical strength from a difference in user skeletal data based on a time series. The transmitting unit 41 also transmits the user's dangerous state identified based on the play video data and the user skeletal data to the transmission target. Specifically, the transmitting unit 41 identifies a dangerous state when it detects a fall from the user skeletal data or when the play video data includes voice data indicating danger, such as "help," and transmits text data, video data, or the like indicating the dangerous state to the transmission target. It is desirable for the dangerous state to be transmitted to the transmission target immediately after it is identified.

[0051] (Exercise Promotion Processing) Next, we will explain the exercise promotion processing performed by the exercise promotion device 1. Fig. 5 is a flowchart of the exercise promotion processing performed by the exercise promotion device 1. This processing is realized by the processor 12 shown in Fig. 2(a) executing a program prepared in advance.

[0052] First, the exercise promotion device 1 receives the user's range of motion data as a pre-set value using a predetermined method. Then, based on the user's range of motion data, the exercise promotion device 1 acquires skeletal animation data corresponding to the user's movements within the range of motion from the server 20 (step S101). The exercise promotion device 1 also transmits a link for selecting an avatar to a pre-set recipient, and acquires avatar face image data and avatar voice data of the user's favorite character via the link. The exercise promotion device 1 then acquires avatar data for generating the user's favorite character avatar (step S102).

[0053] The exercise promotion device 1 then generates model video data by combining the avatar generated from the avatar data with the skeleton video data, and transmits the model video data to the display device 3 for playback, thereby displaying the model video (step S103). The user exercises in accordance with the movements of the avatar of their favorite character that appears in the model video.

[0054] The exercise promotion device 1 captures images of the user's movements in accordance with the model video and acquires the images as play video data (step S104). The acquired play video data is stored in the play history DB 29. The exercise promotion device 1 also extracts user skeletal data, which is the movement of the user's skeleton, from the play video data (step S105). The extracted user skeletal data is stored in the play history DB 29.

[0055] The exercise promotion device 1 determines whether the user is in a dangerous state based on the gameplay video data and the user skeletal data (step S106). If a dangerous state such as the user falling is identified (step S106; Yes), the exercise promotion device 1 acquires a preset contact address from the server 20, transmits text data indicating the dangerous state, and terminates the exercise promotion process (step S107).

[0056] On the other hand, if a dangerous condition is not identified (Step S106; No), the exercise promotion device 1 calculates a score and a reward when the user finishes exercising, and transmits them to the display device 3 to display the results (Step S108). The exercise promotion device 1 also acquires a play history from the server 20 and calculates the user's decline level. Then, the exercise promotion device 1 acquires pre-set contact information from the server 20 and transmits the decline level (Step S109). This ends the exercise promotion process.

[0057] Although not described in this embodiment, the user's desired exercise time may be set in advance. In this case, specifically, after activation, the exercise promotion device 1 asks the user a question such as "How long do you want to exercise?" using voice data or text data via the display device 3. In response, the user can specify the desired exercise time by stating a time, such as "30 minutes."

[0058] As described above, the exercise promotion device 1 can stimulate each user's motivation to exercise by having the avatar of their favorite athlete appear in the model video. Furthermore, the avatar of the favorite athlete can be set to a familiar figure, such as the user's child, grandchild, or deceased spouse, allowing the user to exercise while feeling as if a loved one is cheering them on nearby. Furthermore, by displaying scores and rewards according to the user's play, the motivation to exercise can be maintained. Furthermore, if the exercise promotion device 1 recognizes the user's level of decline or a dangerous condition based on the user's play, it can contact the user's family, care manager, family doctor, etc. Therefore, the user's family or family doctor can provide health care according to the user's level of decline or dangerous condition.

[0059] The user can give voice instructions to the exercise promotion device 1. In addition, by setting the transmission destination in advance, the data required to generate an avatar of a favorite person can be registered by the user's children or grandchildren via a link. Therefore, even elderly users can easily exercise without having to perform complex operations using a controller.

[0060] [First Modification] In the first embodiment described above, the exercise promotion device 1 generates model video data using skeletal video data corresponding to the user's range of motion data. However, the present invention is not limited to this, and a score may be calculated based on the user's range of motion data. Specifically, if the user's range of motion data "excludes the right shoulder at 90 degrees or more," the exercise promotion device 1 calculates the maximum score without deducting points, even if the user in the gameplay video only raises their arms about halfway up, while the avatar in the model video raises their arms high at the time of the specified score calculation. This allows the user to exercise without losing motivation, because a score that takes the user's range of motion into account is displayed, even if the model video shows movements outside the user's range of motion.

[0061] [Second Modification] In the first embodiment described above, the exercise performed by the user in the demonstration video is performed by a favorite avatar. This allows the favorite avatar to act as a trainer, cheering on the user's movements and motivating the user. However, the present invention is not limited to this, and the trainer avatar that performs the user's exercise may be selected from a plurality of avatar candidates based on preset characters.

[0062] Specifically, the exercise promotion device 1 displays images of multiple avatar candidates along with numbers on the display device 3 and asks the user, using voice or text data, a question such as "Which character will you play with today?" In response, the user can select a trainer avatar by reading the number out loud. In this case, avatar data for each character is stored in a predetermined database or the like of the server 20, and the exercise promotion device 1 obtains the avatar data of the character selected by the user as the avatar selection result from the server 20.

[0063] This eliminates the need for the user to take the time to register avatar face image data and avatar voice data, and allows the user to easily exercise using an avatar of a favorite character as a trainer.

[0064] Once generated, avatar data is stored in the user DB 27, so the user's favorite avatar may be a candidate for the trainer's avatar.

[0065] [Third Modification] In the first embodiment described above, only the model video is displayed on the display device 3, but the present invention is not limited to this. The screen of the display device 3 may be divided into two, with the model video displayed on one side and the gameplay video displayed on the other. In this case, the exercise promotion device 1 may combine the favorite avatar with the gameplay video data to generate avatar gameplay video data in which the favorite avatar performs the user's movements and display the generated data on the screen. In this case, in the model video, the avatar of a specified trainer, rather than the favorite avatar, performs the model movements.

[0066] In other words, on one screen, a specific trainer's avatar can perform model movements in the model video, and a favorite avatar can perform the user's own movements in the avatar play video. This allows the favorite avatar to perform the user's own movements, motivating the user to exercise. In this way, the favorite avatar created uniquely for each user can be used as the trainer's avatar in the model video, or as the user's own avatar in the avatar play video.

[0067] [Fourth Modification] Although not described in the first embodiment, the exercise promotion device 1 may construct a skeletal video output model that, when a strengthening area is input, outputs one or more pieces of skeletal video data optimized for training the strengthening area. Training data is used to construct (generate) the skeletal video output model. The training data is data that associates input data input in training the skeletal video output model with correct answer data corresponding to the input data. In this modification, the input data is the strengthening area, and the correct answer data is skeletal video data that trains the strengthening area. Based on the strengthening area input as input data, the exercise promotion device 1 trains the skeletal video output model to output skeletal video data that trains the strengthening area. Examples of machine learning methods include deep learning such as CNN.

[0068] The exercise promotion device 1 may also have, as its functions, a weak exercise identifying unit that identifies weak exercises for the user by referencing the skeletal video DB 28 and play history DB 29 via the server 20, and a strengthening part identifying unit that identifies strengthening parts of the body that should be trained to compensate for the weak exercises. Specifically, the weak part identifying unit refers to the strengthening parts and play history corresponding to the skeletal video data, and identifies, for example, "squats" as a weak exercise from the user's skeletal data and score. Furthermore, the strengthening part identifying unit identifies the "gluteus maximus" as a strengthening part to compensate for the weak exercises.

[0069] The exercise promotion device 1 then inputs the strengthening body parts identified by the strengthening body part identification unit, causing the skeletal animation output model to output skeletal animation data. The skeletal animation data output here can be said to be skeletal exercise data optimized to supplement the user's weak exercises. Specifically, the result output unit 40 of the exercise promotion device 1 generates a service list recommending model animation data that combines the skeletal animation data output by the skeletal animation output model with the user's favorite avatar, along with text data indicating the weak exercise "squats" and the strengthening body part "gluteus maximus." The result output unit 40 then transmits data related to the service list to the display device 3, causing the service list to be displayed.

[0070] If there are multiple recommended model video data, the result output unit 40 displays the multiple model videos together with numbers on the display device 3 and asks the user questions such as "Which exercise do you want to do? Or leave it to me?" using audio data or text data. In response, the user can select the type of exercise they want to do by saying "leave it to me" or reading the number out loud. Furthermore, the model video data recommended as a service list is not limited to those corresponding to exercises that the user is not good at, but may also be those corresponding to the exercise duration desired by the user.

[0071] According to this, the exercise promotion device 1 can provide the user with a list of services tailored to each user and recommend exercises using model videos optimized to supplement exercises that the user is not good at or model videos that fit the desired exercise time.

[0072] The skeletal video output model may be a machine learning model that has been trained to output skeletal video data optimized to supplement an exercise that the user is not good at when that exercise is input.

[0073] [Fifth Modification] In the first embodiment described above, the exercise promotion device 1 does not include a display unit and is connected to a display device 3 such as a television. However, the present invention is not limited to this, and an exercise promotion device including a display unit may also be used. The display unit is, for example, an LCD or the like, which displays a predetermined screen. Like the exercise promotion device 1, the exercise promotion device including a display unit is an information processing device that processes, stores, and transmits and receives various data, such as a smartphone or general-purpose tablet on which an app that executes the exercise promotion process is installed. The exercise promotion device including a display unit can generate model video data and display the model video on the display unit by executing the exercise promotion process. Furthermore, the exercise promotion device including a display unit can calculate a score based on the play video data and display the result on the display unit.

[0074] [Sixth Modification] In the first embodiment described above, the exercise promotion device 1 and the server 20 are two terminals communicably connected via the network 5, and the exercise promotion device 1 executes the exercise promotion process, but the present invention is not limited to this, and the exercise promotion process may be executed by an exercise promotion device 120 having the functions of the server 20. The exercise promotion device 120, like the exercise promotion device 1, is an information processing device that processes, stores, and transmits and receives various data.

[0075] 6 shows an example of the configuration of the exercise promotion system 200 in this case. The exercise promotion device 120 is connected to a user DB 127, a skeleton video DB 128, and a play history DB 129. The exercise promotion device 120 executes exercise promotion processing, and is capable of generating and outputting model video data and calculating and outputting scores based on the play video data.

[0076] [Second Embodiment] In the first embodiment described above, the exercise promotion device 1 executes the exercise promotion process to generate and output model video data and calculate and output scores based on the gameplay video data. However, the present invention is not limited to this, and the server 20x may execute the exercise promotion process to generate and output model video data and calculate and output scores based on the gameplay video data. Note that, for convenience, in the second embodiment, explanations similar to those in the first embodiment will be omitted.

[0077] 7 is a block diagram showing the functional configuration of a server 20x according to the second embodiment. In the second embodiment, the exercise promotion device 1x is an information processing device that processes, stores, and transmits / receives various data related to exercise performed by a user. Specifically, the exercise promotion device 1x transmits instructions recognized based on user voice data, which is the user's voice, to the server 20x. Furthermore, when the exercise promotion device 1x acquires user range of motion data as a pre-setting, it transmits the user range of motion data to the server 20x. Furthermore, when the exercise promotion device 1x captures gameplay video data, it transmits the data to the server 20x.

[0078] Functionally, server 20x comprises a voice instruction acquisition unit 50, a pre-setting acceptance unit 51, an avatar data acquisition unit 52, a skeleton video acquisition unit 53, a generation unit 54, a model video output unit 55, a gameplay video acquisition unit 56, a skeleton extraction unit 57, a score calculation unit 58, a reward calculation unit 59, a result output unit 60, and a transmission unit 61, and is realized by processor 22 executing a program.

[0079] The server 20x generates model video data by combining skeletal video data with avatar data of the user's favorite character in accordance with the user's instructions acquired from the exercise promotion device 1x. The generated model video data is transmitted by the server 20x to the display device 3 via the exercise promotion device 1x and displayed as a model video. The server 20x also calculates a score and a reward for the user's movements based on the gameplay video data and skeletal video data acquired from the exercise promotion device 1x. The server 20x transmits data related to the calculated score and reward to the display device 3 via the exercise promotion device 1x and displays the results. The server 20x also calculates the user's level of decline and identifies a risk condition based on the gameplay video data and user skeletal data, and transmits the level of decline and the risk condition to a transmission target as necessary.

[0080] The voice instruction acquisition unit 50 acquires user instructions from the exercise promotion device 1x. Here, the user instructions include starting the exercise promotion device 1x to start exercising, inputting the user's body range of motion, selecting an avatar, specifying the exercise time, playing or stopping the model video, etc., and the voice instruction acquisition unit 50 can acquire these instructions in real time from the exercise promotion device 1x via the network 5.

[0081] The preset setting accepting unit 51 accepts, as a preset setting, range of motion data indicating the range of motion of the user's body from the exercise promotion device 1x. The range of motion data of the user is stored in the user DB 27 in association with the user ID.

[0082] The avatar data acquisition unit 52 acquires avatar data relating to the avatar of the favorite person to be used in the exercise example video data. The avatar data is stored in the user DB 27 in association with the user ID.

[0083] The transmitting unit 61 (described later) transmits a link for selecting an avatar to the recipient in advance, and registers the avatar face image data and avatar voice data via the link so that the server 20x can receive the data. This allows the avatar data acquiring unit 52 to acquire avatar data based on the avatar face image data and avatar voice data.

[0084] The skeleton video acquisition unit 53 acquires skeleton video data stored in the skeleton video DB 28. When the pre-setting acceptance unit 51 has accepted the user's range of motion data, the skeleton video acquisition unit 53 acquires, from the skeleton video DB 28, skeletal video data corresponding to exercise within the user's range of motion, based on the user's range of motion data.

[0085] The generation unit 54 generates model video data in which the avatar of the favorite character performs the exercise movements performed by the user by combining the avatar of the favorite character generated from the avatar data with the skeletal video data.

[0086] The model video output unit 55 transmits the model video data to the display device 3 via the exercise promotion device 1x and plays it to display the model video. In other words, when the exercise promotion device 1x receives the model video data from the server 20x, it transmits the model video data to the display device 3 and plays it, thereby displaying the model video on the display device 3.

[0087] The gameplay video acquisition unit 56 acquires gameplay video data captured by the exercise promotion device 1x from the exercise promotion device 1x. The gameplay video data may be acquired at any timing, and may be acquired in real time. The gameplay video data is stored in the play history DB 29 in association with the user ID and the play date and time.

[0088] The skeleton extraction unit 57 extracts user skeleton data representing the movement of the user's skeleton from the gameplay video data. The user skeleton data is stored in the play history DB 29 in association with the user ID and the gameplay video data.

[0089] The score calculation unit 58 calculates a similarity between the model skeleton data and the user skeleton data, and calculates a score for the play according to the calculated similarity. The score is stored in the play history DB 29 in association with the user ID and play video data.

[0090] The reward calculation unit 59 calculates the user's growth level based on the play history stored in the play history DB 29, and calculates a reward from the growth level. The reward is stored in the play history DB 29 in association with the user ID and play video data.

[0091] The result output unit 60 transmits data related to the score and the reward to the display device 3 via the exercise promotion device 1x at any timing, and displays the score and the reward as results. In other words, when the exercise promotion device 1x receives data related to the score and the reward from the server 20x, it transmits the data to the display device 3 and displays the score and the reward on the display device 3.

[0092] The transmission unit 61 refers to the user DB 27 and transmits a link related to avatar selection to the transmission target. The transmission unit 61 also refers to the user DB 27 and transmits to the transmission target the user's decline level calculated based on the play history stored in the play history DB 29. The transmission unit 61 also refers to the user DB 27 and transmits to the transmission target the user's risk state identified based on the play video data and the user skeletal data.

[0093] (Exercise Promotion Processing) Next, the exercise promotion processing by the server 20x will be described. Fig. 8 is a flowchart of the exercise promotion processing by the server 20x. This processing is realized by the processor 22 shown in Fig. 2(b) executing a program prepared in advance.

[0094] First, the server 20x receives the user's range of motion data as a pre-set value via the exercise promotion device 1x. Then, based on the user's range of motion data, the server 20x acquires skeletal animation data corresponding to the user's movements within the range of motion from the skeletal animation DB 28 (step S201). The server 20x also transmits a link for selecting an avatar to a pre-set recipient, and acquires avatar face image data and avatar voice data of the favorite character via the link. The server 20x then acquires avatar data for generating the favorite character's avatar (step S202).

[0095] The server 20x then generates model video data by combining the avatar generated from the avatar data with the skeleton video data, and transmits the model video data to the display device 3 via the exercise promotion device 1x (step S203). The display device 3 can display the model video by playing the received model video data. The user exercises in accordance with the movements of the avatar of their favorite person appearing in the model video.

[0096] The exercise promotion device 1x captures the user's movements in accordance with the model video and transmits the captured images to the server 20x as play video data. The server 20x then acquires the play video data (step S204). The acquired play video data is stored in the play history DB 29. The server 20x also extracts user skeletal data, which represents the user's skeletal movements, from the play video data (step S205). The extracted user skeletal data is stored in the play history DB 29.

[0097] The server 20x determines whether the user is in a dangerous state based on the gameplay video data and the user skeletal data (step S206). If a dangerous state such as the user falling is identified (step S206; Yes), the server 20x acquires a preset contact address from the user DB 27, transmits text data indicating the dangerous state, and terminates the exercise promotion process.

[0098] On the other hand, if a risky condition is not identified (step S206; No), the server 20x calculates a score and a reward when the user finishes exercising and transmits data related to the score and the reward to the display device 3 via the exercise promotion device 1x (step S208). The display device 3 displays the score and the reward as results based on the received data. The server 20x also calculates the user's decline level based on the play history stored in the play history DB 29. The server 20x then acquires pre-set contact information from the user DB 27 and transmits the decline level to the contact information (step S209), thereby terminating the exercise promotion process.

[0099] As described above, the server 20x can stimulate the user's motivation to exercise by having the user's favorite avatar appear in the model video, as in the first embodiment. Furthermore, even elderly users can easily exercise without having to perform complex operations using a controller.

[0100] The first to fourth modified examples of the first embodiment can be similarly applied to the second embodiment by replacing the processing of the exercise promotion device 1 with the server 20x.

[0101] [Third Embodiment] In the first embodiment, the exercise promotion device 1 executes the exercise promotion process, and in the second embodiment, the server 20x executes the exercise promotion process. However, the present invention is not limited to this, and the exercise promotion device 1y and the server 20y may execute the exercise promotion process in cooperation with each other. Note that in the third embodiment, for convenience, the same description as in the first embodiment will be omitted.

[0102] FIG. 9 is a block diagram showing the functional configuration of an exercise promotion device 1y and a server 20y according to the third embodiment.

[0103] Functionally, the server 20y comprises an audio instruction acquisition unit 70, a pre-setting acceptance unit 71, an avatar data acquisition unit 72, a skeleton video acquisition unit 73, a generation unit 74, and a model video output unit 75, and is realized by the processor 22 executing a program.

[0104] The server 20y generates model video data by combining the skeleton video data with the avatar data of the favorite character in response to the user's instructions acquired from the exercise promotion device 1y. The generated model video data is transmitted by the server 20y to the display device 3 via the exercise promotion device 1y and displayed as a model video.

[0105] The voice instruction acquisition unit 70 acquires user instructions from the exercise promotion device 1y. Note that the voice instruction acquisition unit 70 can acquire the instructions from the exercise promotion device 1y in real time via the network 5.

[0106] The preset setting accepting unit 71 acquires range of motion data indicating the range of motion of the user's body from the exercise promotion device 1y and accepts the acquired range of motion data as a preset setting. Upon acquiring the user's range of motion data as a preset setting, the exercise promotion device 1y transmits the user's range of motion data to the server 20y. The user's range of motion data is stored in the user DB 27 in association with the user ID.

[0107] The avatar data acquisition unit 72 acquires avatar data relating to the avatar of the favorite person to be used in the exercise example video data. The avatar data is stored in the user DB 27 in association with the user ID.

[0108] The transmitter 91 of the exercise promotion device 1y (described later) transmits a link for selecting an avatar to the recipient in advance, and registers the avatar face image data and avatar voice data via the link so that the server 20y can receive the data. This allows the avatar data acquisition unit 72 to acquire avatar data based on the avatar face image data and avatar voice data.

[0109] The skeleton video acquisition unit 73 acquires skeleton video data stored in the skeleton video DB 28. When the pre-setting acceptance unit 71 has accepted the user's range of motion data, the skeleton video acquisition unit 73 acquires, from the skeleton video DB 28, skeletal video data corresponding to exercise within the user's range of motion, based on the user's range of motion data.

[0110] The generation unit 74 generates model video data in which the avatar of the favorite character performs the exercise movements of the user by combining the avatar of the favorite character generated from the avatar data with the skeletal video data.

[0111] The model video output unit 75 transmits the model video data to the display device 3 via the exercise promotion device 1y and plays it to display the model video. In other words, when the exercise promotion device 1y receives the model video data from the server 20y, it transmits the model video data to the display device 3 and plays it, causing the display device 3 to display the model video.

[0112] Functionally, the exercise promotion device 1y comprises an audio instruction acquisition unit 85, a play video acquisition unit 86, a skeleton extraction unit 87, a score calculation unit 88, a reward calculation unit 89, a result output unit 90, and a transmission unit 91, and is realized by the processor 22 executing a program.

[0113] The exercise promotion device 1y calculates a score and a reward for the user's movements based on the captured gameplay video data and the model video data acquired from the server 20y. The calculated score and data related to the reward are transmitted to the display device 3 and displayed as a result. The exercise promotion device 1y also calculates the user's level of decline and identifies a risk condition based on the gameplay video data and the user's skeletal data, and transmits the level of decline and the risk condition to a transmission target as necessary.

[0114] The voice instruction acquisition unit 85 recognizes and acquires instructions from the user based on user voice data, which is the user's voice.

[0115] The gameplay video acquisition unit 86 acquires the captured gameplay video data. The gameplay video data is tagged with the user ID by the gameplay video acquisition unit 86 and transmitted to the server 20y. The gameplay video data is stored in the play history DB 29 in association with the user ID and the play date and time.

[0116] The skeleton extraction unit 87 extracts user skeleton data representing the movement of the user's skeleton from the gameplay video data. The user skeleton data is tagged with the user ID and identification information of the gameplay video data by the skeleton extraction unit 87 and transmitted to the server 20y, and is stored in the play history DB 29 in association with the user ID and the gameplay video data.

[0117] The score calculation unit 88 calculates a similarity based on the model skeletal data and the user skeletal data, and calculates a play score according to the calculated similarity. The score is tagged with the user ID and identification information of the play video data by the score calculation unit 88 and transmitted to the server 20y, and is stored in the play history DB 29 in association with the user ID and the play video data.

[0118] The reward calculation unit 89 acquires the play history stored in the play history DB 29 from the server 20y, calculates the user's growth level based on the play history, and calculates a reward from the growth level. The reward is transmitted to the server 20y by the reward calculation unit 89 in a state tagged with the user ID and identification information of the play video data, and is stored in the play history DB 29 in association with the user ID and the play video data.

[0119] The result output unit 90 transmits data relating to the score and reward to the display device 3 at any timing, and displays the score and reward as the result.

[0120] The transmitting unit 91 acquires in advance contact information of the transmission target stored in the user DB 27 from the server 20y. Then, the transmitting unit 91 transmits a link related to avatar selection to the transmission target. The transmitting unit 91 also acquires play history stored in the play history DB 29 from the server 20y and transmits the user's decline level calculated based on the play history to the transmission target. The transmitting unit 91 also transmits the user's dangerous state identified based on play video data and user skeletal data to the transmission target.

[0121] (Exercise Promotion Processing) Next, the exercise promotion processing performed by the exercise promotion device 1y and the server 20y in cooperation with each other will be described. Fig. 10 is a flowchart of the exercise promotion processing performed by the exercise promotion device 1y and the server 20y in cooperation with each other. This processing is realized by the processor 12 shown in Fig. 2(a) and the processor 22 shown in Fig. 2(b) executing their respective programs prepared in advance.

[0122] First, the server 20y receives the user's range of motion data as a pre-set value via the exercise promotion device 1y. Then, based on the user's range of motion data, the server 20y acquires skeletal animation data corresponding to the user's movements within the range of motion from the skeletal animation DB 28 (step S301). The server 20y also acquires avatar face image data and avatar voice data of the favorite character via a link related to avatar selection previously transmitted to the transmission target. The server 20y then acquires avatar data for generating the favorite character's avatar (step 302).

[0123] The server 20y then generates model video data by combining the avatar generated from the avatar data with the skeleton video data, and transmits the model video data to the display device 3 via the exercise promotion device 1y (step S303). The display device 3 can display the model video by playing the received model video data. The user exercises in accordance with the movements of the avatar of their favorite person appearing in the model video.

[0124] The exercise promotion device 1y captures images of the user's movements in accordance with the model video and acquires the images as play video data (step S304). The acquired play video data is stored in the play history DB 29. The exercise promotion device 1y also extracts user skeletal data, which is the movement of the user's skeleton, from the play video data (step S305). The extracted user skeletal data is stored in the play history DB 29.

[0125] The exercise promotion device 1y determines whether the user is in a dangerous state based on the gameplay video data and the user skeletal data (step S306). If a dangerous state such as the user falling is identified (step S306; Yes), the exercise promotion device 1y acquires a preset contact address from the server 20y, transmits text data indicating the dangerous state, and terminates the exercise promotion process (step S307).

[0126] On the other hand, if a dangerous condition is not identified (Step S306; No), the exercise promotion device 1y calculates a score and a reward when the user finishes exercising, and transmits them to the display device 3 to display the results (Step S308). The exercise promotion device 1y also acquires a play history from the server 20y and calculates the user's decline level. The exercise promotion device 1y then acquires pre-set contact information from the server 20y and transmits the decline level (Step S309). This ends the exercise promotion process.

[0127] In addition, when the exercise promotion device 1y and the server 20y work together to perform the exercise promotion process as in this embodiment, the functional configuration is not limited to that shown in FIG. 9, and it is possible to arbitrarily set which of the functions of the exercise promotion device 1y and the server 20y the voice instruction acquisition unit, the advance setting acceptance unit, the avatar data acquisition unit, the skeleton video acquisition unit, the generation unit, the model video output unit, the play video acquisition unit, the skeleton extraction unit, the score calculation unit, the reward calculation unit, the result output unit and the transmission unit belong to.

[0128] In addition, the first to fourth variants of the first embodiment can be similarly applied to the third embodiment by replacing the processing of the exercise promotion device 1 with processing performed in cooperation between the exercise promotion device 1y and the server 20y.

[0129] As described above, the exercise promotion device 1y and the server 20y can stimulate the user's motivation to exercise by having the user's favorite avatar appear in the model video, as in the first embodiment. Furthermore, even elderly users can easily exercise without having to perform complex operations using a controller.

[0130] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0131] (Supplementary Note 1) An exercise promotion device comprising: an avatar data acquisition means for acquiring avatar data relating to an avatar to be used in exercise model video data; a skeletal video acquisition means for acquiring skeletal video data, which is video relating to skeletal movements of an exercise performed by a user; a generation means for generating model video data of the avatar performing an exercise by combining an avatar generated from the avatar data with the skeletal video data; a model video output means for outputting the model video data; a play video acquisition means for acquiring play video data, which is video capturing the movements of a user moving in accordance with the model video data; a skeleton extraction means for extracting user skeletal data representing the skeletal movements of the user from the play video data; a score calculation means for calculating a score based on the model skeletal data representing the skeletal movements included in the skeletal video data and the user skeletal data; and a result output means for outputting the score as a result.

[0132] (Supplementary Note 2) The exercise promotion device according to Supplementary Note 1 further comprises a transmitting means for transmitting a link relating to the selection of the avatar to a predetermined recipient, wherein the avatar data acquiring means acquires avatar face image data and avatar voice data via the link, and acquires the avatar data by generating a 3D model based on the avatar face image data and a synthetic voice based on the avatar voice data.

[0133] (Supplementary Note 3) The exercise promotion device according to Supplementary Note 1, further comprising: a preset accepting unit that accepts a preset range of motion of the user; and the score calculating unit that calculates the score by taking the range of motion of the user into consideration.

[0134] (Supplementary Note 4) The exercise promotion device according to Supplementary Note 1, further comprising a pre-setting accepting means for accepting a pre-setting of the user's range of motion, wherein the skeletal video acquisition means acquires skeletal video data corresponding to exercise within the user's range of motion.

[0135] (Appendix 5) An exercise promotion device as described in Appendix 1, comprising: a play history storage means for storing the play video data and the user skeletal data as a play history relating to the user's movements; and a reward calculation means for calculating the user's growth level based on the play history and calculating a reward from the growth level, wherein the result output means outputs the reward as a result.

[0136] (Appendix 6) An exercise promotion device as described in Appendix 1, comprising: a play history storage means for storing the user skeletal data and the score as a play history relating to the user's movements; and a transmission means for calculating the user's degree of decline based on the play history and transmitting the user's degree of decline to a predetermined recipient.

[0137] (Appendix 7) An exercise promotion device as described in Appendix 1, comprising: a play history storage means for storing the gameplay video data and the user skeletal data as a play history relating to the user's movements; and a weak exercise identification means for identifying an exercise that the user is not good at based on the play history, wherein the skeletal video acquisition means acquires skeletal video data optimized to complement the exercise that the user is not good at using a machine learning model trained to output the skeletal video data in response to input of the exercise that the user is not good at, and the result output means outputs a list of services that recommend skeletal video data optimized to complement the exercise that the user is not good at.

[0138] (Appendix 8) An exercise promotion device as described in Appendix 1, comprising a transmission means for transmitting the user's dangerous state to a predetermined transmission target when the user's dangerous state is identified based on the play video data and the user skeletal data.

[0139] (Appendix 9) The exercise promotion device according to Appendix 1, further comprising a display device, wherein the model video output means displays a model video based on the model video data on the display device, and the result output means displays the result on the display device.

[0140] (Supplementary Note 10) The exercise promotion device according to Supplementary Note 1, further comprising: a voice instruction acquisition means for recognizing and acquiring a user's instruction to the exercise promotion device based on user voice data that is the user's voice.

[0141] (Appendix 11) An exercise promotion method comprising: acquiring avatar data relating to an avatar to be used in exercise model video data; acquiring skeletal video data which is video of skeletal movements of an exercise performed by a user; generating model video data of the avatar performing the exercise by combining an avatar generated from the avatar data with the skeletal video data; outputting the model video data; acquiring gameplay video data which is video of the user's movements in accordance with the model video data; extracting user skeletal data which represent the skeletal movements of the user from the gameplay video data; calculating a score based on the user skeletal data and the model skeletal data which represent the skeletal movements included in the skeletal video data; and outputting the score as a result.

[0142] (Appendix 12) A recording medium having recorded thereon a program that causes a computer to execute the following processes: acquire avatar data regarding an avatar to be used in exercise model video data; acquire skeletal video data that is video of skeletal movements of an exercise performed by a user; generate model video data of the avatar performing the exercise by combining an avatar generated from the avatar data with the skeletal video data; output the model video data; acquire gameplay video data that is video of the user's movements in accordance with the model video data; extract user skeletal data that represents the skeletal movements of the user from the gameplay video data; calculate a score based on the user skeletal data and the model skeletal data that represents the skeletal movements included in the skeletal video data; and output the score as a result.

[0143] (Appendix 13) A server comprising: an avatar data acquisition means for acquiring avatar data relating to an avatar to be used in exercise model video data; a skeletal video acquisition means for acquiring skeletal video data, which is a video relating to the skeletal movements of the exercise performed by a user; a generation means for generating model video data in which the avatar performs the exercise by combining an avatar generated from the avatar data with the skeletal video data; and a model video output means for outputting the model video data.

[0144] (Appendix 14) A system comprising the server of claim 13 and an exercise promotion device, wherein the exercise promotion device comprises: a model video output means for outputting the model video data; a play video acquisition means for acquiring play video data which is a video capturing the movements of a user who moves in accordance with the output model video data; a skeleton extraction means for extracting user skeleton data representing the skeletal movements of the user from the play video data; a score calculation means for calculating a score based on the model skeleton data representing the skeletal movements included in the skeleton video data and the user skeleton data; and a result output means for outputting the score as a result.

[0145] Although the present invention has been described above with reference to the embodiments and examples, the present invention is not limited to the above-described embodiments and examples. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0146] 1, 1x, 1y, 1z, 120 Exercise promotion device 3 Display device 5 Network 11, 21 Interface 12, 22 Processor 13, 23 Memory 14, 24 Recording medium 15, 26 Input unit 16 Imaging unit 25 Display unit 27, 127 User DB 28, 128 Skeleton video DB 29, 129 Play history DB

Claims

1. An exercise promotion device comprising: avatar data acquisition means for acquiring avatar data related to an avatar to be used in exercise model video data; skeletal video acquisition means for acquiring skeletal video data, which is video relating to skeletal movements of exercise performed by a user; generation means for generating model video data in which the avatar performs an exercise by combining an avatar generated from the avatar data with the skeletal video data; model video output means for outputting the model video data; play video acquisition means for acquiring play video data, which is video capturing the movements of a user moving in accordance with the model video data; skeleton extraction means for extracting user skeletal data representing the skeletal movements of the user from the play video data; score calculation means for calculating a score based on the model skeletal data representing the skeletal movements included in the skeletal video data and the user skeletal data; and result output means for outputting the score as a result.

2. The exercise promotion device of claim 1, further comprising a transmitting means for transmitting a link regarding the selection of the avatar to a predetermined recipient, wherein the avatar data acquiring means acquires avatar face image data and avatar voice data via the link, and acquires the avatar data by generating a 3D model based on the avatar face image data and a synthetic voice based on the avatar voice data.

3. The exercise promotion device according to claim 1, further comprising a preset setting receiving means for receiving a preset setting of the user's range of motion, wherein the score calculation means calculates the score taking into account the user's range of motion.

4. The exercise promotion device of claim 1, further comprising a pre-setting accepting means for accepting pre-setting of the user's range of motion, wherein the skeletal video acquisition means acquires skeletal video data corresponding to exercise within the user's range of motion.

5. An exercise promotion device as described in claim 1, comprising: a play history storage means for storing the play video data and the user skeletal data as a play history relating to the user's movements; and a reward calculation means for calculating a growth level of the user based on the play history and calculating a reward from the growth level, wherein the result output means outputs the reward as a result.

6. The exercise promotion device of claim 1, further comprising: a play history storage means for storing the user's skeletal data and the score as a play history relating to the user's movements; and a transmission means for calculating the user's degree of decline based on the play history and transmitting the user's degree of decline to a predetermined recipient.

7. An exercise promotion device as described in claim 1, comprising a play history storage means for storing the play video data and the user skeletal data as a play history regarding the user's movements, and a weak exercise identification means for identifying the weak exercises of the user based on the play history, wherein the skeletal video acquisition means acquires skeletal video data optimized to supplement the weak exercises of the user using a machine learning model trained to output the skeletal video data in response to input of the weak exercise, and the result output means outputs a list of services recommending skeletal video data optimized to supplement the weak exercises of the user.

8. The exercise promotion device as described in claim 1, further comprising a transmission means for transmitting the user's dangerous state to a predetermined transmission target when the user's dangerous state is identified based on the play video data and the user skeletal data.

9. The exercise promotion device according to claim 1, further comprising a display device, the model video output means displays a model video based on the model video data on the display device, and the result output means displays the result on the display device.

10. The exercise promotion device according to claim 1, further comprising a voice instruction acquisition means for recognizing and acquiring a user's instruction to said exercise promotion device based on user voice data which is the user's voice.

11. A method for promoting exercise, comprising the steps of: obtaining avatar data relating to an avatar to be used in exercise model video data; obtaining skeletal video data which is a video showing skeletal movements of an exercise performed by a user; generating model video data of the avatar performing exercise by combining an avatar generated from the avatar data with the skeletal video data; outputting the model video data; obtaining play video data which is a video capturing the movements of a user moving in accordance with the model video data; extracting user skeletal data which represents the skeletal movements of the user from the play video data; calculating a score based on the user skeletal data and the model skeletal data which represents the skeletal movements included in the skeletal video data; and outputting the score as a result.

12. A recording medium having recorded thereon a program that causes a computer to execute the following processes: obtaining avatar data regarding an avatar to be used in exercise model video data; obtaining skeletal video data that is a video of skeletal movements of an exercise performed by a user; generating model video data of the avatar performing the exercise by combining an avatar generated from the avatar data with the skeletal video data; outputting the model video data; obtaining play video data that is a video capturing the movements of a user moving in accordance with the model video data; extracting user skeletal data that represents the skeletal movements of the user from the play video data; calculating a score based on the user skeletal data and the model skeletal data that represents the skeletal movements included in the skeletal video data; and outputting the score as a result.

13. A server comprising: an avatar data acquisition means for acquiring avatar data relating to an avatar to be used in exercise model video data; a skeletal video acquisition means for acquiring skeletal video data which is a video relating to skeletal movements of an exercise performed by a user; a generation means for generating model video data in which the avatar performs an exercise by combining an avatar generated from the avatar data with the skeletal video data; and a model video output means for outputting the model video data.

14. A system comprising the server of claim 13 and an exercise promotion device, the exercise promotion device comprising: a model video output means for outputting the model video data; a play video acquisition means for acquiring play video data which is a video capturing the movements of a user moving in accordance with the output model video data; a skeleton extraction means for extracting user skeletal data which represents the skeletal movements of the user from the play video data; a score calculation means for calculating a score based on model skeletal data which represents the skeletal movements included in the skeletal video data and the user skeletal data; and a result output means for outputting the score as a result.

Images