Method and apparatus for outputting information

Abstract

The embodiments of the present disclosure disclose a method and device for outputting information. The specific implementation of the method includes: playing an initial teaching video of a fitness program; obtaining a fitness video of a user following the teaching video to perform fitness; identifying the fitness movements of the user in the fitness video; determining the matching degree between the fitness movements and the movements in the teaching video; combining the matching degree with a pre-built physical fitness knowledge base to determine the physical condition of the user; and updating the teaching video of the fitness program suitable for the user according to the physical condition. This implementation can achieve appropriate exercise for all parts of the body without the need for a professional personal trainer.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly to a method and device for outputting information. Background Art

[0002] With the improvement of living standards, people pay more and more attention to health, which leads to a greater demand for fitness. However, the biggest problem with fitness is that ordinary people cannot master the appropriate level, which makes people either practice without any effect or over-exercise and hurt themselves. At the same time, due to geographical and economic constraints, it is unrealistic for everyone to hire professional coaches for fitness guidance.

[0003] The existing video teaching fitness program requires users to provide feedback. It mainly relies on various hardware devices to collect physiological data information, and the user's usage cost is relatively high. However, physiological data information is instantaneous and can only show the current state of the user at that moment. It cannot be used for a long time and cannot reflect the user's physical fitness in real time. Summary of the invention

[0004] Embodiments of the present disclosure provide methods and devices for outputting information.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for outputting information, comprising: playing an instructional video of an initial fitness program; obtaining a fitness video in which a user follows the instructional video to perform fitness; identifying the user's fitness movements in the fitness video; determining a degree of match between the fitness movements and the movements in the instructional video; combining the degree of match with a pre-built physical fitness knowledge base to determine the user's physical condition; and recommending an instructional video of a fitness program suitable for the user based on the physical condition.

[0006] In some embodiments, the method further includes: if the fitness is finished, recording the user's physical condition.

[0007] In some embodiments, playing the instructional video of the initial fitness program includes: obtaining a facial image of a user; determining the identity of the user based on the facial image; obtaining the physical condition of the user based on the identity; and selecting an instructional video from a set of candidate instructional videos for playing based on the physical condition.

[0008] In some embodiments, the playing of the instructional video of the initial fitness program includes: obtaining a full-body image of the user; predicting the height, weight, and body shape of the user based on the full-body image; determining the user's physical health condition based on the height, weight, and body shape; and selecting an instructional video from a set of candidate instructional videos for playing based on the physical health condition.

[0009] In some embodiments, the method further includes: identifying the user's complexion based on the full-body image; determining the user's facial health status based on the complexion; and selecting a teaching video from a candidate teaching video set for playback based on the body health status and the facial health status.

[0010] In some embodiments, the method further includes: identifying the age and gender of the user based on the full-body image; and selecting a teaching video from a set of candidate teaching videos for playback based on the age, gender, body health status and facial health status.

[0011] In some embodiments, the matching degree includes angle matching degree, time matching degree and count matching degree; and the determination of the matching degree between the fitness action and the action in the teaching video includes: extracting the first bone point data of the target body part of the target action in the teaching video; extracting the second bone point data of the target body part of the target action in the fitness video; comparing the position of the first bone point data with the second bone point data to obtain the angle matching degree; comparing the time of the first bone point data with the second bone point data to obtain the time matching degree; comparing the number of actions of the first bone point data with the second bone point data to obtain the count matching degree.

[0012] In some embodiments, before determining the degree of match between the fitness movement and the movement in the teaching video, the method also includes: obtaining a first threshold P and a second threshold Q for each movement in the teaching video, wherein the first threshold P includes: a full score angle threshold, a full score time threshold, and a full score count threshold; the second threshold Q includes: a basic angle threshold, a basic time threshold, and a basic count threshold.

[0013] In some embodiments, the matching degree is calculated by: calculating the angle matching degree, the time matching degree and the count matching degree according to the following formula, wherein when calculating the angle matching degree: P is the angle full score threshold, Q is the angle basic threshold, and s is the action angle extracted from the second skeleton point data; when calculating the time matching degree: P is the time full score threshold, Q is the time basic threshold, and s is the action duration extracted from the second skeleton point data; when calculating the count matching degree: P is the count full score threshold, Q is the count basic threshold, and s is the action count extracted from the second skeleton point data.

[0014]

[0015] In some embodiments, the physical fitness evaluation index includes at least one of the following: flexibility, stability, and endurance; the evaluation index is pre-set according to different fitness movements.

[0016] In some embodiments, combining the matching degree with a pre-built physical fitness knowledge base to determine the user's physical condition includes: obtaining the matching degree of each fitness movement of the user during fitness; forming a matching matrix M according to the body parts and evaluation indicators corresponding to each fitness movement, wherein flexibility is measured by the average value of angle matching, stability is measured by time matching, and endurance is measured by time matching and / or count matching; for each movement, multiplying the matching matrix M by the difficulty matrix K of the movement to obtain the physical fitness scoring matrix N under the movement; and taking a weighted average of the physical fitness scoring matrix N for each movement to obtain the user's physical condition.

[0017] In some embodiments, the difficulty matrix K is obtained through a pre-trained knowledge base.

[0018] In a second aspect, an embodiment of the present disclosure provides a device for outputting information, comprising: a playback unit configured to play an instructional video of an initial fitness program; an acquisition unit configured to acquire a fitness video in which a user follows the instructional video to perform fitness; an identification unit configured to identify the user's fitness movements in the fitness video; a matching unit configured to determine a degree of match between the fitness movements and the movements in the instructional video; a determination unit configured to determine the user's physical condition by combining the degree of match with a pre-built physical fitness knowledge base; and a recommendation unit configured to update the instructional video of a fitness program suitable for the user based on the physical condition.

[0019] In some embodiments, the device further comprises a storage unit configured to record the user's physical condition if the fitness is finished.

[0020] In some embodiments, the playback unit is further configured to: obtain a facial image of a user; determine the identity of the user based on the facial image; obtain the physical condition of the user based on the identity; and select a teaching video from a candidate teaching video set for playback based on the physical condition.

[0021] In some embodiments, the playback unit is further configured to: obtain a full-body image of the user; predict the height, weight, and body shape of the user based on the full-body image; determine the user's physical health status based on the height, weight, and body shape; and select a teaching video from a candidate teaching video set for playback based on the physical health status.

[0022] In some embodiments, the recommendation unit is further configured to: identify the user's complexion based on the full-body image; determine the user's facial health status based on the complexion; and select a teaching video from a candidate teaching video set for playback based on the body health status and the facial health status.

[0023] In some embodiments, the recommendation unit is further configured to: identify the age and gender of the user based on the full-body image; and select a teaching video from a set of candidate teaching videos for playback based on the age, gender, body health status and facial health status.

[0024] In some embodiments, the matching degree includes angle matching degree, time matching degree and count matching degree; and the matching unit is further configured to: extract the first bone point data of the target body part of the target action in the teaching video; extract the second bone point data of the target body part of the target action in the fitness video; compare the positions of the first bone point data and the second bone point data to obtain the angle matching degree; compare the time of the first bone point data and the second bone point data to obtain the time matching degree; compare the number of actions of the first bone point data and the second bone point data to obtain the count matching degree.

[0025] In some embodiments, the matching unit is further configured to: before determining the degree of matching between the fitness movement and the movement in the teaching video, obtain a first threshold P and a second threshold Q for each movement in the teaching video, wherein the first threshold P includes: a full score angle threshold, a full score time threshold, and a full score count threshold; the second threshold Q includes: a basic angle threshold, a basic time threshold, and a basic count threshold.

[0026] In some embodiments, the matching unit is further configured to: calculate the angle matching degree, the time matching degree and the count matching degree according to the following formula, wherein, when calculating the angle matching degree: P is the angle full score threshold, Q is the angle basic threshold, and s is the action angle extracted from the second skeleton point data; when calculating the time matching degree: P is the time full score threshold, Q is the time basic threshold, and s is the action duration extracted from the second skeleton point data; when calculating the count matching degree: P is the count full score threshold, Q is the count basic threshold, and s is the action count extracted from the second skeleton point data.

[0027]

[0028] In some embodiments, the physical fitness evaluation index includes at least one of the following: flexibility, stability, and endurance; the evaluation index is pre-set according to different fitness movements.

[0029] In some embodiments, combining the matching degree with a pre-built physical fitness knowledge base to determine the user's physical condition includes: obtaining the matching degree of each fitness movement of the user during fitness; forming a matching matrix M according to the body parts and evaluation indicators corresponding to each fitness movement, wherein flexibility is measured by the average value of angle matching, stability is measured by time matching, and endurance is measured by time matching and / or count matching; for each movement, multiplying the matching matrix M by the difficulty matrix K of the movement to obtain the physical fitness scoring matrix N under the movement; and taking a weighted average of the physical fitness scoring matrix N for each movement to obtain the user's physical condition.

[0030] In some embodiments, the difficulty matrix K is obtained through a pre-trained knowledge base.

[0031] In a third aspect, an embodiment of the present disclosure provides an electronic device for outputting information, comprising: one or more processors; a storage device on which one or more computer programs are stored, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect.

[0032] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0033] The method and device for outputting information provided by the embodiments of the present disclosure can formulate a preliminary fitness plan based on the basic conditions of the human body quickly obtained by visual methods without the need for a personal trainer, and then determine the actual state of each part of the user's body according to the specific state during fitness. The fitness plan is continuously adjusted based on the actual state of these parts, thereby helping users choose a fitness plan that is completely suitable for them and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0035] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;

[0036] Figure 2 is a flow chart of an embodiment of a method for outputting information according to the present disclosure;

[0037] Figure 3a-3f is a schematic diagram of a physical fitness knowledge base for a method of outputting information according to the present disclosure;

[0038] Figure 4 is a schematic diagram of an application scenario of the method for outputting information according to the present disclosure;

[0039] Figure 5 is a structural schematic diagram of an embodiment of a device for outputting information according to the present disclosure;

[0040] Figure 6 It is a structural diagram of a computer system of an electronic device suitable for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0042] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0043] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method for outputting information or the apparatus for outputting information disclosed herein may be applied.

[0044] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0045] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as fitness applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0046] Terminal devices 101, 102, 103 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens and supporting video playback, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Group Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) players, laptop computers and desktop computers, etc. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0047] The server 105 may be a server that provides various services, such as a background server that provides support for videos played on the terminal devices 101, 102, and 103. The background server may analyze and process the received fitness request and other data (which may include user information, user images, and videos), and feed back the processing results (such as recommended fitness videos) to the terminal device.

[0048] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, multiple software or software modules used to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0049] It should be noted that the method for outputting information provided in the embodiments of the present disclosure is generally executed by a terminal device, and accordingly, the device for outputting information is generally set in the terminal device. Optionally, the method for outputting information can also be executed by a server, and accordingly, the device for outputting information is set in the server.

[0050] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0051] Continue to refer Figure 2, shows a process 200 of an embodiment of a method for outputting information according to the present disclosure. The method for outputting information comprises the following steps:

[0052] Step 201, play the teaching video of the initial fitness program.

[0053] In this embodiment, the user can use the execution subject (for example Figure 1 The terminal device shown in FIG. 10A sends a fitness request to the server. The fitness request may include the user's identity information, thereby obtaining the user's registered personal information and historical exercise data. The user may send a fitness request to the server by entering the account number, and the server may query the registration information through the account number to obtain the user's identity information.

[0054] In addition, the terminal device can directly collect facial images to generate fitness requests. The server performs facial recognition and matches the user's previously registered face to identify the user.

[0055] The server can select a default fitness program teaching video based on the user's identity and send it to the terminal device for playback. For example, for a 20-year-old male, a muscle-building fitness video with a high training intensity is recommended. For a 50-year-old female, a weight-loss fitness video with a low training intensity is recommended. These teaching videos are divided into sections, with the coach demonstrating standard fitness movements and the user following along. Each section is targeted to train a part of the human body. For example, 10 minutes of leg training, 10 minutes of waist training, and 10 minutes of arm training.

[0056] Optionally, the terminal device can directly play the instructional video of the fitness program stored locally.

[0057] In some optional implementations of this embodiment, playing an instructional video of an initial fitness program includes: obtaining a facial image of a user; determining the identity of the user based on the facial image; obtaining the physical condition of the user based on the identity; and selecting an instructional video from a set of candidate instructional videos for playing based on the physical condition.

[0058] The face image is sent to the server, and the server recognizes the user's identity based on the face, and then obtains the user's physical condition during the previous stage of fitness. The physical condition is obtained through step 205. After the user completes the recommended course, if he wants to continue exercising, the server will analyze the amount of training he can bear this time based on the user's identity status data obtained in the last course. For example, the heart rate after exercise in the last course has reached 140, then it is not suitable to recommend strenuous exercises at this time, but relaxing exercises can be recommended.

[0059] In some optional implementations of this embodiment, playing the teaching video of the initial fitness program includes: obtaining a full-body image of the user; predicting the height, weight, and body shape of the user based on the full-body image; determining the user's physical health status based on the height, weight, and body shape; and selecting a teaching video from a candidate teaching video set for playing based on the physical health status. The user's height, weight, and body shape can be estimated from the full-body image by a pre-trained neural network. The user's BMI (body mass index) and body fat percentage can be calculated from these data. Then, a suitable fitness program is recommended based on the BMI and body fat percentage. The fitness program can also be determined based on the body shape, for example, a fitness program with more arm training is recommended for an apple-shaped body, and a fitness program with more leg training is recommended for a pear-shaped body.

[0060] In some optional implementations of the present embodiment, playing an instructional video of an initial fitness program includes: recognizing the complexion of the user based on the full-body image; determining the facial health status of the user based on the complexion; and selecting an instructional video from a set of candidate instructional videos for playing based on the body health status and the facial health status.

[0061] The face can be detected from the full-body image, and then the complexion can be identified. If the terminal device or server has recorded the user's previous complexion, it can be directly compared to detect whether the user's current complexion is normal. For example, if it is paler than usual, it indicates that the user is weak and it is not suitable for strenuous exercise. If the terminal device and the server have not recorded the user's complexion, the user's complexion can be compared with the average skin color of people of the same age and gender to determine whether it is abnormal. A classifier can be trained to predict and identify the user's facial health status.

[0062] Then we combine body health and facial health to recommend instructional videos together.

[0063] In some optional implementations of this embodiment, the method further includes: identifying the age and gender of the user based on the full-body image; selecting a teaching video from a candidate teaching video set for playback based on the age, the gender, the body health condition and the facial health condition. Suitable fitness programs can be recommended by comprehensively considering various factors, and recommendations can be made in a targeted manner. The teaching video may have some labels, for example, 20-25 years old, male, back exercises, and good health. Match the user's various information (such as age, gender, body health condition and facial health condition) with the labels and sort them by matching degree. Prioritize the fitness program with the highest matching degree. Teaching videos with a slightly lower matching degree but within the range that the user's physical condition can bear can also be recommended for the user to choose.

[0064] Step 202, obtaining a fitness video in which a user follows a teaching video to perform fitness.

[0065] In this embodiment, the terminal device collects in real time the fitness video of the user following the teaching video to perform fitness, which can be used for local recognition or sending the video to the server.

[0066] Step 203: Identify the user's fitness movements in the fitness video.

[0067] In this embodiment, the terminal device or server can detect the key points of the human body in the video through a pre-trained human key point recognition model. For example, the neck, elbow joint, wrist joint, etc. The second skeleton point data of the target body part of the target action in the fitness video is extracted. The user's action is determined by the second skeleton point data, including the position, angle, duration, etc. of the key points, for example, the head turns to the left and the arm is stretched flat.

[0068] Step 204, determining the matching degree between the fitness movements and the movements in the teaching video.

[0069] In this embodiment, the terminal device or server can use the same method to extract the key points of the human body from the teaching video and determine the demonstration action. When making the teaching video, the action information can be extracted and stored for future use. The first skeleton point data of the target body part of the target action in the teaching video is extracted. The demonstration action is determined by the first skeleton point data. The user's image and the demonstration action image can be aligned to match the key point positions. The image matching algorithm of the prior art can be used to calculate the matching degree of the action. If an action can include multiple parts, the matching degrees of multiple parts can be included.

[0070] There are at least three types of matching requirements: the first is the angle requirement, that is, certain joints must reach a certain angle; the second is the time requirement, which is applicable to static movements; and the third is the counting requirement, which is applicable to movements that require counting. Therefore, the matching degree at least includes the angle matching degree S1, the time matching degree S2, and the counting matching degree S3. Some movements do not need to meet the three matching requirements, = for example, rope skipping only has counting requirements and time requirements, and no requirements for angles; some movements are not limited to the above three matching requirements. The following description takes only the above three matching requirements as an example.

[0071] Each action has two achievement requirements: full score requirement P and basic requirement Q.

[0072] 1. Angle requirements such as Figure 3f As shown:

[0073] Full score requirement: Set the angle full score threshold, for example, the angle between the arm and the shoulder is > 45 degrees.

[0074] Basic requirements: Set a basic angle threshold, for example, the arm is raised above the shoulder, that is, the angle is > 0 degrees.

[0075] 2. Time requirements:

[0076] Full score requirement: Set the time threshold for full score, for example, persistence time > 15 minutes.

[0077] Basic requirement: Set a basic time threshold, for example, persistence time > 5 minutes

[0078] 3. Counting requirements:

[0079] Full score requirement: Set the full score threshold for counting, for example, the number > 45

[0080] Basic requirement: Set the basic counting threshold, for example, the number > 30

[0081] The angle matching degree S1, time matching degree S2 and count matching degree S3 are calculated according to the following formula, wherein, when calculating the angle matching degree: P is the angle full score threshold, Q is the angle basic threshold, s is the action angle extracted from the second skeleton point data; when calculating the time matching degree: P is the time full score threshold, Q is the time basic threshold, s is the action duration extracted from the second skeleton point data; when calculating the count matching degree: P is the count full score threshold, Q is the count basic threshold, s is the action count extracted from the second skeleton point data; the above s value is the angle between the various body parts being exercised (for example, the angle between the left arm and the right arm in jumping jacks) obtained by using AI technologies such as skeleton recognition, as well as the value obtained by counting / timing with the help of AI-coaching technology.

[0082] Each matching degree S1, S2, S3 is a number between 0 and 1, and the calculation formula is as follows, where s is the actual movement angle, count, and time, and Score represents the matching degree S:

[0083]

[0084] Step 205 , combining the matching degree with a pre-built physical fitness knowledge base to determine the user's physical condition.

[0085] The matching degree types of various parts of the body required to be calculated for each action, as well as the full score requirement P and basic requirement Q for each matching degree are preset in the physical fitness knowledge base, which is a pre-trained database and is updated as needed, such as Figure 3a-3e As shown, it can be saved in the server or downloaded to the terminal device. The following information is preset in the physical fitness knowledge base:

[0086] (1) Different movements, such as push-ups, jumping jacks, etc.

[0087] The physical fitness knowledge base is pre-set to obviously involve actions that reflect various physical fitness of body parts. For example, most of the stretching actions are set as actions involving flexibility, some actions that need to be held statically are set as actions involving stability, and some actions that last longer are set as actions involving endurance. An action can reflect only one physical fitness, or it may reflect multiple physical fitness.

[0088] (2) The body parts exercised by each movement. For example, arms, legs, etc. Body parts can also be subdivided, for example, arms can be subdivided into upper arms, forearms, hands, etc.

[0089] like Figure 3c The stretch shown exercises the neck and arms. Figure 3d The movements shown exercise the waist and abdomen. Figure 3e The movements shown exercise the waist and abdomen.

[0090] (3) In each action, the matching requirements for each body part include the matching type and matching degree threshold. Each action can have multiple matching requirements, and the matching degree threshold is the basic requirement threshold and the full score requirement threshold mentioned above.

[0091] The full score requirements and basic requirements can be set manually or through big data statistics. For example, the full score threshold for jumping jacks can be set by counting the average number of jumping jacks performed by athletes. The basic threshold for sit-ups can be set by counting the average score of sit-ups performed by college students.

[0092] (4) Each matching requirement corresponds to the physical qualities required of one or more body parts, such as flexibility, stability, endurance, etc.

[0093] According to all the above requirements, a difficulty coefficient is set for each part of the physical fitness corresponding to each action. All difficulty coefficients constitute the difficulty matrix of each action.

[0094] For example, the difficulty coefficient is composed of integers from 0 to 5, where 0 means it is irrelevant to the part / ability, and 1 to 5 means the difficulty coefficient is from easy to difficult. For example, if you are required to do 45 jumping jacks within 30 seconds, you can get the difficulty matrix K, for example:

[0095]

[0096] The specific process of determining the user's physical condition based on the above physical fitness knowledge base is as follows:

[0097] Step 2051: Under a specific action, according to the matching degrees S1, S2, S3 of the body parts calculated in step 204, the physical fitness of the body parts corresponding to the specific action is obtained from the knowledge base, and the matching matrix M under the specific action is obtained.

[0098] For example, flexibility can be measured by the average of angle matching, stability can be measured by time matching, and endurance can be measured by time matching and / or count matching.

[0099] Taking jumping jacks as an example, since jumping jacks involve arms and legs, the flexibility of the arms and legs can be measured by calculating the average of the angle matching of the arms and legs in multiple jumping jacks. The details are as follows:

[0100] For arm flexibility, each time you complete a jumping jack, you will get an arm angle matching degree. Add up the arm angle matching degrees in all completed movements and divide it by the number of completions to get the average angle matching degree of the arm part, which is the arm flexibility score.

[0101] Similarly, each time you complete a jumping jack, you will get a leg angle match. By adding up the leg angle match of all completed movements and dividing it by the number of completions, you can get the average leg angle match, which is the leg flexibility score.

[0102] The endurance of the user can also be assessed by jumping jacks. At the same time, the endurance of the arms and legs can be assessed by counting matching (how many jumping jacks are completed within a specified time).

[0103] In some exercises, endurance can also be measured by timing, for example, using push-ups to assess the endurance of the arms and legs.

[0104] The endurance can also be measured by both time match and count match. For example, the time taken to complete a predetermined number of jumping jacks can be counted. The shorter the time, the higher the time match, and the more counts, the higher the count match. The endurance of the arms and legs can be evaluated by the weighted sum of the time match and the count match.

[0105] Stability can be measured by time matching. For example, Figure 3d In the movements shown, the stability of the waist and abdomen can be evaluated by timing matching.

[0106] The following is the matching matrix M obtained by taking the jumping jack as an example:

[0107]

[0108]

[0109] Step 2052: multiply the matching matrix M under a specific action by the preset difficulty matrix K of the action to obtain the user's physical fitness matrix N under the action.

[0110] As mentioned above, the difficulty matrix K of each action is preset in the knowledge base. Taking jumping jacks as an example, its difficulty matrix K is as follows:

[0111] Part / Ability Flexibility stability Durability …… neck 0 0 0 0 arm 2 1 2 0 Legs 1 0 3 0 …… 0 0 0 0

[0112] The difficulty matrix mainly reflects the requirements of the movement for different physical fitness of different parts of the body. As shown in the table above, since the load requirement of this movement is greater for the legs than for the arms, in the item of endurance, the difficulty coefficient of the arms is set to 2 and the difficulty coefficient of the legs is set to 3.

[0113] The matching matrix M obtained in step 2051 is multiplied by the difficulty matrix K, and the physical fitness matrix N obtained for the jumping jack is:

[0114]

[0115] For each action, steps 2051 and 2052 are repeatedly performed to obtain the physical fitness matrix N for each action.

[0116] Step 2053: Perform weighted averaging on the physical fitness matrix N of all actions to obtain the physical condition S of the current user.

[0117] Combined with the user's historical data, the latest physical condition is updated. The physical condition S obtained according to the above steps is compared and updated with the historical condition to obtain the latest physical condition.

[0118] Step 206, updating the instructional video of the fitness program suitable for the user according to the physical condition.

[0119] In this embodiment, a variety of fitness programs are designed in advance according to the combination of different physical conditions. A fitness program that matches the latest physical condition of the user is selected. A fitness program may correspond to multiple videos, which can be arbitrarily combined. For example, if you need to exercise your waist, abdomen, and arms, you can splice together a video of exercising your waist and abdomen and a video of exercising your arms with an intensity that matches your physical condition and send them to the terminal device. You can also search for a suitable teaching video from the fitness program stored in the terminal device for update. If there is no suitable teaching video in the terminal device, you can send the physical condition to the server, and the server will find a teaching video suitable for the user and send it to the terminal device.

[0120] In some optional implementations of the present embodiment, a teaching video of a fitness plan suitable for the user is recommended based on the physical condition, including: inputting the physical condition and the initial fitness plan into a pre-trained neural network to obtain the matching degree of each part of the human body; and recommending a teaching video of a fitness plan suitable for the user based on the matching degree of each part of the human body.

[0121] The neural network is used to determine the matching degree between the physical condition and the current fitness program. If the matching degree of a certain part is positive, the difficulty of the movement of that part can be increased. If the matching degree of a certain part is negative, the difficulty of the movement of that part can be reduced. If the matching degree of a certain part is 0, the movement of the same difficulty is maintained. For example, the flexibility score of the neck in the physical condition is 0.9, and the current fitness program requires too low flexibility of the neck, which is only 0.5. The matching degree is 0.9-0.5=0.4, which cannot fully demonstrate the user's expertise and further improve the user's ability. Other more difficult movements will be recommended to exercise flexibility. If the user's neck flexibility score is 0.3, it means that the user's flexibility is not good. If the current fitness program requires the neck flexibility to be 0.6, it is difficult for the user to achieve, which may cause injury to the user. At this time, the matching degree is 0.3-0.6=-0.3, and the difficulty of the movement needs to be reduced.

[0122] Continue to see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of an application scenario of the method for outputting information according to this embodiment. Figure 4 In the application scenario, implement the following steps:

[0123] 1. Use the camera to obtain the user portrait, and use the existing AI technology to identify the user's basic information, such as gender, age, complexion, body shape, weight, height, etc.

[0124] 2. Provide a preliminary (historical) fitness plan suitable for the user based on the user's basic information

[0125] 3 Users start using AI-coaching (movement videos demonstrated by a virtual artificial intelligence coach) to exercise and collect their fitness status.

[0126] 4. Combine the knowledge base with the deep learning network to get the user's latest physical condition

[0127] 5. Get a new fitness plan that is more suitable for the user based on his or her physical condition.

[0128] This application provides an intelligent method for assisting users in designing and adjusting fitness plans, so that appropriate exercises can be achieved for all parts of the body without the need for a professional personal trainer. For the entire plan, only camera information is needed, and no other hardware facilities are needed to collect information, which is low cost. The user's physical fitness obtained based on the action is a long-term condition, and it can be used to continuously improve the experience.

[0129] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for outputting information. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0130] like Figure 5 As shown, the device 500 for outputting information in this embodiment includes: a playback unit 501, an acquisition unit 502, an identification unit 503, a matching unit 504, a determination unit 505 and a recommendation unit 506. The playback unit 501 is configured to play the teaching video of the initial fitness program; the acquisition unit 502 is configured to acquire a fitness video in which the user follows the teaching video to perform fitness; the identification unit 503 is configured to identify the fitness movements of the user in the fitness video; the matching unit 504 is configured to determine the matching degree between the fitness movement and the movement in the teaching video; the determination unit 505 is configured to combine the matching degree with a pre-built physical fitness knowledge base to determine the physical condition of the user; and the recommendation unit 506 is configured to update the teaching video of the fitness program suitable for the user according to the physical condition.

[0131] In this embodiment, the specific processing of the playback unit 501, the acquisition unit 502, the identification unit 503, the matching unit 504, the determination unit 505 and the recommendation unit 506 of the device for outputting information 500 can be referred to. Figure 2 Corresponding to step 201, step 202, step 203, step 204, step 205 and step 206 in the embodiment.

[0132] In some optional implementations of this embodiment, the device 500 further includes a storage unit (not shown in the drawings) configured to record the user's physical condition after the fitness session is completed.

[0133] In some optional implementations of this embodiment, the playback unit 501 is further configured to: obtain a facial image of a user; determine the identity of the user based on the facial image; obtain the physical condition of the user based on the identity; and select a teaching video from a set of candidate teaching videos for playback based on the physical condition.

[0134] In some optional implementations of this embodiment, the playback unit 501 is further configured to: obtain a full-body image of the user; predict the height, weight, and body shape of the user based on the full-body image; determine the physical health status of the user based on the height, weight, and body shape; and select a teaching video from a set of candidate teaching videos for playback based on the physical health status.

[0135] In some optional implementations of this embodiment, the recommendation unit 506 is further configured to: identify the user's complexion based on the full-body image; determine the user's facial health status based on the complexion; and select a teaching video from a set of candidate teaching videos for playback based on the body health status and the facial health status.

[0136] In some optional implementations of this embodiment, the recommendation unit 506 is further configured to: identify the age and gender of the user based on the full-body image; and select a teaching video from a set of candidate teaching videos for playback based on the age, gender, body health status and facial health status.

[0137] In some optional implementations of this embodiment, the recommendation unit 506 is further configured to: input the physical condition and the initial fitness plan into a pre-trained neural network to obtain the matching degree of each part of the human body; and recommend a teaching video of the fitness plan suitable for the user based on the matching degree of each part of the human body.

[0138] In some optional implementations of this embodiment, the matching degree is a weighted sum of an angle matching degree, a time matching degree and a count matching degree; and the matching unit 504 is further configured to: extract the first bone point data of the target body part of the target action in the teaching video; extract the second bone point data of the target body part of the target action in the fitness video; compare the positions of the first bone point data and the second bone point data to obtain the angle matching degree; compare the time of the first bone point data and the second bone point data to obtain the time matching degree; compare the number of actions of the first bone point data and the second bone point data to obtain the count matching degree.

[0139] In some optional implementations of this embodiment, the matching unit 504 is further configured to: before determining the degree of matching between the fitness movement and the movement in the teaching video, obtain a first threshold P and a second threshold Q for each movement in the teaching video, wherein the first threshold P includes: a full score angle threshold, a full score time threshold, and a full score count threshold; the second threshold Q includes: a basic angle threshold, a basic time threshold, and a basic count threshold.

[0140] In some optional implementations of this embodiment, the matching unit 504 is further configured to: calculate the angle matching degree, the time matching degree and the count matching degree according to the following formula, wherein when calculating the angle matching degree: P is the angle full score threshold, Q is the angle basic threshold, and s is the action angle extracted from the second skeleton point data; when calculating the time matching degree: P is the time full score threshold, Q is the time basic threshold, and s is the action duration extracted from the second skeleton point data; when calculating the count matching degree: P is the count full score threshold, Q is the count basic threshold, and s is the action count extracted from the second skeleton point data.

[0141]

[0142] In some optional implementations of this embodiment, the evaluation index of physical fitness includes at least one of the following: flexibility, stability, and endurance; the evaluation index is pre-set according to different fitness movements.

[0143] In some optional implementations of this embodiment, the determination unit 505 is further configured to: obtain the matching degree of each fitness movement of the user during fitness; form a matching matrix M according to the body parts and evaluation indicators corresponding to each fitness movement, wherein flexibility is measured by the average value of angle matching, stability is measured by time matching, and endurance is measured by time matching and / or count matching; for each movement, multiply the matching matrix M by the difficulty matrix K of the movement to obtain the physical fitness scoring matrix N under the movement; and perform weighted averaging on the physical fitness scoring matrix N of each movement to obtain the physical condition of the user.

[0144] In some embodiments, the difficulty matrix K is obtained through a pre-trained knowledge base.

[0145] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0146] An electronic device for outputting information comprises: one or more processors; a storage device on which one or more computer programs are stored, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement the method described in process 200.

[0147] A computer-readable medium stores a computer program, wherein the computer program implements the method described in process 200 when executed by a processor.

[0148] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0149] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0150] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0151] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as a method for outputting information. For example, in some embodiments, the method for outputting information may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the method for outputting information described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute a method for outputting information in any other appropriate manner (e.g., by means of firmware).

[0152] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0153] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0154] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0155] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0156] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0157] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a server of a distributed system, or a server combined with a blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server may be a server of a distributed system, or a server combined with a blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0158] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0159] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for outputting information, comprising: Identify the identity of the user through the full-body image of the user, and play the initial fitness program teaching video according to the identity of the user, wherein the identity includes at least one of the following: height, weight, body shape, body health condition, and facial health condition; if the user's complexion has not been recorded, compare the user's complexion with the average skin color of people of the same age and gender to determine whether it is abnormal; Obtaining a fitness video in which the user follows the instructional video to perform fitness; Identify the user's fitness movements in the fitness video, including the positions, angles, and durations of key points of the human body; Determine the matching degree between the fitness action and the action in the teaching video, wherein the matching degree includes angle matching degree, time matching degree, and count matching degree; Combining the matching degree with a pre-built physical fitness knowledge base to determine the physical condition of the user, wherein the physical fitness knowledge base sets matching requirements corresponding to each body part in each action, including matching type and matching degree threshold; The teaching video of the fitness program suitable for the user is updated according to the physical condition. If the matching degree of a certain part of the human body is positive, the movement difficulty of the part is increased. If the matching degree of a certain part of the human body is negative, the movement difficulty of the part is reduced.

2. The method according to claim 1, wherein: The method further comprises: If the fitness is finished, the user's physical condition is recorded.

3. The method according to claim 2, wherein: The method of playing the teaching video of the initial fitness program according to the identity of the user includes: Get the user's face image; determining the identity of the user according to the facial image; Acquiring the user's physical condition according to the identity; A teaching video is selected from a candidate teaching video set for playing according to the physical condition.

4. The method according to claim 1, wherein: The method of playing the teaching video of the initial fitness program according to the identity of the user includes: Obtaining a full-body image of the user; Predicting the height, weight, and body shape of the user based on the full-body image; Determining the user's physical health status based on height, weight, and body shape; A teaching video is selected from a candidate teaching video set for playing according to the physical health condition.

5. The method according to claim 4, wherein: The method further comprises: recognizing the user's facial expression based on the full-body image; determining the user's facial health status according to the complexion; A teaching video is selected from a candidate teaching video set for playing according to the body health condition and the facial health condition.

6. The method according to claim 5, wherein: The method further comprises: Identify the age and gender of the user according to the full-body image; A teaching video is selected from a candidate teaching video set for playing according to the age, the gender, the body health condition and the facial health condition.

7. The method according to claim 1, wherein: Determining the matching degree between the fitness action and the action in the teaching video includes: Extracting first bone point data of a target body part of a target action in the teaching video; Extracting second skeleton point data of a target body part of a target action in the fitness video; Comparing the positions of the first skeleton point data and the second skeleton point data to obtain the angle matching degree; Performing a time comparison between the first skeleton point data and the second skeleton point data to obtain the time matching degree; The first skeleton point data and the second skeleton point data are compared in terms of the number of actions to obtain the count matching degree.

8. The method according to claim 7, wherein: Before determining the matching degree between the fitness action and the action in the teaching video, the method further comprises: obtaining a first threshold value P and a second threshold value Q for each action in the teaching video, Wherein, the first threshold value P includes: angle full score threshold value, time full score threshold value, and count full score threshold value; The second threshold Q includes: an angle basic threshold, a time basic threshold, and a count basic threshold.

9. The method according to claim 8, wherein: The calculation method of the matching degree is: The angle matching, time matching and count matching are calculated according to the following formulas, wherein when calculating the angle matching: P is the angle full score threshold, Q is the angle basic threshold, and s is the action angle extracted from the second skeleton point data; when calculating the time matching: P is the time full score threshold, Q is the time basic threshold, and s is the action duration extracted from the second skeleton point data; when calculating the count matching: P is the count full score threshold, Q is the count basic threshold, and s is the action count extracted from the second skeleton point data.

10. The method according to claim 9, wherein: The physical fitness assessment index includes at least one of the following: flexibility, stability, and endurance; The evaluation index is preset according to different fitness movements.

11. The method according to claim 10, wherein: The step of combining the matching degree with a pre-built physical fitness knowledge base to determine the user's physical condition includes: Obtaining the matching degree of each fitness action of the user during fitness; A matching matrix M is formed according to the body parts and evaluation indicators corresponding to each fitness action, wherein flexibility is measured by the average value of angle matching, stability is measured by time matching, and endurance is measured by time matching and / or count matching; For each action, multiply the matching matrix M by the difficulty matrix K of the action to obtain the physical fitness score matrix N for the action; The physical fitness score matrix N of each action is weighted averaged to obtain the physical condition of the user.

12. The method according to claim 11, wherein: The difficulty matrix K is obtained through a pre-trained knowledge base.

13. A device for outputting information, comprising: The playback unit is configured to recognize the identity of the user through the full-body image of the user, and to play the teaching video of the initial fitness program according to the identity of the user, wherein the identity includes at least one of the following: height, weight, figure, body health condition, and facial health condition; if the complexion of the user has not been recorded, the complexion of the user is compared with the average complexion of people of the same age and gender to determine whether it is abnormal; an acquisition unit, configured to acquire a fitness video in which a user performs fitness following the teaching video; an identification unit configured to identify the user's fitness movements in the fitness video, including positions, angles, and durations of key points of a human body; A matching unit, configured to determine the matching degree between the fitness action and the action in the teaching video, wherein the matching degree includes angle matching degree, time matching degree, and count matching degree; a determination unit configured to determine the user's physical condition by combining the matching degree with a pre-built physical fitness knowledge base, wherein the physical fitness knowledge base sets matching requirements corresponding to each body part in each action, including a matching type and a matching degree threshold; The recommendation unit is configured to update the teaching video of the fitness program suitable for the user according to the physical condition, if the matching degree of a certain part of the human body is positive, the movement difficulty of the part is increased, if the matching degree of a certain part of the human body is negative, the movement difficulty of the part is reduced.

14. An electronic device for outputting information, comprising: one or more processors; a storage device having one or more computer programs stored thereon, When the one or more computer programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 12.

15. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images