A fitness live broadcast interactive method and system

Abstract

The present invention discloses a method for interactive live fitness broadcast, including: showing a real-time video of a coach to a user during live broadcast, and obtaining the user's IMU data as first IMU data through an IMU sensor worn by the user; when the first IMU data is detected as instruction data, generating a feedback instruction according to the first IMU data and sending the feedback instruction to the coach end; the coach end generates feedback information according to the feedback instruction and displays it to the coach. The present invention discloses a method and system for interactive live fitness broadcast, which realizes interaction with the coach without traditional keyboard and mouse interaction by converting the user's action collection and recognition method into feedback instructions during live fitness broadcast, thereby improving the user experience; and only collecting the user's IMU data without collecting video information can also protect the user's privacy.

Description

Technical Field

[0001] The present invention relates to intelligent fitness technology, and in particular to a fitness live broadcast interactive method and system. Background Art

[0002] Live webcasting absorbs and continues the advantages of the Internet. It uses video to broadcast live online. It can publish content such as product displays, related meetings, background introductions, program evaluations, online surveys, dialogue interviews, and online training on the Internet. It uses the Internet's intuitiveness, speed, good presentation, rich content, strong interactivity, no geographical restrictions, and audience segmentation to enhance the promotion effect of the event site. After the live broadcast is completed, readers can continue to be provided with replays and on-demand broadcasts at any time, effectively extending the time and space of the live broadcast and maximizing the value of the live broadcast content.

[0003] However, when live streaming is applied to smart fitness, since fitness users are often in motion, interaction between them and live coaches is often difficult, which is not conducive to the user experience of fitness users. Summary of the invention

[0004] The technical problem to be solved by the present invention is that in live fitness broadcasts, the interaction between the fitness person and the live broadcast coach is often difficult to carry out, which is not conducive to the user experience of the fitness person. The purpose is to provide a method and system for interactive live fitness broadcasts to solve the above problem.

[0005] The present invention is achieved through the following technical solutions:

[0006] In one aspect, this embodiment provides a fitness live broadcast interaction method, including:

[0007] The real-time video of the coach is displayed to the user during the live broadcast, and the IMU data of the user is obtained as the first IMU data through the IMU sensor worn by the user;

[0008] When it is detected that the first IMU data is instruction data, generating a feedback instruction according to the first IMU data and sending the feedback instruction to the coaching end;

[0009] The coach terminal generates feedback information according to the feedback instruction and displays it to the coach.

[0010] In the prior art, the current online live broadcast technology interaction is achieved through direct interaction between users and terminal devices, and these interactions are mainly achieved through keyboard and mouse input. However, in fitness live broadcasts, since the users themselves are often in motion, when users want to provide feedback to the coach, they still need to operate the mouse and keyboard to provide information feedback, which is very inconvenient. For example, the coach explains a certain movement too quickly and needs the coach to repeat it. The coach explains a certain movement that is not understood and needs the coach to explain it in more detail. The user has something urgent to do (for example, he wants to go to the bathroom) and hopes that the coach can pause for a while. When the user is done (after going to the bathroom) and comes back, the coach is prompted to be ready and continue to start, etc.

[0011] When this embodiment is implemented, by obtaining the first IMU data of the user as the basis for issuing the instruction, the method of identifying the first IMU data as the feedback instruction can be a preset method, or it can be set by the user according to his own needs. For example, the pause gesture is set as a cross-handed action. When the user makes a cross-handed action, the first IMU data detected by the IMU sensor set on the user's hands can determine that the user has completed the cross-handed gesture, and generate a pause feedback instruction to be sent to the coach end.

[0012] In this embodiment, the feedback instruction includes at least one of all the information that the user needs to express to the coach, such as pause, restart, slow down, speed up, next action, previous action, etc.; similarly, if a feedback instruction includes at least two pieces of information that need to be expressed to the coach, it should also be within the scope of this embodiment. The present invention converts the user's action collection and recognition method into feedback instructions during fitness live broadcast, thereby realizing interaction with the coach without traditional keyboard and mouse interaction, thereby improving the user experience; and only collecting the user's IMU data without collecting video information can also protect the user's privacy.

[0013] Further, when it is detected that the first IMU data is instruction data, generating a feedback instruction according to the first IMU data and sending the feedback instruction to the coach end includes:

[0014] Determine whether a fitness action is currently being performed;

[0015] If the fitness action is not in progress, inputting the detected first IMU data into the first recognition model, and receiving the output result output by the first recognition model; judging whether the first IMU data is instruction data according to the output result;

[0016] If a fitness action is in progress, the difference between the first IMU data and the current fitness action is obtained, and it is determined whether the first IMU data is instruction data based on the difference.

[0017] Further, determining whether a fitness action is currently being performed includes:

[0018] The coach terminal obtains the coach's IMU data as second IMU data;

[0019] Inputting the second IMU data into a second recognition model configured on the coaching terminal, and receiving a judgment result output by the second recognition model;

[0020] It is determined whether the second IMU data is IMU data corresponding to the fitness action according to the determination result.

[0021] Further, if a fitness action is in progress, obtaining a difference between the first IMU data and the current fitness action, and determining whether the first IMU data is instruction data according to the difference includes:

[0022] When the second IMU data is IMU data corresponding to a fitness action, obtaining the IMU data corresponding to the current fitness action as the reference IMU data according to the judgment result;

[0023] Acquire a difference between the reference IMU data and the first IMU data as difference data;

[0024] Inputting the difference data into a third recognition model, and receiving recognition data output by the third recognition model;

[0025] Determine whether the first IMU data is instruction data according to the identification data.

[0026] Further, when it is detected that the first IMU data is instruction data, generating a feedback instruction according to the first IMU data and sending the feedback instruction to the coach end further includes:

[0027] If the first IMU data is determined to be instruction data according to the identification data, a feedback instruction is generated by retrieving an instruction table according to the difference data; the instruction table is a correspondence between the difference data and the feedback instruction.

[0028] In another aspect, this embodiment provides a fitness live interactive system, including:

[0029] The user terminal configured at the user is configured to display the real-time video of the coach to the user in the live broadcast, and obtain the IMU data of the user as the first IMU data through the IMU sensor worn by the user;

[0030] A coach terminal configured at the coach's location, configured to collect the real-time video of the coach;

[0031] When it is detected that the first IMU data is instruction data, the user terminal generates a feedback instruction according to the first IMU data and sends the feedback instruction to the coach terminal;

[0032] The coach terminal generates feedback information according to the feedback instruction and displays it to the coach.

[0033] Further, the coaching end determines whether a fitness action is currently being performed;

[0034] If the fitness action is not in progress, the user terminal inputs the detected first IMU data into the first recognition model, and receives the output result output by the first recognition model; and determines whether the first IMU data is instruction data according to the output result;

[0035] If a fitness action is in progress, the user terminal obtains the difference between the first IMU data and the current fitness action, and determines whether the first IMU data is instruction data based on the difference.

[0036] Further, when the coaching end determines whether a fitness action is currently being performed, the coaching end obtains the coach's IMU data as the second IMU data;

[0037] The coach terminal inputs the second IMU data into a second recognition model configured on the coach terminal, and receives a judgment result output by the second recognition model;

[0038] The coaching end determines whether the second IMU data is IMU data corresponding to the fitness action according to the determination result.

[0039] Further, when the second IMU data is IMU data corresponding to a fitness action, the coaching end obtains the IMU data corresponding to the current fitness action as the reference IMU data according to the judgment result;

[0040] The user terminal obtains the difference between the reference IMU data and the first IMU data as difference data;

[0041] The user terminal inputs the difference data into a third recognition model configured in the user terminal, and receives recognition data output by the third recognition model;

[0042] The user terminal determines whether the first IMU data is instruction data based on the identification data.

[0043] Furthermore, if the user terminal determines that the first IMU data is instruction data based on the identification data, the user terminal generates a feedback instruction by retrieving an instruction table based on the difference data; the instruction table is a correspondence between the difference data and the feedback instruction, and the instruction table is configured on the user terminal.

[0044] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0045] The present invention provides a fitness live broadcast interactive method and system. In the fitness live broadcast, the user's movements are collected and identified and converted into feedback instructions, so that the interaction with the coach can be carried out without traditional keyboard and mouse interaction, thereby improving the user experience; and only the user's IMU data is collected instead of the video information, so as to protect the user's privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0047] Figure 1 This is a schematic diagram of the steps of the method according to an embodiment of the present invention;

[0048] Figure 2 Schematic diagram of the system architecture of an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0050] Example

[0051] Please refer to Figure 1 , which is a flow chart of a fitness live broadcast interactive method provided by an embodiment of the present invention, wherein the fitness live broadcast interactive method can be applied to Figure 2 A fitness live broadcast interactive system, further, the fitness live broadcast interactive method can specifically include the contents described in the following steps S1-S3.

[0052] S1: showing the real-time video of the coach to the user in the live broadcast, and obtaining the IMU data of the user as the first IMU data through the IMU sensor worn by the user;

[0053] S2: when it is detected that the first IMU data is instruction data, generating a feedback instruction according to the first IMU data and sending the feedback instruction to the coaching end;

[0054] S3: The coach terminal generates feedback information according to the feedback instruction and displays it to the coach.

[0055] In the prior art, the current online live broadcast technology interaction is achieved through direct interaction between users and terminal devices, and these interactions are mainly achieved through keyboard and mouse input. However, in fitness live broadcasts, since the users themselves are often in motion, when users want to provide feedback to the coach, they still need to operate the mouse and keyboard to provide information feedback, which is very inconvenient. For example, the coach explains a certain movement too quickly and needs the coach to repeat it. The coach explains a certain movement that is not understood and needs the coach to explain it in more detail. The user has something urgent to do (for example, he wants to go to the bathroom) and hopes that the coach can pause for a while. When the user is done (after going to the bathroom) and comes back, the coach is prompted to be ready and continue to start, etc.

[0056] When this embodiment is implemented, by obtaining the first IMU data of the user as the basis for issuing the instruction, the method of identifying the first IMU data as the feedback instruction can be a preset method, or it can be set by the user according to his own needs. For example, the pause gesture is set as a cross-handed action. When the user makes a cross-handed action, the first IMU data detected by the IMU sensor set on the user's hands can determine that the user has completed the cross-handed gesture, and generate a pause feedback instruction to be sent to the coach end.

[0057] In this embodiment, the feedback instruction includes at least one of all the information that the user needs to express to the coach, such as pause, restart, slow down, speed up, next action, previous action, etc.; similarly, if a feedback instruction includes at least two pieces of information that need to be expressed to the coach, it should also be within the scope of this embodiment. The present invention converts the user's action collection and recognition method into feedback instructions during fitness live broadcast, thereby realizing interaction with the coach without traditional keyboard and mouse interaction, thereby improving the user experience; and only collecting the user's IMU data without collecting video information can also protect the user's privacy.

[0058] In one embodiment, when it is detected that the first IMU data is instruction data, generating a feedback instruction according to the first IMU data and sending the feedback instruction to the coach end includes:

[0059] Determine whether a fitness action is currently being performed;

[0060] If the fitness action is not in progress, inputting the detected first IMU data into the first recognition model, and receiving the output result output by the first recognition model; judging whether the first IMU data is instruction data according to the output result;

[0061] If a fitness action is in progress, the difference between the first IMU data and the current fitness action is obtained, and it is determined whether the first IMU data is instruction data based on the difference.

[0062] When implementing this embodiment, the inventor found in practice that since fitness users are often in motion during live broadcasts, IMU data will be continuously generated. When the IMU data generated by various coincidences is just identified to generate feedback instructions, the coach will misjudge the user's feedback. In this embodiment, first determine whether it is in fitness, that is, determine whether it is currently in fitness action. If it is not in fitness action, it can be performed by directly detecting the first IMU data; and if it is in fitness action, obtain the difference between the first IMU data and the current fitness action, and determine whether a certain feedback instruction has been issued by the difference. In this way, the feedback instruction issued by the user can be accurately identified during the user's fitness. It should be understood that in this embodiment, the determination of whether it is currently in fitness action can be achieved by detecting the coach's action, or by detecting the user's fitness course time. No matter which method is used, it should be considered to be within the scope of this embodiment.

[0063] In one embodiment, determining whether a fitness action is currently being performed includes:

[0064] The coach terminal obtains the coach's IMU data as second IMU data;

[0065] Inputting the second IMU data into a second recognition model configured on the coaching terminal, and receiving a judgment result output by the second recognition model;

[0066] It is determined whether the second IMU data is IMU data corresponding to the fitness action according to the determination result.

[0067] When this embodiment is implemented, judging whether the current fitness action is being performed is achieved by judging whether the coach is performing the fitness action. The second recognition model in this embodiment is trained by samples; the result output by the second recognition model can be a simple yes or no. In this case, a simple support vector machine can be used, and the output result can also be what action the current fitness coach is doing or not doing. According to the judgment result, it can be understood whether the current fitness process is being performed.

[0068] In one embodiment, if a fitness action is in progress, obtaining a difference between the first IMU data and the current fitness action, and determining whether the first IMU data is instruction data according to the difference includes:

[0069] When the second IMU data is IMU data corresponding to a fitness action, obtaining the IMU data corresponding to the current fitness action as the reference IMU data according to the judgment result;

[0070] Acquire a difference between the reference IMU data and the first IMU data as difference data;

[0071] Inputting the difference data into a third recognition model, and receiving recognition data output by the third recognition model;

[0072] Determine whether the first IMU data is instruction data according to the identification data.

[0073] In one embodiment, when it is detected that the first IMU data is instruction data, generating a feedback instruction according to the first IMU data and sending the feedback instruction to the coaching end further comprises:

[0074] If the first IMU data is determined to be instruction data according to the identification data, a feedback instruction is generated by retrieving an instruction table according to the difference data; the instruction table is a correspondence between the difference data and the feedback instruction.

[0075] When this embodiment is implemented, the difference data is identified by the third identification model. For example, the acquisition of the difference data includes:

[0076] Selecting a plurality of IMU data points having the largest difference from the reference IMU data from the first IMU data as a reference point set;

[0077] The IMU data corresponding to the IMU data points in the reference point set is used as the difference data.

[0078] In this embodiment, a specific solution for identifying the first IMU data is provided, in which the baseline IMU data and the first IMU data are compared, and the differences between different detection points are sorted, and multiple detection points with the largest differences are selected as data sources for the difference data.

[0079] For example, when a user is doing jumping jacks, he needs to pause for something and makes a pause gesture. Because human body movement is a continuous process, when the user makes the pause gesture, the user's lower limbs are still in a certain stage of the jumping jacks. At this time, the solution of this embodiment can detect the pause gesture, and send a pause instruction to the coach. In this way, this embodiment effectively improves the gesture detection accuracy during user movement and reduces misjudgment.

[0080] See also Figure 2 Based on the same inventive concept, a fitness live broadcast interactive system is also provided, the system comprising:

[0081] The user terminal configured at the user is configured to display the real-time video of the coach to the user in the live broadcast, and obtain the IMU data of the user as the first IMU data through the IMU sensor worn by the user;

[0082] A coach terminal configured at the coach's location, configured to collect the real-time video of the coach;

[0083] When it is detected that the first IMU data is instruction data, the user terminal generates a feedback instruction according to the first IMU data and sends the feedback instruction to the coach terminal;

[0084] The coach terminal generates feedback information according to the feedback instruction and displays it to the coach.

[0085] In one embodiment, the coach determines whether a fitness action is currently being performed;

[0086] If the fitness action is not in progress, the user terminal inputs the detected first IMU data into the first recognition model, and receives the output result output by the first recognition model; and determines whether the first IMU data is instruction data according to the output result;

[0087] If a fitness action is in progress, the user terminal obtains the difference between the first IMU data and the current fitness action, and determines whether the first IMU data is instruction data based on the difference.

[0088] In one embodiment, when the coach terminal determines whether a fitness action is currently being performed, the coach terminal obtains the coach's IMU data as the second IMU data;

[0089] The coach terminal inputs the second IMU data into a second recognition model configured on the coach terminal, and receives a judgment result output by the second recognition model;

[0090] The coaching end determines whether the second IMU data is IMU data corresponding to the fitness action according to the determination result.

[0091] In one embodiment, when the second IMU data is IMU data corresponding to a fitness action, the coaching end obtains the IMU data corresponding to the current fitness action as the reference IMU data according to the judgment result;

[0092] The user terminal obtains the difference between the reference IMU data and the first IMU data as difference data;

[0093] The user terminal inputs the difference data into a third recognition model configured in the user terminal, and receives recognition data output by the third recognition model;

[0094] The user terminal determines whether the first IMU data is instruction data based on the identification data.

[0095] In one embodiment, if the user terminal determines that the first IMU data is instruction data based on the identification data, the user terminal generates a feedback instruction by retrieving an instruction table based on the difference data; the instruction table is a correspondence between the difference data and the feedback instruction, and the instruction table is configured on the user terminal.

[0096] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0097] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0098] The units described as separate components may or may not be physically separated. As units, it is obvious that a person of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0099] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0100] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a grid device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0101] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A fitness live interactive method, characterized in that: include: The real-time video of the coach is displayed to the user during the live broadcast, and the IMU data of the user is obtained as the first IMU data through the IMU sensor worn by the user; When it is detected that the first IMU data is instruction data, generating a feedback instruction according to the first IMU data and sending the feedback instruction to the coaching end; The coach terminal generates feedback information according to the feedback instruction and displays it to the coach; When it is detected that the first IMU data is instruction data, generating a feedback instruction according to the first IMU data and sending the feedback instruction to the coach end includes: Determine whether a fitness action is currently being performed; If the fitness action is not in progress, inputting the detected first IMU data into the first recognition model, and receiving the output result output by the first recognition model; judging whether the first IMU data is instruction data according to the output result; If a fitness action is in progress, the difference between the first IMU data and the current fitness action is obtained, and it is determined whether the first IMU data is instruction data based on the difference.

2. A fitness live broadcast interactive method according to claim 1, characterized in that: Determining whether a fitness action is currently being performed includes: The coach terminal obtains the coach's IMU data as second IMU data; Inputting the second IMU data into a second recognition model configured on the coaching terminal, and receiving a judgment result output by the second recognition model; It is determined whether the second IMU data is IMU data corresponding to the fitness action according to the determination result.

3. A fitness live broadcast interactive method according to claim 2, characterized in that: If a fitness action is in progress, obtaining a difference between the first IMU data and the current fitness action, and determining whether the first IMU data is instruction data according to the difference includes: When the second IMU data is IMU data corresponding to a fitness action, obtaining the IMU data corresponding to the current fitness action as the reference IMU data according to the judgment result; Acquire a difference between the reference IMU data and the first IMU data as difference data; Inputting the difference data into a third recognition model, and receiving recognition data output by the third recognition model; Determine whether the first IMU data is instruction data according to the identification data.

4. A fitness live broadcast interactive method according to claim 3, characterized in that: When it is detected that the first IMU data is instruction data, generating a feedback instruction according to the first IMU data and sending the feedback instruction to the coaching end further includes: If the first IMU data is determined to be instruction data according to the identification data, a feedback instruction is generated by retrieving an instruction table according to the difference data; the instruction table is a correspondence between the difference data and the feedback instruction.

5. A fitness live interactive system, characterized in that: include: The user terminal configured at the user is configured to display the real-time video of the coach to the user in the live broadcast, and obtain the IMU data of the user as the first IMU data through the IMU sensor worn by the user; A coach terminal configured at the coach's location, configured to collect the real-time video of the coach; When it is detected that the first IMU data is instruction data, the user terminal generates a feedback instruction according to the first IMU data and sends the feedback instruction to the coach terminal; The coach terminal generates feedback information according to the feedback instruction and displays it to the coach; The coach determines whether a fitness action is currently being performed; If the fitness action is not in progress, the user terminal inputs the detected first IMU data into the first recognition model, and receives an output result output by the first recognition model; Determine whether the first IMU data is instruction data according to the output result; If a fitness action is in progress, the user terminal obtains the difference between the first IMU data and the current fitness action, and determines whether the first IMU data is instruction data based on the difference.

6. A fitness live interactive system according to claim 5, characterized in that: When the coaching end determines whether a fitness action is currently being performed, the coaching end obtains the coach's IMU data as the second IMU data; The coach terminal inputs the second IMU data into a second recognition model configured on the coach terminal, and receives a judgment result output by the second recognition model; The coaching end determines whether the second IMU data is IMU data corresponding to the fitness action according to the determination result.

7. A fitness live interactive system according to claim 6, characterized in that: When the second IMU data is the IMU data corresponding to the fitness action, the coach terminal obtains the IMU data corresponding to the current fitness action as the reference IMU data according to the judgment result; The user terminal obtains the difference between the reference IMU data and the first IMU data as difference data; The user terminal inputs the difference data into a third recognition model configured in the user terminal, and receives recognition data output by the third recognition model; The user terminal determines whether the first IMU data is instruction data based on the identification data.

8. A fitness live interactive system according to claim 7, characterized in that: If the user terminal determines that the first IMU data is instruction data based on the identification data, the user terminal generates a feedback instruction based on the difference data retrieved instruction table; the instruction table is the correspondence between the difference data and the feedback instruction, and the instruction table is configured on the user terminal.

Images