An interface switchable fitness system

Abstract

The present invention discloses a fitness system with switchable interfaces, the system comprising: a display operable to display multiple user interfaces, the user interfaces having at least one user interface object, the multiple user interfaces being switchable; each of the user interfaces comprising at least one page object, the page object being used to display at least one or at least one group of page information; a mirror, the mirror being used to display the image of the user on the mirror, the display showing the user interface to the user through the mirror. The invention relates to the fields of smart home and smart fitness, and provides a smart fitness mirror system with independent page switchability.

Description

Technical Field

[0001] The present invention relates to the fields of smart home and smart fitness, and in particular to a fitness system with switchable interfaces. Background Art

[0002] With the rapid development of economy, people's pace of life has accelerated. High-speed life and high-load work have caused various health problems for people living in cities. As a result, fitness has become a hot topic of concern. Smart fitness mirror is a new fitness product that integrates artificial intelligence, hardware content and services.

[0003] The current interface of smart mirrors mainly controls the selection or delivery of video content through mobile devices, and there is no independent interface system for fitness smart mirrors. Summary of the invention

[0004] In order to provide a smart fitness mirror system with independent pages that can be switched, the present invention provides an interface switchable fitness system, the system comprising:

[0005] A display, used to display a plurality of user interfaces, wherein the plurality of user interfaces can be switched; each of the user interfaces includes at least one page object, wherein the page object is used to display at least one or at least one group of page information;

[0006] A mirror is used to display the image of the user on the mirror, and the display displays the user interface to the user through the mirror.

[0007] Among them, the system can independently display user interfaces and can switch between user interfaces.

[0008] Preferably, the page information includes exercise course videos, visual data, operable interactive objects and interactive guides. Rich page information can provide users with rich functions and options.

[0009] Preferably, the user interface includes at least one main page, any of which can be switched to other user interfaces, or a specified operation can be completed on the main page. The main interface is usually the interface currently used by the user, and the user can switch to other desired interfaces through the main interface.

[0010] Preferably, the user interface also includes a page switching page, and the page information displayed by the page object of the page switching page includes at least one preview image, and the preview image corresponds to the other switchable user interfaces one by one. The user interface also includes a page switching page, and the user can use the preview image to preview the subsequent interface to facilitate the selection of the page when switching.

[0011] Preferably, the user interface also includes at least one exercise page, and the page information displayed by the page object of the exercise page includes at least one of exercise course videos, course playback information, accessory connection information and user physiological information.

[0012] Preferably, the user interface also includes a pre-installed function page and an application page accessed by a third party.

[0013] Preferably, the user interface also includes page animations, which include page switching animations and page interaction animations, wherein the page switching animations are used to produce dynamic effects when the page is operated or changed, and the page interaction effects are used to provide guidance to the user when interacting.

[0014] Preferably, the user interface includes an interactive object for entering a page switching page, and the user enters the page switching page by performing an interaction with the interactive object for entering a page switching page.

[0015] Preferably, the page switching page includes preview images of the first main page, the second main page, and the third main page and highlights one of the pages.

[0016] Preferably, the page switching page is provided with an interactive object for displaying return and confirmation of entry into the highlighted page, and the page switching page is provided with an interactive object for displaying switching of the highlighted page.

[0017] Preferably, the user interface includes page animations, and the page animations include page switching animations, and the page switching animations include page turning animations, returning to the previous level, and entering the next level animations.

[0018] The present invention also provides a reflective video display device (also referred to herein as a "smart fitness mirror" and an "interactive fitness system") that is configured to display video content such as pre-recorded or live workouts led by a trainer to a user and provide an interface that allows the user to interact and personalize the video content. The smart fitness mirror can be a networked device that is communicatively coupled to a content provider (e.g., a server, a cloud service) and / or a smart device (e.g., a smartphone, a tablet computer, a computer). The smart fitness mirror may include a display panel and a speaker to output video content and audio to the user. The smart fitness mirror may also include a camera and a microphone to capture video and audio of the user during a workout. Thus, the smart fitness mirror can enable two-way communication between the user and the trainer during a workout. In this way, the smart fitness mirror can provide the user with a convenient option to receive guided workouts while enabling a greater degree of personalization and personal guidance similar to workouts provided by a personal trainer or coach at a regular gym.

[0019] An example of a smart fitness mirror includes a communication interface for receiving a video image of a fitness trainer, a display operably coupled to the communication interface to display the video image of the fitness trainer, and a mirror disposed in front of the display to reflect an image of a person opposite the display. The mirror has a partially reflective portion to transmit the video image of the fitness trainer to the person opposite the display so that the video image of the fitness trainer appears superimposed on a portion of the image of the person.

[0020] An example of an interactive fitness method includes the following steps: (1) streaming fitness content to an interactive video system, the interactive video system including a mirror having a partially reflective portion and a display disposed on one side of the partially reflective portion; (2) displaying the fitness content to a user via the display and the partially reflective portion of the mirror; and (3) reflecting an image of the user using the mirror so that the image of the user is at least partially superimposed on the fitness content displayed via the display and the partially reflective portion of the mirror.

[0021] An example of a method for using a smart fitness mirror includes the following steps: when displaying fitness content to a user on a video display behind a partially transmissive mirror: (1) reflecting an image of the user using the partially transmissive mirror; (2) measuring the user's heart rate using a heart rate monitor attached to the user; (3) transmitting the heart rate from the heart rate monitor to an antenna operably coupled to the video display; (4) displaying the user's heart rate on the video display; and (5) displaying the user's target heart rate on the video display.

[0022] In order to add a personalized interface and a new interaction mode to the smart fitness mirror, optimize the interaction mode between the smart fitness mirror and the user, and increase the frequency of use of the smart fitness mirror in daily life, the present invention designs a set of operating system Launcher for the smart fitness mirror, which specifically includes: (1) GUI interface; (2) system control method for motion control; (3) system control method for voice control; (4) control method for multimodal fusion; (5) executable control instructions; (6) data acquisition management method; (7) some specific function management methods. The core of the present invention is to customize a set of systems and operation methods for the smart fitness mirror, so that the function of the smart fitness mirror is not only to watch fitness exercise videos, but also to serve as an important part of smart home to facilitate users to view more information and improve the use efficiency. At the same time, the customized operation method overcomes the disadvantage that the traditional smart fitness mirror needs to be controlled by touch and is easy to leave fingerprints on the partial reflective mirror surface. Since the core function of the smart fitness mirror is to allow users to compare their own mirrored movements when watching fitness exercise videos, fingerprints left on the partial reflective mirror surface have a greater impact on the function of the smart fitness mirror, resulting in most users being unwilling to use touch interaction, and using smart terminals to control the smart fitness mirror is not intuitive and troublesome. The method of motion control does not have these problems. Although gesture control is also used in some other electronic products, there have been breakthroughs in its application to smart fitness mirrors. Since the smart fitness mirror includes a partially reflective mirror, we can clearly see the actions we are doing when performing motion control, reducing the possibility of miscontrol and misoperation. At the same time, since motion recognition is the core function of the smart fitness mirror, the smart fitness mirror can achieve higher-precision complex motion recognition without additional cost, so that the control instructions that can be completed are much richer than those of existing gesture control. Finally, since the main function of the smart fitness mirror is exercise and fitness, applying motion control to the smart fitness mirror also needs to avoid miscontrol caused by fitness movements, which is also significantly different from traditional gesture control.

[0023] All combinations of the foregoing concepts and additional concepts discussed in more detail below (assuming such concepts are not mutually contradictory) may be expected to be part of the inventive subject matter disclosed herein. Specifically, all combinations of the claimed subject matter appearing at the end of this disclosure may be expected to be part of the inventive subject matter disclosed herein. Terms that are expressly adopted herein and that may also appear in any disclosure incorporated by reference should be given the meaning most consistent with the specific concepts disclosed herein.

[0024] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:

[0025] The present invention provides an independent intelligent fitness mirror system, which enriches the functions of the fitness mirror, improves the user experience of multiple user interface objects, and increases the user's enthusiasm and interest in fitness by switching the user interface having one or more user interface objects on the fitness mirror. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0027] Figure 1 is a block diagram of an exemplary smart fitness mirror;

[0028] Figure 2 is an exemplary GUI page on a smart fitness mirror;

[0029] Figure 3 is an exemplary GUI page on a smart fitness mirror;

[0030] Figure 4 is an exemplary GUI page on a smart fitness mirror;

[0031] Figure 5-a is an exemplary GUI page on a smart fitness mirror;

[0032] Figure 5-b is an exemplary GUI page on a smart fitness mirror;

[0033] Figure 6 is an exemplary GUI page on a smart fitness mirror;

[0034] Figure 7 is an exemplary GUI page on a smart fitness mirror;

[0035] Figure 8 is an exemplary GUI page on a smart fitness mirror;

[0036] Fig. 9 is an exemplary GUI page on a smart fitness mirror;

[0037] Fig.10 Schematic diagram of motion recognition of smart fitness mirror. DETAILED DESCRIPTION

[0038] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0039] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those within the scope of this description. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0040] Those skilled in the art should understand that, in the disclosure of the present invention, the orientation or position relationship indicated by the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. are based on the orientation or position relationship shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.

[0041] Thus, the present patent relates to an interactive mirror device (also referred to as a "smart fitness mirror" and an "interactive training system") and a method of using an interactive training device. The smart fitness mirror includes a display configured to display workout content (pre-recorded video or live stream) and an interface that enables a user to personalize a workout. In addition, the smart fitness mirror can allow a user and / or a coach to interact with each other during a workout in a manner similar to a conventional workout performed in a gym or small fitness studio in which the user and coach are in the same room (e.g., providing feedback to the coach on the pace of the workout, correcting the user's form during a particular fitness program).

[0042] An exemplary smart fitness mirror.

[0043] Figure 1An exemplary representation of a smart fitness mirror is shown. The smart fitness mirror may include a processor 110 for partially controlling the operation of various subcomponents in the smart fitness mirror and managing data flows (e.g., video content, audio from a coach or user, biometric feedback analysis) flowing into / out of the smart fitness mirror. The smart fitness mirror may include a display 120 for displaying video content, a graphical user interface (GUI) with which a user can interact and control the smart fitness mirror, biometric feedback data, and / or other visual content. A sensor 130 may be coupled to the processor 110 to collect user-related data. An antenna 140 may be coupled to the processor 110 to provide data transmission between the smart fitness mirror and another device (e.g., a remote control device, a biometric sensor, a wireless router). The antenna 140 may include a plurality of transmitters and receivers, each of which is customized for a specific frequency and / or wireless standard (e.g., Bluetooth, 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 2G, 3G, 4G, 4G LTE, 5G). The amplifier 150 may be coupled to the processor 110 to receive audio signals from the processor 110 to output subsequent sounds through the left speaker 152 and / or the right speaker 154 .

[0044] Smart fitness mirrors can also include Figure 1 Additional components not shown in the figure. For example, the smart fitness mirror may include a switch mode power supply (SMPS), a switch, and onboard memory and storage devices (non-volatile and / or volatile memory), including but not limited to a hard disk drive (HDD), a solid state drive (SDD), a flash memory, a random access memory (RAM), and a secure digital (SD) card. The onboard memory and / or storage device can be used to store firmware and / or software for the operation of the smart fitness mirror. As described above, the onboard memory and / or storage device can also be used to (temporarily and / or permanently) store other data, including but not limited to video content, audio, user video, biometric feedback data, and user settings. In another example, the smart fitness mirror may include various components for mounting and supporting the smart fitness mirror.

[0045] Antenna 140 may include multiple antennas, each antenna being used as a receiver and / or transmitter to communicate with various external devices, such as a user's smart terminal (e.g., computer, smartphone, tablet), an external sensor (e.g., heart rate belt, inertial sensor, body fat scale) and / or a remote server or cloud server for streaming or playing video content. Once again, antenna 140 may comply with various wireless standards, including but not limited to Bluetooth, 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 2G, 3G, 4G, 4G LTE and 5G standards.

[0046] The sensor 130 may include a plurality of sensors, and the plurality of sensors may be one or more of an image recognition sensor (camera), an infrared recognition sensor, or a voice recognition sensor (microphone).

[0047] The image recognition sensor (camera) in the smart fitness mirror can be used to obtain a video and / or static image of the user while the user is performing an activity (e.g., exercising). The user's video can then be shared with the coach to allow the coach to observe and provide guidance to the user during the exercise. The video can also be shared with other users of other smart fitness mirrors for comparison or competition. The user's video can also be displayed in real time on the display 120 or stored for later playback. For example, by providing a visual comparison between the user and the coach, the user's video can be used for self-assessment during or after exercise. The stored video can also allow the user to evaluate their progress or improvement over time when performing similar fitness. The image recognition sensor (camera) can also be configured to obtain a dynamic and / or static image of the user while the user is performing an activity (e.g., exercising) and process it into a skeleton point image, and determine whether the user's action matches the standard action based on the skeleton point image and output the judgment result, if the user's action does not match the standard action, then output the action correction information. The dynamic and / or static images can also be processed in real time when the user uses the smart fitness mirror or after the exercise is completed to control the smart fitness mirror based on the user's movement and action or export the user's biometric data.

[0048] The infrared recognition sensor in the smart fitness mirror can be used to obtain the user's thermal image when the user is doing activities (for example, exercising), and determine the user's exercised parts, force exertion parts and calories consumed according to the thermal image. It can also determine whether the user's movements match the standard movements according to the user's exercised parts and force exertion parts, and output the judgment result. If the user's movements do not match the standard movements, the movement correction information is output.

[0049] The voice recognition sensor (microphone) in the smart fitness mirror can be used to obtain and process the user's voice information in real time when the user uses the smart fitness mirror, so as to control the smart fitness mirror or communicate with the user based on the user's voice information.

[0050] Graphical User Interface (GUI)

[0051] The smart fitness mirror may display a graphical user interface (GUI) through the display 120 to facilitate user interaction with the smart fitness mirror.

[0052] The graphical user interface (GUI) includes but is not limited to a wake-up page, a main page, a workout page, and other pages.

[0053] The wake-up page refers to the initial page when the smart fitness mirror is awakened, which is used to display some basic page information. Only when it detects that the user performs a specified operation, the smart fitness mirror is completely awakened and enters the main page, exercise page or other pages.

[0054] The main page displays at least one page information, and the page information is used to display data, operable interactive objects or interactive guidance to the user. The main page is used to display preset page information or page information set by the user to the user, and the user can control the smart fitness mirror by performing specified operations on the main page, and the control includes but is not limited to page switching, function switching or entering third-party access applications.

[0055] The exercise page displays a video playback window for playing exercise videos to guide users to exercise, and can also display at least one of course playback information (such as total course duration, played duration, etc.), accessory connection information (such as heart rate belt connection information, etc.) and user physiological information (such as heart rate, blood pressure, calories consumed, etc.). The exercise page can be entered from the main page or the wake-up page and then returned to the main page or wake-up page, or can be directly entered when the smart fitness mirror is turned on.

[0056] Among them, other pages display pre-installed function pages (such as multimedia player page, weather display page, food recommendation page, etc.) or third-party access application pages (such as WeChat, Tik Tok, Himalaya, etc.) entered through the main page. Other pages are generally entered from the main page or the wake-up page and return to the main page or the wake-up page.

[0057] Page Information

[0058] A graphical user interface (GUI) can display page information through page objects. Each page object is used to display at least one or a group of page information. The displayed page information may be information stored in a local storage device of the smart fitness mirror, or information received by the smart fitness mirror from other devices through a communication interface, including but not limited to course preview information, user information, environmental information, multimedia information, external device information, etc. that can be switched for display.

[0059] The course preview information includes but is not limited to the name of the coach, course type, course duration, course difficulty, course label and / or estimated consumption, etc. The course type includes but is not limited to displaying HIIT, aerobics, yoga, strength shaping, combat training, Barre, Pilates, stretching, dance, meditation, challenges, etc. to users through a smart fitness mirror. The course label includes but is not limited to displaying to users through a smart fitness mirror whether it is an AI-recognized course, whether it is a parent-child course, the main body parts involved in the course (such as the whole body, arms, waist and abdomen, legs, etc.) and the props needed in the course (such as no equipment, yoga mats, elastic circles, elastic bands, dumbbells, etc.). The page object that displays the course preview information is also called a course card.

[0060] The user information includes but is not limited to usage history information, physiological information, social information, calendar information, etc. The usage history information includes but is not limited to displaying completed courses, favorite courses, historical best scores, frequency of classes, etc. to users through smart fitness mirrors. The physiological information includes but is not limited to displaying height, weight, gender, age, heart rate, blood pressure, injury history, etc. to users through smart fitness mirrors. The social information includes but is not limited to displaying information related to people / friends that users follow to users through smart fitness mirrors, and can also be information related to groups / interest groups that users join. This information can specifically be an invitation to invite users to complete a specified course, an invitation to invite users to a one-to-one or multi-person challenge, rankings between users, and dynamics uploaded by users themselves or other users. The dynamics can be synchronized to other social platforms (such as WeChat, Weibo, Facebook, Twitter, etc.). The calendar information includes but is not limited to displaying the user's schedule information, to-do items, course plans, etc. to users through smart fitness mirrors. The page object that displays user information is also called a user card.

[0061] The environmental information includes but is not limited to weather information. The weather information includes but is not limited to showing the weather forecast, temperature, humidity, clothing recommendations, etc. to the user through the smart fitness mirror. The page object showing weather information is also called a weather card.

[0062] The multimedia information includes but is not limited to music and video information played by the smart fitness mirror. The page object displaying the multimedia information is also called a multimedia card.

[0063] The external device information includes, but is not limited to, information of other fitness devices connected to the smart fitness mirror and other smart furniture (such as sweeping robots, smart speakers, smart gateways, etc.) displayed through the smart fitness mirror. The page object displaying the external device information is also called a device card.

[0064] The GUI can also display page information through interactive objects, which can be icons, text, highlighted selections, other forms that can prompt users to interact, or a combination thereof. The interactive objects are used to display to users the controls or operations that users can perform (such as page switching, selection, pause / play of courses, etc.). Furthermore, the interactive objects can also be combined with the pages or page objects of the GUI to display different interactive functions.

[0065] In addition to being fixed on each page, the page objects and interactive objects can also be displayed in the form of pop-up windows or scrolling playback on each page.

[0066] Some sample pages

[0067] The same plan Figure 2 , 3 , 4, 5-a, 5-b, where Figure 2 It is the first main page. Figure 3 It is a page switching page. Figure 4 , 5-a, 5-b are the second main page, the third main page and the fourth main page respectively. The first main page displays multiple page information through multiple page objects, including weather cards, multimedia cards and recommended course cards, etc. At the same time, the first main page also includes an interactive object for entering the page switching page and an interactive object combined with the course card displayed in the upper right corner. The user can enter the second main page by performing an interaction with the interactive object combined with the course card, or enter the page switching page by performing an interaction with the interactive object for entering the page switching page. The page switching page includes preview images of the first main page, the second main page and the third main page and highlights one of the pages. At the same time, the interactive objects for returning and confirming to enter the highlighted page are also displayed above the page switching page, and the interactive objects for switching the highlighted page are also displayed below the page switching page. The user can switch the highlighted page by performing an interaction with the interactive object for switching the highlighted page, and enter the selected page by confirming the interaction with the interactive object for entering the highlighted page. When the user selects and enters the second main page, the second main page includes multiple course cards and highlights one of the course cards. At the same time, the second main page also displays the interactive objects for returning and confirming to enter the course of the highlighted course card at the top and the interactive objects for switching to the highlighted course card at the bottom. The user can switch the highlighted course card by executing the interaction of switching the interactive objects of the highlighted course card, and enter the exercise page of the selected course card by executing the interaction of confirming to enter the interactive objects of the highlighted course card. When the user selects and enters the third main page, the third main page includes multiple page objects displayed in parallel. At the same time, the third main page also displays the selected page object and the interactive objects for entering the selected page object, and the interactive objects for switching page objects are also displayed at the bottom of the third main page. The user can switch the displayed page object by the interaction of the interactive objects of the switching page object, and enter the page corresponding to the selected page object by executing the interaction of entering the interactive objects of the selected page object.

[0068] Option 2 Figure 6 , 7 , 8, and 9, where Figure 6 It is the fifth main page. Figure 7 It is the sixth main page. Figure 8 , 9They are the seventh and eighth main pages respectively. The fifth main page displays multiple page information through multiple page objects, including weather cards, multimedia cards, and recommended course cards, etc. At the same time, the fifth main page also includes interactive objects for switching main pages and interactive objects combined with course cards displayed below. Users can enter the eighth main page by performing interactions with interactive objects combined with course cards, or switch the displayed main page by performing interactions with interactive objects for switching main pages. When the user chooses to switch the displayed main page to the sixth main page, the sixth main page includes multiple page objects displayed in parallel. At the same time, the sixth main page also displays the selected page object and the interactive object for entering the selected page object, and the interactive object for switching the main page is also displayed below the sixth main page. Users can switch the displayed main page by performing interactions with the interactive objects for switching main pages, and enter the page corresponding to the selected page object by performing interactions with the interactive objects for entering the selected page object. When the user chooses to switch the displayed main page to the seventh main page, the seventh main page includes a plurality of page objects displayed in parallel, and the seventh main page also displays the selected page object and the interactive object for entering the selected page object, and the interactive object for switching the main page is also displayed below the seventh main page. The user can switch the displayed main page by executing the interaction of the interactive object for switching the main page, and enter the page corresponding to the selected page object by executing the interaction of the interactive object for entering the selected page object. When the user chooses to enter the page corresponding to the selected page object to be the eighth main page, the eighth main page includes a plurality of course cards displayed in parallel, and the eighth main page also displays the selected course card and the interactive object for entering the selected course card, and the interactive object for switching the course card is also displayed below the eighth main page. The user can switch the displayed course card by executing the interaction of the interactive object for switching the course card, and enter the exercise page corresponding to the selected course card by executing the interaction of the interactive object for entering the selected course card.

[0069] The page switching page includes preview images of the first main page, the second main page, and the third main page, and highlights one of the pages. At the same time, the interactive objects for returning and confirming to enter the highlighted page are displayed at the top of the page switching page, and the interactive objects for switching the highlighted page are also displayed below the page switching page. The user can switch the highlighted page by executing the interaction of the interactive object for switching the highlighted page, and enter the selected page by confirming the interaction of the interactive object for entering the highlighted page. When the user selects and enters the second main page, the second main page includes multiple course cards and highlights one of the course cards. At the same time, the interactive objects for returning and confirming to enter the course of the highlighted course card are also displayed at the top of the second main page, and the interactive objects for switching the highlighted course card are also displayed below the page switching page. The user can switch the highlighted course card by executing the interaction of the interactive object for switching the highlighted course card, and enter the exercise page of the selected course card by executing the interaction of the interactive object for confirming to enter the course of the highlighted course card. When the user selects and enters the third main page, the third main page includes multiple page objects displayed in parallel. The third main page also displays the selected page object and the interactive object for entering the selected page object, and the interactive object for switching page objects is also displayed below the third main page. The user can switch the displayed page object by interacting with the interactive object for switching page objects, and enter the page corresponding to the selected page object by executing the interaction of the interactive object for entering the selected page object.

[0070] Page animation

[0071] The graphical user interface (GUI) also includes page animations, which include but are not limited to page switching animations and page interaction animations.

[0072] The page switching effects include page turning effects, returning to the previous level, entering the next level, etc. The page switching effects are mainly for page operations and changes, with the aim of providing a better user experience when page operations and changes are displayed.

[0073] Among them, page interaction effects include selection confirmation effects, action guidance effects, etc. Page interaction effects are mainly aimed at providing guidance to users when they are interacting, such as prompting users of the current interaction, the progress of the interaction, and prompting users of the operations they need to perform when interacting. Further details include, but are not limited to, highlighting or flashing the page information involved when the user is interacting, displaying a progress bar on the page information involved, etc. It also includes identifying the actions the user is doing and displaying prompt actions or suggested actions when performing action control. It also includes providing completion prompts based on the user's voice commands when performing voice control.

[0074] Smart Fitness Mirror Control

[0075] The user can directly interface with the smart fitness mirror to control the smart fitness mirror (e.g., motion commands, voice commands, touch commands, etc.). A graphical user interface (GUI) displayed on the display 120 may be provided to facilitate user interaction with the smart fitness mirror.

[0076] Action command control (including "posture control" and "gesture control")

[0077] The smart fitness mirror can control the graphical user interface (GUI) based on user movements captured by the image recognition sensor.

[0078] Specifically, the actions controlled by the action command include but are not limited to static actions and dynamic actions. Static actions refer to the user making a specific posture with a specific body part and maintaining it for a specified time; dynamic actions refer to the user making a specific movement with a specific body part.

[0079] Specifically, static actions include but are not limited to raising the left / right hand, raising the left / right leg, raising both hands above the head and making a heart shape, and various static gestures, etc., and the range of maintaining a specified time is generally 0-5 seconds or longer. As long as the action is determined by the positional relationship between a specific part of the human body relative to space, the human body as a whole, or other specific parts, and is maintained for a specified time, it should fall within the protection scope of static actions. Further specific examples include: straightening the left hand at an angle of 15° / 30° to the horizontal plane for 1.5 seconds, holding the left hand at a right angle to the right hand for 1 second, and extending the left hand forward for 0.1 seconds.

[0080] Specifically, dynamic actions include but are not limited to horizontal sliding of the left / right hand, sliding the left / right hand up and down, clapping hands, jumping up, squatting, and various dynamic gestures, etc. Any action determined by the movement trajectory of a specific part of the human body relative to space, the human body as a whole, or other specific parts should fall within the protection scope of dynamic actions. Further specific examples include: horizontally sliding the left hand to move the left hand from the left side of the body to the right side of the body, horizontally sliding the left hand to move the left hand from the right side of the body to the left side of the body, horizontally sliding the right hand to move the right hand from the right side of the body to the left side of the body, sliding the left hand up and down to move the left hand from above the shoulder to below the waist, sliding the left hand up and down to move the left hand from above the upper 20% of the human body to below the upper 40%, clapping hands above the head, clapping hands above the left shoulder, etc.

[0081] Specifically, the control instructions executed by the action command control include but are not limited to the control instructions for non-exercise pages (wake-up page, main page, other pages) and the control instructions for the exercise page. The control instructions for non-exercise pages mainly involve the user's interaction instructions with non-exercise pages. The control instructions for the exercise page mainly involve the user's control instructions for playing exercise courses.

[0082] Specifically, the control instructions for non-exercise pages include but are not limited to switching non-exercise pages, previous page, next page, confirm, return, course selection, course collection, adding to plan, entering the page switching interface, waking up the voice assistant, multimedia volume control, etc.

[0083] Specifically, the control instructions for the exercise page include but are not limited to course pause, course play, previous link, next link, course background music volume control, course coach volume control, course evaluation, etc.

[0084] Specifically, the user can also achieve different functions through a combination of multiple actions. Specifically, it includes but is not limited to using the first action to enter the first-level control instruction menu, and then using the second action to select an instruction in the first-level control instruction menu or enter the second-level control instruction menu, and the control instruction menu can be displayed to the user on the display. Further specific includes but is not limited to the first-level control instruction menu can be a course selection control instruction menu, including playing courses, collecting courses, displaying course details, booking courses and entering the second-level control instruction menu instructions, the second-level control instruction menu can be a course evaluation menu, including good reviews, medium reviews, and bad reviews.

[0085] Specifically, it also includes prompting the user's action command control in the graphical user interface (GUI). Specifically, it includes but is not limited to displaying icons of operable actions in the graphical user interface (GUI), marking or highlighting the control instructions pointed to by the user's static actions in the graphical user interface (GUI), displaying a progress bar for the duration of the user's static actions in the graphical user interface (GUI), and guiding the user's dynamic actions in the graphical user interface (when the completion degree of the user's dynamic actions exceeds a specified ratio).

[0086] Specifically, the smart fitness mirror can rely on other sensors to provide auxiliary judgment when controlling the graphical user interface (GUI) based on the user actions collected by the image recognition sensor. Specifically, it includes but is not limited to starting / interrupting / confirming action command control through voice recognition sensors, judging the front or back of the user through facial recognition sensors (action command control is only enabled when the user faces the smart fitness mirror), distinguishing different users through facial recognition sensors to load customized action command control instructions, and optimizing action / trajectory recognition accuracy through IMU, etc.

[0087] Specifically, the action controlled by the action command can be an action pre-set in the smart fitness mirror, or an action recorded by the user. The control instruction executed by the action command control can be an instruction pre-set in the smart fitness mirror, or an automated instruction set by the user. The action controlled by the action command and the executed control instruction can be a pre-set correspondence relationship, or a correspondence relationship set by the user, and can be a one-to-one relationship or a many-to-one relationship.

[0088] In addition, control actions correspond to control instructions. Matching a control action means that the processor executes the control instruction that matches the control action.

[0089] Specifically, the turning on or off of the motion command control function can be determined by a specific graphical user interface (GUI) or exercise page, such as the motion command control function is turned on by default on the main page of the graphical user interface (GUI) and turned off by default on other pages; the turning on or off of the motion command control function can also be determined based on data detected by other sensors; the turning on or off of the motion command control function can also be the turning on / off of all functions or the turning on / off of some functions.

[0090] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) according to user actions collected by an image recognition sensor. The smart fitness mirror first displays a first page of the graphical user interface (GUI) through the display 120. The first page displays a first prompt icon for raising the left hand to enter a page switching mode. The user raises his left hand, the first prompt icon enters a highlighted state and turns on a progress bar. After 1.5 seconds, the progress bar is completed, and the smart fitness mirror displays a page switching page for switching page display through the display 120. In addition to the highlighted first page, the switching page also includes the previous page of the first page "second page" and the next page "third page" of the first page, the switching page also displays prompt icons for the previous page, next page, exit and confirmation, wherein the prompt icon for the previous page represents the action of waving the left hand horizontally from left to right, the prompt icon for the next page represents the action of waving the right hand horizontally from right to left, the prompt icon for exit represents raising the left hand, and the prompt icon for confirmation represents raising the right hand. At this time, the user waves the left hand horizontally from left to right, the switching page changes to highlighting the second page, and the user raises his right hand at this time, the icon representing confirmation enters the highlighted state and turns on the progress bar. After 1.5 seconds, the progress bar is completed, and the smart fitness mirror displays the second page through the display 120.

[0091] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) based on user actions captured by an image recognition sensor is that the smart fitness mirror first displays the fourth page of the graphical user interface (GUI) through the display 120. The fourth page displays a second prompt icon representing a horizontal wave of the left hand from left to right to enter the previous page and a third prompt icon representing a horizontal wave of the right hand from right to left to enter the next page. At this time, the user makes a horizontal wave of the right hand from right to left, and the smart fitness mirror displays through the display 120 that the fourth page switches to the fifth page.

[0092] In a specific example, a smart fitness mirror can control a graphical user interface (GUI) according to user actions collected by an image recognition sensor. The smart fitness mirror first displays the following information on the display 120: Figure 5-a The third main page includes multiple page objects displayed in parallel. At the same time, the third main page also displays the selected page object and the interactive object for entering the selected page object, and the interactive object for switching page objects is also displayed below the third main page. The user can switch the displayed page object through the interactive object of switching page objects, and enter the page corresponding to the selected page object by executing the interactive object of entering the selected page object. The user can select the selected page object by raising the right / left hand to the upper left, lower left, upper right, or lower right, and maintain the posture of the right / left hand after the selection is completed, and make the hand perform a specified static action such as opening the five fingers for 1 second to enter the page corresponding to the selected page object. At the same time, the user can make a horizontal waving action of the right hand from left to right to switch the displayed page object and enter the fourth main page.

[0093] Further, a specific example of controlling the suspension action of the smart fitness mirror is as follows: the smart fitness mirror first displays the following through the display 120: Figure 5-aThe third main page includes page objects arranged in four grids and displayed in parallel. When the user uses the smart fitness mirror, the page object selected by the user is determined according to the position of the user's right hand, that is, when the user's right hand is above the right shoulder of the human body, the page object "weather" is selected, when the user's right hand is below the right shoulder of the human body, the page object "reminder" is selected, when the user's right hand is above the left shoulder of the human body, the page object "music" is selected, and when the user's right hand is below the right shoulder of the human body, the page object "body fat scale" is selected. When the user's right hand moves, the smart fitness mirror determines the page object selected by the user according to the identified right hand position. When the right hand stops moving for more than a certain period of time, the preset interaction for the selected page object is performed. The preset interaction for the selected page object can be a function to open the page object, or it can be a contextual menu of the page object. The page objects arranged in four grids and displayed in parallel can also be arranged in left-right, six-grid, nine-grid, and other arrangements.

[0094] The actions can be set according to personal preferences (left-handedness, disability, etc.), and have different modes for one hand and two hands, including left / right hand mode, and two-handed mode (such as left hand selection and right hand screen switching).

[0095] The first page, the second page, the third page, the fourth page and the fifth page may be a wake-up page, a main page, a workout page or other pages, respectively.

[0096] Voice command control

[0097] The smart fitness mirror can control the graphical user interface (GUI) according to the user's voice collected by the voice recognition sensor.

[0098] Specifically, the user can use the voice recognition result of the voice command control to replace the action of the action command control and achieve the same control effect.

[0099] Specifically, the control instructions executed by the action command control include but are not limited to the control instructions for non-exercise pages (wake-up page, main page, other pages) and the control instructions for the exercise page. The control instructions for non-exercise pages mainly involve the user's interaction instructions with non-exercise pages. The control instructions for the exercise page mainly involve the user's control instructions for playing exercise courses.

[0100] Specifically, the control instructions for non-exercise pages include but are not limited to switching non-exercise pages, previous page, next page, confirm, return, course selection, course collection, adding to plan, entering the page switching interface, waking up the voice assistant, multimedia volume control, etc.

[0101] Specifically, the control instructions for the exercise page include but are not limited to course pause, course play, previous link, next link, course background music volume control, course coach volume control, course evaluation, etc.

[0102] Specifically, the user can also achieve different functions through a combination of multiple actions. Specifically, it includes but is not limited to using the first voice to enter the first-level control command menu, and then using the second voice to select commands in the first-level control command menu or enter the second-level control command menu, and the control command menu can be displayed to the user on the display. Further specific includes but is not limited to that the first-level control command menu can be a course selection control command menu, including playing courses, collecting courses, displaying course details, booking courses and entering the second-level control command menu. The second-level control command menu can be a course evaluation menu, including good reviews, medium reviews, and bad reviews.

[0103] Specifically, it also includes prompting the user's voice command control in the graphical user interface (GUI). Specifically, it includes but is not limited to displaying the keywords of the available voice in the graphical user interface (GUI), completing the user's voice command in the graphical user interface (when the completion degree of the user's dynamic action exceeds a specified ratio), etc.

[0104] Specifically, the smart fitness mirror can rely on other sensors to provide auxiliary judgment when controlling the graphical user interface (GUI) based on the user voice collected by the voice recognition sensor. Specifically, it includes but is not limited to starting / interrupting / confirming motion command control through motion recognition sensors, judging the front or back of the user through facial recognition sensors (motion command control is enabled only when the user faces the smart fitness mirror), distinguishing different users through facial recognition sensors to load customized voice command control instructions, etc.

[0105] Specifically, the keywords of the voice command control may be keywords pre-set in the smart fitness mirror, or keywords recorded by the user. The control instructions executed by the voice command control may be instructions pre-set in the smart fitness mirror, or automated instructions set by the user. The keywords of the voice command control and the executed control instructions may be pre-set corresponding relationships, or may be corresponding relationships set by the user, and may be one-to-one relationships, or may be many-to-one relationships.

[0106] Specifically, the voice command control function can be turned on or off by a specific graphical user interface (GUI) or exercise page, such as the voice command control function is turned on by default on the main page of the graphical user interface (GUI) and the motion command control function is turned off by default on other pages; the voice command control function can also be turned on by recognizing wake-up words; the voice command control function can also be turned on or off based on data detected by other sensors; the voice command control function can also be turned on / off for all functions or on / off for some functions.

[0107] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) based on user actions collected by an image recognition sensor, the smart fitness mirror first displays the first page of the graphical user interface (GUI) through the display 120, the user turns on the voice command control function through a wake-up word, and an icon representing the voice command control function is displayed on the fourth page. The user then says the command "enter the switching page" or "page switch", and the smart fitness mirror displays a page switching page for switching page display through the display 120. In addition to the highlighted first page, the switching page also includes the previous page "second page" of the first page and the next page "third page" of the first page. The switching page also displays an icon representing the voice command control function is displayed. The user then says the command "previous page" or "turn forward one page", and the switching page changes to highlight the second page. At this time, the user says the command "enter" or "confirm", and the smart fitness mirror displays the second page through the display 120.

[0108] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) based on user actions captured by an image recognition sensor is that the smart fitness mirror first displays the fourth page of the graphical user interface (GUI) through the display 120. The user turns on the voice command control function through a wake-up word. An icon representing the voice command control function is displayed on the fourth page. At this time, the user says the command "next page" or "turn to the previous page", and the smart fitness mirror displays the transition from the fourth page to the fifth page through the display 120.

[0109] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) based on user actions captured by an image recognition sensor. The smart fitness mirror first displays the sixth page of the graphical user interface (GUI) through the display 120. The user turns on the voice command control function through action command control. An icon representing the voice command control function is displayed on the fourth page. The user then says the command "start training" or "OK", and the smart fitness mirror displays the display 120 to switch from the sixth page to the seventh page.

[0110] The first page, the second page, the third page, the fourth page and the fifth page may be a wake-up page, a main page, a workout page or other pages respectively; the sixth page is a page for displaying workout cards; and the seventh page is a workout page.

[0111] Touch command control

[0112] The smart fitness mirror can control the graphical user interface (GUI) according to the user touch instructions collected by the touch screen.

[0113] Multimodal command control

[0114] Multimodal command control includes combined control, enhanced control and conflict control.

[0115] The combination control includes but is not limited to action combination control of multiple actions and voice combination control of multiple keywords. More complex control instructions (such as selecting and collecting, selecting and recommending to friends, taking screenshots and sharing, etc.) can be achieved through multiple consecutive actions or keywords (the interval time does not exceed the threshold).

[0116] The combined control includes but is not limited to enhancing and optimizing the user's motion command control or voice command control through facial recognition or scene recognition, and further includes but is not limited to recognizing that the user's facial expression is unhappy when the user's motion command control or voice command control is playing music, presenting the user with soothing recommendation results while showing care for the user; recognizing that the scene temperature is low when the user's motion command control or voice command control is recommending courses, reducing recommendations of meditation courses.

[0117] The conflict control includes but is not limited to when a user uses multiple different control methods to control the smart fitness mirror at the same time, multiple conflicting control instructions appear, and the control instructions that the user wants to execute are determined and executed by the smart fitness mirror by making priority judgments on different control methods.

[0118] Control instructions

[0119] The control instructions include, but are not limited to, system wake-up instructions, page switching instructions, page interaction instructions, and course control instructions, and users can control the smart fitness mirror through these control instructions.

[0120] The system wake-up instructions include instructions for waking up the smart fitness mirror from a standby state to a wake-up page, a main page, an exercise page, or other pages, and instructions for switching the smart fitness mirror from a wake-up page to a main page, an exercise page, or other pages.

[0121] The page switching instruction refers to an instruction for controlling the smart fitness mirror to switch between multiple different pages, and the different pages may be pages of different types or pages of different contents. Further page switching instructions include but are not limited to page turning instructions (next page / previous page) and instructions for controlling the smart fitness mirror to enter a page switching page.

[0122] Among them, page interaction instructions refer to instructions used to control the smart fitness mirror to perform interaction or input, and the page interaction instructions include but are not limited to control interaction instructions and input interaction instructions. Further, the control interaction instructions refer to instructions for controlling the graphical user interface (GUI) or the page objects or interactive objects in the graphical user interface (GUI) to realize their interactive functions, such as confirmation / return, adding to favorites, adding to a plan, waking up a voice assistant, volume control, switching accounts, switching modes, opening a contextual menu, etc. Further, the input interaction instructions refer to instructions for inputting information in the graphical user interface (GUI) or the page objects or interactive objects in the graphical user interface (GUI), such as adding a heart for each instruction entered when evaluating a course; or switching a preset expression / reply for each instruction entered when chatting with other users through a smart fitness mirror, etc.

[0123] The course control instructions are used to control the smart fitness mirror to play exercise videos on the exercise page, and are mainly used to control the playback of exercise videos. Further course control instructions include but are not limited to pause / play, previous / next section, background volume control, coach volume control, course evaluation, etc.

[0124] Data Management

[0125] The data sources of the smart fitness mirror include but are not limited to sensor data and background data, and the sensor data and background data are used for inputting user control instructions and displaying page information.

[0126] The sensor data includes but is not limited to the on-mirror sensor data and the external sensor data. The on-mirror sensor data includes but is not limited to the data detected by sensors installed on the smart fitness mirror, such as image recognition sensors, infrared recognition sensors and voice recognition sensors. More specifically, the data detected by the image recognition sensor includes but is not limited to the image detected by the image recognition sensor and the data obtained after the image processing and recognition, such as user action data, user posture data, facial recognition data, user expression data and environmental image data. More specifically, the data detected by the infrared recognition sensor includes but is not limited to the thermal imaging image detected by the infrared recognition sensor and the data obtained after the thermal imaging image processing and recognition, such as user thermal imaging data, user exercise part data, environmental thermal imaging data, etc. More specifically, the data detected by the voice recognition sensor includes but is not limited to the voice data detected by the sound recognition sensor and the data obtained after the voice data processing and recognition, such as voice control instructions, keyword instructions, wake-up word instructions, etc. The on-mirror sensor data includes but is not limited to inertial sensor data, physiological data sensor data (such as heart rate belt data, body fat scale data, etc.), and other data detected by sensors connected to the smart fitness mirror wirelessly. More specifically, the data detected by the inertial sensor includes but is not limited to the speed / acceleration data detected by the inertial sensor and the data obtained after processing and identifying the speed / acceleration data, such as user motion data, etc. More specifically, the data detected by the physiological data sensor includes but is not limited to the physiological data detected by the physiological data sensor and the data obtained after processing and identifying the physiological data, such as heart rate data, blood pressure data, blood oxygen saturation data, etc.

[0127] The background data includes but is not limited to background user data and other background data. The background user data is data directly associated with the user stored in the smart fitness mirror and / or the server connected to the smart fitness mirror. The background user data includes but is not limited to user physiological data, user social data, user calendar data and usage history data. The usage history data includes but is not limited to the user's completed course data, collection of course data, historical best performance data, physiological data and performance data during the course, class frequency data, etc. The user physiological data includes but is not limited to the user's height, weight, gender, age, heart rate, blood pressure, injury history, etc. The user social data includes but is not limited to data related to the people / friends followed by the user and data related to the groups / interest groups joined by the user. This information can specifically be class invitation data for inviting users to complete designated courses, invitation data for inviting users to one-on-one or multi-person challenges, rankings between users, and dynamic data uploaded by users themselves or other users. The dynamics can be synchronized to other social platforms (such as WeChat, Weibo, Facebook, Twitter, etc.). The user calendar data includes but is not limited to the user's schedule information data, to-do data, course plan data, etc. The other background data is data with low correlation with users stored in the smart fitness mirror and / or the server connected to the smart fitness mirror, and the other background data includes but is not limited to IP data, environmental data, course data and third-party data. Among them, IP data includes but is not limited to the IP address data and Mac address data of the smart fitness mirror. Among them, environmental data includes but is not limited to the time data, positioning data, weather data, temperature data, clothing index data, etc. of the area where the IP of the smart fitness mirror is located. The course data includes but is not limited to exercise video data, coach name, course type, course duration, course difficulty, course label and / or expected consumption, etc. The course type includes but is not limited to showing HIIT, aerobic dance, yoga, strength shaping, combat training, Barre, Pilates, stretching, dance, meditation, challenge, etc. to users through the smart fitness mirror. The course label includes but is not limited to showing users through the smart fitness mirror whether it is an AI recognition course, whether it is a parent-child course, the main body parts involved in the course (such as the whole body, arms, waist and abdomen, legs, etc.) and the props needed in the course (such as no equipment, yoga mat, elastic circle, elastic band, dumbbell, etc.). The third-party data includes but is not limited to data of third-party functions installed on the smart fitness mirror.

[0128] Data storage

[0129] The smart fitness mirror can also store user information locally on the smart fitness mirror and / or a remote storage device (e.g., a cloud service) depending on the amount of storage space used. For example, user information that uses very little storage space can be stored locally on the smart fitness mirror, including but not limited to the user's name, age, height, weight, and gender. In addition, course data can also be stored in the smart fitness mirror to reduce the impact of network latency that may affect the quality of the video stream. The amount of video content stored may be limited by the storage capacity of the smart fitness mirror. In some configurations, video content may be temporarily stored only daily or weekly, or depending on the percentage of the capacity of the smart fitness mirror used. Background user data that uses a large amount of storage space can be stored on a remote storage device, including but not limited to physiological data, such as the user's heart rate and calories consumed, and videos or skeletal point data of the user taken during exercise. The smart fitness mirror can retrieve this information for subsequent analysis and display.

[0130] Data transmission between the smart fitness mirror and the remote storage device can be protected (e.g., encrypted) in various ways to prevent unwanted loss or theft of user information. For example, the low-power Bluetooth protocol includes built-in security features that can be used by devices that utilize the protocol. However, these security features can only be used if the Bluetooth binding step is completed before the connection is established with encryption. In some cases, various security mechanisms may not be implemented or various security mechanisms may fail, and application-level security can be implemented in conjunction with the block specification of the above data. For example, Advanced Encryption Standard (AES) encryption of the message can be applied before the preamble of the preamble message. In some aspects, the low-power Bluetooth protocol performs a similar process via built-in security features at the firmware level, and similar protection can be provided to prevent personnel from reading communications between the client and the server.

[0131] When a client disconnects from a server, the GATT service added for the client to read / notify messages can be removed from the service record on the server device. This ensures that no connections are left open and the system does not accidentally leak information to nefarious snoopers. This connection termination can be triggered by either the server or the client and relies on the Bluetooth low energy stack to provide notification to both parties that the connection has been closed. If Bluetooth bonding is used to provide firmware-level cryptographic security during the initial connection setup, the bonding information can be stored on each device so that bonding does not need to be repeated after subsequent connections between the client and server.

[0132] Feature Management

[0133] The smart fitness mirror can also realize the following functions based on the received data: user identification, mode switching, attention judgment, physical / emotional assessment, and clothing recommendations.

[0134] The user identification function allows the smart fitness mirror to identify the user who is using the smart fitness mirror through the input data. This function is mainly implemented by the following method: the user inputs the user's characteristic data through the data input device, and the smart fitness mirror compares the characteristic data input by the user with the characteristic data in the database, determines the user who is using the smart fitness mirror according to the comparison result, and allows the smart fitness mirror to adjust the graphical user interface (GUI) according to the identification result, such as adjusting the font size, page information preference, etc. The data input device includes but is not limited to the sensor data on the mirror and the external sensor, and the user's characteristic data includes but is not limited to the user's bone points, voiceprint, weight, resting heart rate and other mirror sensor data and the data collected or processed by the external sensor after collection, and the mapping relationship between the user and the characteristic data is recorded in the database. Further, the user identification function can be used for the identification and switching of multiple sub-accounts under a main account, and can also be used for the identification and switching between multiple main accounts. In another example, the user identification function allows the smart fitness mirror to identify the characteristics of the user who is using the smart fitness mirror through the input data, such as male users, female users, elderly people, children, and allows the smart fitness mirror to adjust the graphical user interface (GUI) according to the identification result, such as adjusting the font size, page information, etc.

[0135] Among them, the mode switching function allows the smart fitness mirror to switch between multiple working modes to adapt to different working scenarios. For example, the smart fitness mirror is allowed to switch to the youth mode. In the youth mode, the youth account is bound to the parent account or the main account of the device. In the youth mode, the page information displayed by the smart fitness mirror will be adaptively adjusted, such as the course recommendation will be adjusted to give priority to the presentation of content suitable for teenagers, such as skipping, somatosensory games, etc. At the same time, the real-time situation and motion data in this mode can be synchronized to the smart terminal of the parent account or the main account of the device. In another example, the smart fitness mirror is allowed to switch to the guest mode. In the guest mode, the smart fitness mirror can work without being bound to the account. The page information displayed by the smart fitness mirror can also be adaptively adjusted, such as the course recommendation will be adjusted to give priority to the presentation of content suitable for novices, such as content with a low difficulty label, etc. At the same time, even if the smart fitness mirror is bound to the account, the motion data in this mode will not be counted in the bound account, and will not affect the training plan or completion of the bound account.

[0136] Among them, attention judgment allows the smart fitness mirror to determine whether the user's attention is on the smart fitness mirror when the device is working, so that the smart fitness mirror can judge the user's intention to interact with the mirror and activate or shut down sensors, other digital input devices and functions. Attention judgment is mainly based on the user's performance to determine whether the user wants to use the smart fitness mirror. It can be judged by but not limited to the following features: distance judgment (the distance between the user and the smart fitness mirror does not exceed the threshold), posture judgment (whether the user is facing the smart fitness mirror), action judgment (whether the user has made a specified action or a common exercise action), semantic judgment (whether the recognition result of the user's voice is related to a specific keyword), etc.

[0137] The physical / emotional assessment allows the smart fitness mirror to determine the physical / emotional condition of the user when using the device, so that the smart fitness mirror can adjust the page information or other feedback according to the user's physical / emotional condition. The user's physical / emotional condition can be identified by physiological data sensors and image recognition sensors (facial recognition). When the user's mood is identified as unhappy, the user is presented with soothing page information recommendation results and the user is cared for during voice interaction; in a further example, when the user wakes up the voice function of the smart fitness mirror through the wake-up word, the smart fitness mirror recognizes that the user's expression is relatively depressed through the image recognition sensor (facial recognition), and the smart fitness mirror initiates the first round of dialogue "Hey, what's wrong? You don't look very happy, do you want to listen to a song to relax?" The identification of the physical / emotional condition can be determined by identifying the user's age, body posture, expression and other information to determine the user's physical / emotional state such as emotion, stress, fatigue, etc.; it can also be identified through blood pressure and heart rate data detected by the physiological data sensor.

[0138] Among them, clothing recommendation allows the smart fitness mirror to initiate active interaction with the user based on specific conditions. The smart fitness mirror determines whether the user needs the smart fitness mirror to initiate active interaction based on the collected time data and positioning data, combined with user information and the user's actions in front of the smart fitness mirror. For example, when the user first appears in front of the smart fitness mirror in the morning and turns around, the smart fitness mirror turns on the clothing recommendation function and recommends to the user clothes that are suitable for today's wear at the current location. Furthermore, when the user appears in front of the smart fitness mirror on the weekend and turns around, the smart fitness mirror turns on the clothing recommendation function and outputs the user's current clothing matching suggestions. The above functions can also be used for other functions of the smart fitness mirror that actively interact with users and external information.

[0139] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the multimodal interactive fitness mirror are implemented.

[0140] The processor may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0141] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the multimodal interactive fitness mirror device in the invention by running or executing the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0142] Video streaming method

[0143] The smart fitness mirror is configured to display video content from a video production factory on the display surface 120. The video content can be streamed as live content or pre-recorded recorded content. The live content can also be recorded and stored in a cloud server so that users can request and play the video content later, thus becoming recorded content. When streaming the video, the smart fitness mirror first sends a request for a specified course to the API server. The data returned by the API server is divided into two parts. The first part is an overview of the specified course (including course pictures and course introduction); the second part is the address (URL) of the course video, and then the smart fitness mirror requests the corresponding course video on the OSS server according to the received URL. When the course is recorded content, the smart fitness mirror uses the HLS protocol for video playback, that is, the address (URL) of the course video is the M3U8 file of the corresponding course, and then the player of the smart fitness mirror downloads the corresponding course video slice from the OSS server according to the record of the M3U8 file. When the course is live content, the smart fitness mirror uses the RTMP protocol to play the video, that is, the address (URL) of the course video is the RTMP live address of the corresponding course, and then the player of the smart fitness mirror downloads the corresponding course video live data from the OSS server according to the RTMP live address.

[0144] In some applications, the smart fitness mirror can be connected to an online streaming service that provides users with third-party video content streamed from a server (e.g., directly through a network router or indirectly through a user's smart device). Third-party content can be provided to users on a subscription basis. A third party can provide content to a centralized distribution platform that communicates with the smart fitness mirror over a network. One benefit of a centralized distribution platform is that distribution of content to the smart fitness mirror is simpler. Alternatively, a third party can develop a separate distribution platform that can use a separate software application on a smart device for users to access content.

[0145] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the multimodal interactive fitness mirror are implemented.

[0146] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, but is not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0147] A method for action recognition,

[0148] like Fig.10 As shown, based on the above embodiment, the device specifically includes the following steps in the interaction process:

[0149] S121 The fitness mirror collects user images in the area and obtains user action images;

[0150] S122 generates skeleton point information of the user according to the action image, and establishes a coordinate system according to the skeleton point information;

[0151] S122.1 Skeletal point information includes the top of the head skeleton points, pelvic skeleton points, left hand skeleton points and right hand skeleton points;

[0152] S122.2 obtaining a straight-line distance between a head bone point and a pelvic bone point, and obtaining a unit length of a coordinate system according to the straight-line distance;

[0153] S122.21 Obtain the straight-line distance between the top bone point and the pelvic bone point, and use one sixth of the straight-line distance as the unit length of the coordinate system, that is, the size of the unit length 1 of the coordinate system is one sixth of the straight-line distance;

[0154] S122.3 Take the straight line between the top bone point and the pelvic bone point as the ordinate, the pelvic bone point as the origin, and construct a rectangular coordinate system with the obtained unit length;

[0155] S123 establishes a plurality of interactive areas according to the coordinate system;

[0156] In this embodiment, there are five interaction areas, namely, a first interaction area, a second interaction area, a third interaction area, a fourth interaction area and a fifth interaction area. Fig.10 As shown, the first interactive area is Fig.10 The A area in the second interaction area is Fig.10 The third interactive area is Fig.10 The C region, the fourth interaction region is Fig.10 The D region, the fifth interaction region is Fig.10 The E area.

[0157] The first interactive area is centered on the top bone point, with a length of 5 units in the vertical direction and 6 units in the horizontal direction, that is, the length of the first interactive area is 12 units in length and the height is 10 units in length. The first interactive area is centered on the top bone point, with a length of 5 units in the vertical direction upward as the top edge and a length of 5 units in the vertical direction downward as the bottom edge; centered on the top bone point, with a length of 6 units in the horizontal direction to the left as the left edge and a length of 6 units in the horizontal direction to the right as the right edge. The preset control instruction for the first interactive area is to determine A, that is, when the interactive posture is in the first interactive area, the control instruction corresponding to the first interactive area is output, that is, the control instruction for determining A is output.

[0158] The control instruction of the second interactive area is left, and the control instruction of the third interactive area is right. The second interactive area takes the pelvic bone point as the starting point, 2 unit lengths vertically upward as the upper side, 10 unit lengths vertically downward as the lower side, takes the top of the head bone point as the starting point, 3 unit lengths horizontally to the left as the right side, and 16 unit lengths horizontally to the left as the left side, forming an area with a length of 13 units and a height of 12 units.

[0159] The third interactive area is an area with the pelvic bone point as the starting point, 2 units of length vertically upward as the upper side, 10 units of length vertically downward as the lower side, and with the top of the head bone point as the starting point, 3 units of length horizontally to the right as the left side, and 16 units of length horizontally to the right as the right side, forming an area with a length of 13 units and a height of 12 units.

[0160] The control instruction of the fourth interactive area is to determine D, and the control instruction of the fifth interactive area is to determine E. The fourth interactive area takes the top bone point as the starting point, 4 unit lengths vertically upward as the upper side, 5 unit lengths vertically downward as the lower side, takes the top bone point as the starting point, 7 unit lengths horizontally to the left as the right side, and 16 unit lengths horizontally to the left as the left side, forming an area with a length of 11 units and a height of 9 units.

[0161] The fifth interactive area is an area with the top of the head bone point as the starting point, 4 units of length vertically upward as the upper side, 5 units of length vertically downward as the lower side, and 11 units of length with the top of the head bone point as the starting point, 7 units of length horizontally to the right as the left side, and 16 units of length horizontally to the right as the right side, and a height of 9 units of length.

[0162] S124 identifies the interaction posture according to the skeleton point information;

[0163] S124.1 obtain the distance between the left hand skeleton point and the right hand skeleton point;

[0164] S124.2 compares the distance between the left hand skeleton point and the right hand skeleton point with the threshold value T. If the distance between the left hand skeleton point and the right hand skeleton point is less than or equal to the threshold value T, the interaction gesture is recognized. In this embodiment, the threshold value T is 15 cm. If the distance between the left hand skeleton point and the right hand skeleton point is less than or equal to the threshold value T, it is judged that the user makes an interaction gesture and the interaction gesture is recognized. For example, in this embodiment, the interaction gesture is clapping, that is, the user makes a clapping action; if the distance between the left hand skeleton point and the right hand skeleton point is greater than the threshold value T, the user does not make a clapping action; in this embodiment, the specific interaction gesture is only used as an example and is not limited.

[0165] S124.3 When the user makes an interactive gesture, the coordinates of the middle point between the left hand bone point and the right hand bone point are obtained;

[0166] S125 determines whether the coordinates of the middle point are located in the interactive area;

[0167] S125.1 Determine whether the coordinates of the middle point are located in the interactive area;

[0168] S125.11 If the coordinates of the middle point are located in one of the interactive areas, the interactive area is activated, and the control instruction corresponding to the interactive area is output; the other interactive areas are in an inactive state;

[0169] S125.12 When the control instruction corresponding to the interactive area is output, the interactive area is activated and the other interactive areas are inactive; the area of ​​the interactive area in the activated state increases, and the area of ​​the interactive area in the inactive state decreases.

[0170] In this embodiment, the length of the interactive area in the activated state increases by 1.5 unit lengths, and the height increases by 0.5 unit lengths. Specifically, in this embodiment, the length of the interactive area in the activated state increases to the left and right sides successively, that is, expands outward by 0.75 unit lengths, and the height increases to the upper and lower ends successively, that is, expands outward by 0.25 unit lengths. The length of the interactive area in the inactivated state decreases by 2 unit lengths, and the height decreases by 1 unit length. Specifically, the length of the interactive area in the inactivated state decreases to the left and right sides successively, that is, shrinks inward by 1 unit length, and the height decreases to the upper and lower ends, that is, shrinks inward by 0.5 unit length.

[0171] When the control instruction corresponding to the interactive area is not output, the area of ​​the interactive area does not change. After the control instruction is output, the area of ​​the interactive area is adjusted. After the control instruction is executed, the area of ​​the interactive area is restored to the original state.

[0172] When the user makes an interactive gesture in the first interactive area, the fitness mirror recognizes the interactive gesture and outputs a control instruction corresponding to the first interactive area. The area of ​​the first interactive area expands, that is, the upper side increases by 0.25 unit length vertically upward, the lower side increases by 0.25 unit length vertically downward, the left side increases by 0.75 unit length horizontally to the left, and the right side increases by 0.75 unit length horizontally to the right.

[0173] All parameters, dimensions, materials and configurations described herein are intended to be exemplary, and actual parameters, dimensions, materials and / or configurations will depend on one or more specific applications for which the present invention is used. It should be understood that the foregoing embodiments are presented primarily by way of example, and within the scope of the appended claims and their equivalents, embodiments of the present invention may be practiced in a manner other than as specifically described and claimed. Inventive embodiments of the present disclosure relate to each individual feature, system, article, material, kit and / or method described herein.

[0174] In addition, if such features, systems, articles, materials, kits and / or methods are not mutually contradictory, any combination of two or more such features, systems, articles, materials, kits and / or methods is included in the inventive scope of the present disclosure. Without departing from the spirit of the present disclosure, other substitutions, modifications, changes and omissions may be made in the design, operating conditions and arrangement of the corresponding elements of the exemplary embodiments. The use of numerical ranges does not exclude equivalents that fall outside the range and perform the same function in the same manner to produce the same results.

[0175] The above embodiments may be implemented in a variety of ways. For example, the embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be executed on a suitable processor or set of processors, whether provided in a single computer or distributed among multiple computers.

[0176] Furthermore, the computer may be embodied in any of a variety of forms, such as a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer. In addition, the computer may be embedded in a device not generally regarded as a computer but having suitable processing capabilities, including a personal digital assistant (PDA), a smart phone, or any other suitable portable or fixed electronic device.

[0177] In addition, the computer may have one or more input and output devices. Among other things, these devices may also be used to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visual presentation of the output and a speaker or other sound-generating device for audible presentation of the output. Examples of input devices that may be used for a user interface include a keyboard and a pointing device, such as a mouse, a touch pad, and a digitized tablet computer. As another example, a computer may receive input information through speech recognition or other audible formats.

[0178] Such computers may be interconnected in a suitable form through one or more networks, including local area networks or wide area networks, such as enterprise networks, intelligent networks (IN) or the Internet. Such networks may be based on suitable technologies, may operate according to suitable protocols, and may include wireless networks, wired networks or fiber optic networks.

[0179] The various methods or processes outlined herein may be encoded as software that can be executed on one or more processors using any of a variety of operating systems or platforms. In addition, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may also be compiled into executable machine language code or intermediate code that is executed on a framework or virtual machine. Some embodiments may specifically employ one or more of a specific operating system or platform and a specific programming language and / or scripting tool to facilitate execution.

[0180] In addition, various inventive concepts can be embodied as one or more methods, wherein at least one example of these methods has been provided. The actions performed as a part of the method can be sorted in different ways in some cases. Therefore, in some embodiments of the invention, the corresponding actions of a given method can be performed in an order different from the order specifically shown, which can include performing some actions simultaneously (even if such actions are shown as sequential actions in exemplary embodiments).

[0181] All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety.

[0182] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0183] Unless clearly indicated to the contrary, the indefinite articles "a" and "an" as used in this specification and claims should be understood to mean "at least one".

[0184] The phrase "and / or" used in this specification and claims should be understood to mean "one or two" elements so combined, i.e., elements that exist in combination in some cases and separately in other cases. Multiple elements listed with "and / or" should be interpreted in the same manner, i.e., "one or more" elements are so combined. In addition to the elements specifically represented by the "and / or" clause, other elements may optionally be present, whether related or unrelated to those specifically represented. Thus, as a non-limiting example, when used in conjunction with open language such as "comprising", a reference to "A and / or B" may refer to only A (optionally including elements other than B) in one embodiment; to only B (optionally including elements other than A) in another embodiment; to both A and B (optionally including other elements) in yet another embodiment; and so on.

[0185] As used in this specification and claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as inclusive, that is, including at least one, but also including many elements or more than one element in an element list and (optionally) additional unlisted items. Only when it is clearly indicated that they are contradictory, otherwise, such as "only one" or "exactly one" or "consisting of..." when used in the claims will refer to an element that just includes many elements or an element list. In general, when the term "or" as used herein is followed by exclusive terms such as "one of the two", "one in", "only one of" or "exactly one of", it should only be interpreted as indicating exclusive alternatives (that is, "one or the other but not two"). Its ordinary meaning when "consisting essentially of..." is used in the claims is as used in the field of patent law.

[0186] As used in this specification and claims, with respect to a list of one or more elements, the phrase "at least one" should be understood to mean at least one element selected from any one or more elements in the list of elements, but does not necessarily include at least one element of each element specifically listed in the list of elements, and does not exclude any combination of elements in the list of elements. This definition also allows for the optional presence of elements other than the elements specifically represented in the list of elements to which the phrase "at least one" refers, whether related or unrelated to those specifically represented. Thus, as a non-limiting example, "at least one of A and B" (or equivalently, "at least one of A or B", or equivalently "at least one of A and / or B") may refer to at least one A, optionally including more than one A, without the presence of B (and optionally including elements other than B) in one embodiment; to at least one B, optionally including more than one B, without the presence of A (and optionally including elements other than A) in another embodiment; to at least one A, optionally including more than one A and at least one B, optionally including more than one B (and optionally including other elements) in yet another embodiment; and so on.

[0187] In the claims and the description above, all transitional phrases such as "includes," "comprising," "carrying," "having," "containing," "involving," "holding," "consisting of," etc. should be understood as open-ended, i.e., including but not limited to.

[0188] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0189] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An interface switchable fitness system, characterized in that: The system comprises: A display, used to display a plurality of user interfaces, wherein the plurality of user interfaces can be switched; each of the user interfaces includes at least one page object, wherein the page object is used to display at least one or at least one group of page information; a mirror, the mirror being used to display the image of the user on the mirror, and the display showing the user interface to the user through the mirror; The user interface further includes a page switching page, and the page information displayed by the page object of the page switching page includes at least one preview image, and the preview image corresponds to the other switchable user interfaces one by one; The user interface further includes at least one exercise page, wherein the page information displayed by the page object of the exercise page includes at least one of an exercise course video, course playback information, accessory connection information, and user physiological information; The user interface includes an interactive object for entering a page switching page, and the user enters the page switching page by performing an interaction with the interactive object for entering a page switching page; The page switching page includes preview images of the first main page, the second main page, and the third main page and highlights one of the pages; The system implements user interface switching by identifying a switching operation action, wherein the step of identifying the switching operation action includes: S121 The fitness mirror collects user images in the area and obtains user action images; S122 generates skeleton point information of the user according to the action image, and establishes a coordinate system according to the skeleton point information; S123 establishes a plurality of interactive areas according to the coordinate system; S124 identifies the interaction gesture according to the skeleton point information, the user makes the interaction gesture, and obtains the coordinates of the middle point between the left hand skeleton point and the right hand skeleton point; S125 determines whether the coordinates of the middle point are located in the interactive area; S125.1 Determine whether the coordinates of the middle point are located in the interactive area; S125.11 If the coordinates of the middle point are located in one of the interactive areas, the interactive area is activated, and the control instruction corresponding to the interactive area is output; the other interactive areas are in an inactive state; S125.12 When the control instruction corresponding to the interactive area is output, the interactive area is activated and the other interactive areas are inactive; the area of ​​the interactive area in the activated state increases, and the area of ​​the interactive area in the inactive state decreases.

2. The interface switchable fitness system according to claim 1, characterized in that: The page information includes exercise course videos, visual data, operable interactive objects and interactive guides.

3. The interface switchable fitness system according to claim 1, characterized in that: The user interface includes at least one main page, and any of the at least one main page can be switched to other user interfaces, or a specified operation can be completed on the main page.

4. The interface switchable fitness system according to claim 1, characterized in that: The user interface also includes a pre-installed function page and an application page accessed by a third party.

5. The interface switchable fitness system according to claim 1, characterized in that: The user interface also includes page animations, which include page switching animations and page interaction animations. The page switching animations are used to produce dynamic effects when the page is operated or changed, and the page interaction animations are used to provide guidance to the user when interacting.

6. The interface switchable fitness system according to claim 1, characterized in that: The page switching page is provided with an interactive object for displaying return and confirming entry into the highlighted page, and the page switching page is provided with an interactive object for displaying switching to the highlighted page.

7. The interface switchable fitness system according to claim 1, characterized in that: The user interface includes page animations, and the page animations include page switching animations, and the page switching animations include page turning animations, returning to the previous level, and entering the next level animations.

Images