Interactive smart fitness mirror device

Abstract

The present invention discloses an interactive intelligent fitness mirror device, wherein the processor is further used to obtain the motion trajectory of the user's motion according to the user's motion data collected by the sensor, and control the floating icon to be displayed on the user interface according to the motion trajectory, and the floating icon is used to indicate the position indicated by the user's motion in the user interface. The present invention obtains the motion trajectory of the user's motion through the user's motion data, and controls the floating icon to be displayed on the user interface according to the motion trajectory, so as to ensure the accuracy of the user's motion during the exercise, and improve the use efficiency, fluency and comfort.

Description

Technical Field

[0001] The present invention relates to the fields of smart home and smart fitness, and in particular to an interactive smart fitness mirror device. Background Art

[0002] With the rapid development of economy, people's pace of life has accelerated. High-speed life and high-load work have caused various health problems for people living in cities. As a result, fitness has become a hot topic of concern. Smart fitness mirror is a new fitness product that integrates artificial intelligence, hardware content and services.

[0003] At present, most people who exercise at home watch videos through smart fitness mirrors to learn training movements. Smart fitness mirrors can be used as mirrors and can also show high-quality coach images. The existing smart fitness mirrors are cumbersome to interact with. They generally need to be controlled by external smart devices (such as smart phones) or touch screens. The operation is relatively complicated when controlled by external smart devices, and it is also necessary to configure the network for smart devices and smart fitness mirrors. Touch control is easy to leave fingerprints and sweat on the mirror surface, affecting the display effect of the screen. At the same time, the current fitness mirrors are large in size and occupy more family space, but the current smart fitness mirrors can only display fitness content and fitness-related data, and are used less frequently in daily life. Based on this, the inventors optimize the interaction between the smart fitness mirror and the user by adding a personalized interface and a new interaction method to the smart fitness mirror, and increase the frequency of use of the smart fitness mirror in daily life.

[0004] As research and development progresses, the inventors have discovered that when user actions control the user interface, such as entering or selecting or switching into a control command menu, especially when selecting a control command menu, the user cannot intuitively obtain the location reached by his or her actions, and may make actions but fail to trigger the corresponding control commands, thereby reducing the accuracy of the selection and affecting the user's comfort and fluency. Summary of the invention

[0005] One object of the present invention is to provide an interactive intelligent fitness mirror device, which displays a floating icon according to the motion trajectory of the user's action when the user makes an action, thereby improving the accuracy of selection and the comfort of the user.

[0006] This purpose is achieved by adopting the following technical solutions:

[0007] An example of a smart fitness mirror includes a communication interface for receiving a video image of a fitness trainer, a display operably coupled to the communication interface to display the video image of the fitness trainer, and a mirror disposed in front of the display to reflect an image of a person opposite to the display. The mirror is used to display the image of the user on the mirror, and the display displays the user interface to the user through the mirror.

[0008] An example of an interactive fitness method includes the following steps: (1) streaming fitness content to an interactive video system, the interactive video system including a mirror having a partially reflective portion and a display disposed on one side of the partially reflective portion; (2) displaying the fitness content to a user via the display and the partially reflective portion of the mirror; and (3) reflecting an image of the user using the mirror so that the image of the user is at least partially superimposed on the fitness content displayed via the display and the partially reflective portion of the mirror.

[0009] In order to add a personalized interface and a new interaction mode to the smart fitness mirror, optimize the interaction mode between the smart fitness mirror and the user, and increase the frequency of use of the smart fitness mirror in daily life, the present invention designs a set of operating system Launcher for the smart fitness mirror, which specifically includes: (1) GUI interface; (2) system control method for motion control; (3) system control method for voice control; (4) control method for multimodal fusion; (5) executable control instructions; (6) data acquisition management method; (7) some specific function management methods. A set of systems and operation methods are customized for the smart fitness mirror, so that the function of the smart fitness mirror is not only to watch fitness exercise videos, but also to be an important part of smart home to facilitate users to view more information and improve the use efficiency. At the same time, the customized operation method overcomes the disadvantage that the traditional smart fitness mirror needs to be controlled by touch and is easy to leave fingerprints on the partial reflective mirror surface. Since the core function of the smart fitness mirror is to allow users to compare their own mirrored movements when watching fitness exercise videos, fingerprints left on the partial reflective mirror surface have a greater impact on the function of the smart fitness mirror, resulting in most users being unwilling to use touch interaction, and using smart terminals to control the smart fitness mirror is not intuitive and troublesome. The method of motion control does not have these problems. Although gesture control is also used in some other electronic products, there have been breakthroughs in its application to smart fitness mirrors. Since the smart fitness mirror includes a partially reflective mirror, we can clearly see the actions we are doing when performing motion control, reducing the possibility of miscontrol and misoperation. At the same time, since motion recognition is the core function of the smart fitness mirror, the smart fitness mirror can achieve higher-precision complex motion recognition without additional cost, so that the control instructions that can be completed are much richer than those of existing gesture control. Finally, since the main function of the smart fitness mirror is exercise and fitness, applying motion control to the smart fitness mirror also needs to avoid miscontrol caused by fitness movements, which is also significantly different from traditional gesture control.

[0010] However, the present invention determines whether to generate a control instruction through user action data. Each person's action amplitude and senses are different. When a user makes an action to interact on the fitness mirror, it cannot be ensured that the action reaches the interactive area and selects the desired interactive interface. Based on this, the present invention optimizes the above interaction method, obtains the motion trajectory of the user action through the user action data, and displays a floating icon on the user interface according to the motion trajectory. The floating icon is used to indicate the position indicated by the user action in the user interface. Then, the user is intuitively informed whether the user action reaches the desired position, ensuring the accuracy of the user action and improving the fluency and comfort of use.

[0011] Preferably, when the processor determines that the action indicated by the user action data collected by the sensor matches the floating icon wake-up action corresponding to the preset floating icon wake-up interaction, the floating icon is enabled so that the floating icon moves with the user's action. The user can enable or disable the floating icon according to needs.

[0012] Preferably, when the processor determines that the user has completed the selection operation in the current user interface, the processor turns off the floating icon function so that the floating icon no longer moves with the user's action. The selection operation includes any one of gesture pause, gesture grab, pushing the palm forward, pushing the palm backward, waving the hand, and making a fist.

[0013] Preferably, when the processor determines that the floating icon moves to the position of the user interface object in the user interface according to the user action data, and determines that the user performs the selection operation at the position of the user interface object, the processor executes the operation corresponding to the user interface object.

[0014] All combinations of the foregoing concepts and additional concepts discussed in more detail below (assuming such concepts are not mutually contradictory) may be expected to be part of the inventive subject matter disclosed herein. Specifically, all combinations of the claimed subject matter appearing at the end of this disclosure may be expected to be part of the inventive subject matter disclosed herein. Terms that are expressly adopted herein and that may also appear in any disclosure incorporated by reference should be given the meaning most consistent with the specific concepts disclosed herein.

[0015] One or more technical solutions provided by the present invention have at least the following technical effects or advantages: the present invention obtains the motion trajectory of the user's motion through the user's motion data, and controls the floating icon to be displayed on the user interface according to the motion trajectory, and the floating icon is used to indicate the position indicated by the user's motion in the user interface, so as to ensure the accuracy of the user's motion during the motion process, and improve the efficiency, fluency and comfort of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0017] Figure 1 is a block diagram of an exemplary smart fitness mirror;

[0018] Figure 2 is an exemplary GUI page on a smart fitness mirror;

[0019] Figure 3 is an exemplary GUI page on a smart fitness mirror;

[0020] Figure 4 is an exemplary GUI page on a smart fitness mirror;

[0021] Figure 5-a is an exemplary GUI page on a smart fitness mirror;

[0022] Figure 5-b is an exemplary GUI page on a smart fitness mirror;

[0023] Figure 6 is an exemplary GUI page on a smart fitness mirror;

[0024] Figure 7 is an exemplary GUI page on a smart fitness mirror;

[0025] Figure 8 is an exemplary GUI page on a smart fitness mirror;

[0026] Fig. 9 is an exemplary GUI page on a smart fitness mirror;

[0027] Fig.10 Schematic diagram of motion recognition of smart fitness mirror. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those within the scope of this description. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0030] Those skilled in the art should understand that, in the disclosure of the present invention, the orientation or position relationship indicated by the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. are based on the orientation or position relationship shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.

[0031] Therefore, the present patent relates to an interactive smart fitness mirror device (also referred to as a "smart fitness mirror" and an "interactive training system") and a method of using an interactive training device. The smart fitness mirror includes a display configured to display exercise content (pre-recorded video or live stream) and an interface that enables a user to personalize the exercise. In addition, the smart fitness mirror can allow the user and / or the coach to interact with each other during the exercise in a manner similar to a conventional exercise performed in a gym or small fitness studio in which the user and the coach are in the same room (e.g., providing feedback to the coach on the rhythm of the exercise, correcting the user's form during a particular fitness program).

[0032] An exemplary smart fitness mirror.

[0033] An image recognition sensor (e.g., a camera) in a smart fitness mirror can be used to capture a video and / or still image of the user while the user is performing an activity (e.g., exercising). The user's video can then be shared with a coach to allow the coach to observe and provide guidance to the user during exercise. The video can also be shared with other users of other smart fitness mirrors for comparison or competition. The user's video can also be displayed in real time on the display 120 or stored for later playback. For example, by providing a visual comparison between the user and the coach, the user's video can be used for self-assessment during or after exercise. The stored video can also allow users to evaluate their progress or improvement over time when performing similar fitness.

[0034] The sensor can be configured to obtain dynamic and / or static images of the user when the user is performing activities (e.g., exercising), and determine whether the user's movements match standard movements based on the dynamic and / or static images and output the determination result; if the user's movements do not match standard movements, the sensor outputs movement correction information.

[0035] At the same time, the smart fitness mirror first displays the page through the display 120. When the user uses the smart fitness mirror, the page object selected by the user is determined according to the position of the user's right hand. During the interaction, the user's right hand makes an action instruction, and the user's skeletal point coordinates are generated according to the user's action data. The coordinates corresponding to the floating icon on the user interface are obtained according to the skeletal point coordinates, and the motion trajectory corresponding to the coordinates is called to control the floating icon to be displayed on the user interface according to the motion trajectory. The floating icon is used to indicate the position indicated by the user's action in the user interface. Therefore, when the user makes an action instruction, the floating icon is displayed on the user interface along the trajectory of the user's action, so that the user can intuitively know the position of the corresponding user interface after the action is made, thereby improving the accuracy of the selection.

[0036] In one or more embodiments, the floating icon may be in any shape, including but not limited to a sphere, a triangle, and a star. The floating icon may be in any color, including but not limited to black, gray, and white.

[0037] Specifically, turning the floating icon function on or off can be determined by a specific graphical user interface (GUI) or exercise page. For example, the action command control function is turned on by default on the main page of the graphical user interface (GUI) and turned off by default on other pages. Turning the floating icon function on or off can also be determined based on data detected by other sensors.

[0038] A specific example of a smart fitness mirror that can control the floating icon function based on the user action collected by the image recognition sensor is that the floating icon wake-up page is used to enable the floating icon when the user action data collected by the sensor matches the preset floating icon wake-up action. The preset floating icon wake-up action can be an action pre-set in the smart fitness mirror, or an action recorded by the user himself. The floating icon wake-up action includes but is not limited to raising both hands above the head, making a heart shape, and various static gestures, which are generally maintained for a specified time range of 0-5 seconds or longer.

[0039] A specific example of a smart fitness mirror is that it can control the floating icon function based on the voice collected by the voice recognition sensor, and turn the floating icon function on or off through a wake-up word.

[0040] A specific example of a smart fitness mirror that can control the floating icon function according to a touch sensor is that the smart fitness mirror can control the floating icon function according to a user touch command collected by the touch screen.

[0041] When waking up, the position where the floating icon appears includes but is not limited to appearing from the top of the page, the bottom of the page, the left side of the page, the right side of the page, and the middle of the page.

[0042] The appearance of the floating icon may include but is not limited to sliding out, popping up, and fading out.

[0043] A specific example of a smart fitness mirror that can control the floating icon function based on a touch sensor is that after the processor of the smart fitness mirror determines that the user has completed the selection operation in the current user interface, the processor turns off the floating icon function so that the floating icon no longer moves with the user's movements. When the floating icon is needed again, the floating icon is awakened by the floating icon awakening action corresponding to the preset floating icon awakening interaction. In one or more embodiments, the selection operation includes any one of a gesture stay, a gesture grab, pushing the palm forward, pushing the palm backward, waving the hand, and making a fist.

[0044] Bone point coordinates

[0045] In one or more embodiments, the coordinates of the user's skeletal points are generated and the coordinates corresponding to the floating icon are obtained on the user interface, including a variety of methods. In this embodiment, the top skeletal points, pelvic skeletal points, left hand skeletal points and right hand skeletal points can be obtained according to the user's motion data; the straight-line distance between the top skeletal points and the pelvic skeletal points is obtained, and the unit length of the coordinate system is obtained according to the straight-line distance; the straight-line distance between the top skeletal points and the pelvic skeletal points is obtained, and one-sixth of the straight-line distance is used as the unit length of the coordinate system, that is, the unit length 1 of the coordinate system is one-sixth of the straight-line distance; the straight line between the top skeletal points and the pelvic skeletal points is used as the ordinate, the pelvic skeletal points are used as the origin, and a rectangular coordinate system is constructed with the obtained unit lengths; a number of interactive areas are established according to the coordinate system, and the interactive area corresponding to the page is the interactive area established by the coordinate system.

[0046] The actions can be set according to personal preferences (left-handedness, disability, etc.), and have different modes for one hand and two hands, including left / right hand mode, and two-handed mode (such as left hand selection and right hand screen switching).

[0047] In addition, control actions correspond to control instructions. Matching a control action means that the processor executes the control instruction that matches the control action.

[0048] Page animation

[0049] The graphical user interface (GUI) also includes page effects, which include but are not limited to page switching effects and page interaction effects. Among them, page switching effects include page turning effects, returning to the previous level, entering the next level, etc. Page switching effects are mainly for page operations and changes, with the purpose of making page operations and changes have a better user experience when displayed. Among them, page interaction effects include selection confirmation effects, action guidance effects, etc. Page interaction effects are mainly for providing guidance to users when interacting, such as prompting users of the current interaction, the progress of the interaction, and prompting users to perform operations when interacting. Further specifics include but are not limited to highlighting or flashing the page information involved when the user interacts, displaying a progress bar on the page information involved, etc. It also includes identifying the actions the user is doing and displaying prompt actions or suggested actions when performing action control. It also includes completing prompts according to the user's voice instructions when performing voice control.

[0050] Further, a specific example of controlling the suspension action of the smart fitness mirror is as follows: the smart fitness mirror first displays the following through the display 120: Figure 5-aThe third main page includes page objects displayed in parallel in a four-grid arrangement. The user raises both hands above the head and makes a heart shape for 5 seconds, and a floating ball slides out from the top of the third main page. When the user uses the smart fitness mirror, the page object selected by the user is determined according to the position of the user's right hand, that is, when the user's right hand is above the right shoulder of the human body, the page object "Weather" is selected, and the floating icon appears in the "Weather" area with the user's right hand; when the user's right hand is below the right shoulder of the human body, the page object "Reminder" is selected, and the floating icon appears in the "Reminder" area with the user's right hand; when the user's right hand is above the left shoulder of the human body, the page object "Music" is selected, and the floating icon appears in the "Music" area with the user's right hand; when the user's right hand is below the right shoulder of the human body, the page object "Body Fat Scale" is selected, and the floating icon appears in the "Body Fat Scale" area with the user's right hand.

[0051] sensor

[0052] In one or more embodiments, the sensor may also be an image recognition sensor (camera), which acquires dynamic and / or static images of the user, processes the acquired dynamic and / or static images of the user into skeleton point images, and determines whether the user's movements match the standard movements or the movements displayed by the video images of the exercise interface based on the skeleton point images and outputs the judgment results. The dynamic and / or static images may also be processed in real time when the user uses the smart fitness mirror or after the exercise is completed, so as to control the smart fitness mirror or export the user's biometric data based on the user's movements and actions.

[0053] The sensor can also be an infrared recognition sensor in a smart fitness mirror. The infrared recognition sensor can be used to obtain a thermal image of the user when the user is performing an activity (for example, exercising), and determine the user's exercised parts, force exerted parts, and calories consumed based on the thermal image. The sensor can also determine whether the user's movements match the standard movements based on the user's exercised parts and force exerted parts, and output the judgment result. If the user's movements do not match the standard movements, the sensor outputs movement correction information.

[0054] The sensor can also be a voice recognition sensor (microphone) in the smart fitness mirror. The voice recognition sensor (microphone) can be used to obtain and process the user's voice information in real time when the user uses the smart fitness mirror, so as to control the smart fitness mirror or communicate with the user based on the user's voice information.

[0055] An exemplary smart fitness mirror.

[0056] The user can directly interface with the smart fitness mirror to control the smart fitness mirror (e.g., motion commands, voice commands, touch commands, etc.). A graphical user interface (GUI) displayed on the display 120 may be provided to facilitate user interaction with the smart fitness mirror.

[0057] Action command control (including "posture control" and "gesture control")

[0058] The smart fitness mirror can control the graphical user interface (GUI) based on user movements captured by the image recognition sensor.

[0059] Specifically, the actions controlled by the action command include but are not limited to static actions and dynamic actions. Static actions refer to the user making a specific posture with a specific body part and maintaining it for a specified time; dynamic actions refer to the user making a specific movement with a specific body part.

[0060] Specifically, static actions include but are not limited to raising the left / right hand, raising the left / right leg, raising both hands above the head and making a heart shape, and various static gestures, etc., and the range of maintaining a specified time is generally 0-5 seconds or longer. As long as the action is determined by the positional relationship between a specific part of the human body relative to space, the human body as a whole, or other specific parts, and is maintained for a specified time, it should fall within the protection scope of static actions. Further specific examples include: straightening the left hand at an angle of 15° / 30° to the horizontal plane for 1.5 seconds, holding the left hand at a right angle to the right hand for 1 second, and extending the left hand forward for 0.1 seconds.

[0061] Specifically, dynamic actions include but are not limited to horizontal sliding of the left / right hand, sliding the left / right hand up and down, clapping hands, jumping up, squatting, and various dynamic gestures, etc. Any action determined by the movement trajectory of a specific part of the human body relative to space, the human body as a whole, or other specific parts should fall within the protection scope of dynamic actions. Further specific examples include: horizontally sliding the left hand to move the left hand from the left side of the body to the right side of the body, horizontally sliding the left hand to move the left hand from the right side of the body to the left side of the body, horizontally sliding the right hand to move the right hand from the right side of the body to the left side of the body, sliding the left hand up and down to move the left hand from above the shoulder to below the waist, sliding the left hand up and down to move the left hand from above the upper 20% of the human body to below the upper 40%, clapping hands above the head, clapping hands above the left shoulder, etc.

[0062] Specifically, the control instructions executed by the action command control include but are not limited to the control instructions for non-exercise pages (wake-up page, main page, other pages) and the control instructions for the exercise page. The control instructions for non-exercise pages mainly involve the interaction instructions between the user and the non-exercise page. The control instructions for the exercise page mainly involve the user's control instructions for playing the exercise course.

[0063] Specifically, the control instructions for non-exercise pages include but are not limited to switching non-exercise pages, previous page, next page, confirm, return, course selection, course collection, adding to plan, entering the page switching interface, waking up the voice assistant, multimedia volume control, etc.

[0064] Specifically, the control instructions for the exercise page include but are not limited to course pause, course play, previous link, next link, course background music volume control, course coach volume control, course evaluation, etc.

[0065] Specifically, the user can also achieve different functions through a combination of multiple actions. Specifically, it includes but is not limited to using the first action to enter the first-level control instruction menu, and then using the second action to select an instruction in the first-level control instruction menu or enter the second-level control instruction menu, and the control instruction menu can be displayed to the user on the display. Further specific includes but is not limited to the first-level control instruction menu can be a course selection control instruction menu, including playing courses, collecting courses, displaying course details, booking courses and entering the second-level control instruction menu instructions, the second-level control instruction menu can be a course evaluation menu, including good reviews, medium reviews, and bad reviews.

[0066] Specifically, it also includes prompting the user's action command control in the graphical user interface (GUI). Specifically, it includes but is not limited to displaying icons of operable actions in the graphical user interface (GUI), marking or highlighting the control instructions pointed to by the user's static actions in the graphical user interface (GUI), displaying a progress bar for the duration of the user's static actions in the graphical user interface (GUI), and guiding the user's dynamic actions in the graphical user interface (when the completion degree of the user's dynamic actions exceeds a specified ratio).

[0067] Specifically, the smart fitness mirror can rely on other sensors to provide auxiliary judgment when controlling the graphical user interface (GUI) based on the user actions collected by the image recognition sensor. Specifically, it includes but is not limited to starting / interrupting / confirming action command control through voice recognition sensors, judging the front or back of the user through facial recognition sensors (action command control is only enabled when the user faces the smart fitness mirror), distinguishing different users through facial recognition sensors to load customized action command control instructions, and optimizing action / trajectory recognition accuracy through IMU, etc.

[0068] Specifically, the action controlled by the action command can be an action pre-set in the smart fitness mirror, or an action recorded by the user. The control instruction executed by the action command control can be an instruction pre-set in the smart fitness mirror, or an automated instruction set by the user. The action controlled by the action command and the executed control instruction can be a pre-set correspondence relationship, or a correspondence relationship set by the user, and can be a one-to-one relationship or a many-to-one relationship.

[0069] Specifically, the turning on or off of the motion command control function can be determined by a specific graphical user interface (GUI) or exercise page, such as the motion command control function is turned on by default on the main page of the graphical user interface (GUI) and turned off by default on other pages; the turning on or off of the motion command control function can also be determined based on data detected by other sensors; the turning on or off of the motion command control function can also be the turning on / off of all functions or the turning on / off of some functions.

[0070] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) according to user actions collected by an image recognition sensor. The smart fitness mirror first displays a first page of the graphical user interface (GUI) through the display 120. The first page displays a first prompt icon for raising the left hand to enter a page switching mode. The user raises his left hand, the first prompt icon enters a highlighted state and turns on a progress bar. After 1.5 seconds, the progress bar is completed, and the smart fitness mirror displays a page switching page for switching page display through the display 120. In addition to the highlighted first page, the switching page also includes the previous page of the first page "second page" The next page of the first page, "third page", is displayed on the switching page. The prompt icons of the previous page, next page, exit and confirmation are also displayed. The prompt icon of the previous page represents the action of waving the left hand horizontally from left to right, the prompt icon of the next page represents the action of waving the right hand horizontally from right to left, the prompt icon of exit represents raising the left hand, and the prompt icon of confirmation represents raising the right hand. The user then makes a horizontal wave of the left hand from left to right, and the switching page changes to highlight the second page. At this time, the user raises his right hand, and the icon representing confirmation enters the highlighted state and turns on the progress bar. After 1.5 seconds, the progress bar is completed, and the smart fitness mirror displays the second page through the display 120. The first page, the second page, the third page, the fourth page and the fifth page can be a wake-up page, a main page, a training page or other pages, respectively.

[0071] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) based on user actions captured by an image recognition sensor is that the smart fitness mirror first displays the fourth page of the graphical user interface (GUI) through the display 120. The fourth page displays a second prompt icon representing a horizontal wave of the left hand from left to right to enter the previous page and a third prompt icon representing a horizontal wave of the right hand from right to left to enter the next page. At this time, the user makes a horizontal wave of the right hand from right to left, and the smart fitness mirror displays through the display 120 that the fourth page switches to the fifth page.

[0072] In a specific example, a smart fitness mirror can control a graphical user interface (GUI) according to user actions collected by an image recognition sensor. The smart fitness mirror first displays the following information on the display 120: Figure 5-a The third main page includes multiple page objects displayed in parallel. At the same time, the third main page also displays the selected page object and the interactive object for entering the selected page object, and the interactive object for switching page objects is also displayed below the third main page. The user can switch the displayed page object through the interactive object of switching page objects, and enter the page corresponding to the selected page object by executing the interactive object of entering the selected page object. The user can select the selected page object by raising the right / left hand to the upper left, lower left, upper right, or lower right, and maintain the posture of the right / left hand after the selection is completed, and make the hand perform a specified static action such as opening the five fingers for 1 second to enter the page corresponding to the selected page object. At the same time, the user can make a horizontal waving action of the right hand from left to right to switch the displayed page object and enter the fourth main page.

[0073] Voice command control

[0074] The smart fitness mirror can control the graphical user interface (GUI) according to the user's voice collected by the voice recognition sensor.

[0075] Specifically, the user can use the voice recognition result of the voice command control to replace the action of the action command control and achieve the same control effect.

[0076] Specifically, the control instructions executed by the action command control include but are not limited to the control instructions for non-exercise pages (wake-up page, main page, other pages) and the control instructions for the exercise page. The control instructions for non-exercise pages mainly involve the user's interaction instructions with non-exercise pages. The control instructions for the exercise page mainly involve the user's control instructions for playing exercise courses.

[0077] Specifically, the control instructions for non-exercise pages include but are not limited to switching non-exercise pages, previous page, next page, confirm, return, course selection, course collection, adding to plan, entering the page switching interface, waking up the voice assistant, multimedia volume control, etc.

[0078] Specifically, the control instructions for the exercise page include but are not limited to course pause, course play, previous link, next link, course background music volume control, course coach volume control, course evaluation, etc.

[0079] Specifically, the user can also achieve different functions through a combination of multiple actions. Specifically, it includes but is not limited to using the first voice to enter the first-level control command menu, and then using the second voice to select commands in the first-level control command menu or enter the second-level control command menu, and the control command menu can be displayed to the user on the display. Further specific includes but is not limited to that the first-level control command menu can be a course selection control command menu, including playing courses, collecting courses, displaying course details, booking courses and entering the second-level control command menu. The second-level control command menu can be a course evaluation menu, including good reviews, medium reviews, and bad reviews.

[0080] Specifically, it also includes prompting the user's voice command control in the graphical user interface (GUI). Specifically, it includes but is not limited to displaying the keywords of the available voice in the graphical user interface (GUI), completing the user's voice command in the graphical user interface (when the completion degree of the user's dynamic action exceeds a specified ratio), etc.

[0081] Specifically, the smart fitness mirror can rely on other sensors to provide auxiliary judgment when controlling the graphical user interface (GUI) based on the user voice collected by the voice recognition sensor. Specifically, it includes but is not limited to starting / interrupting / confirming motion command control through motion recognition sensors, judging the front or back of the user through facial recognition sensors (motion command control is enabled only when the user faces the smart fitness mirror), distinguishing different users through facial recognition sensors to load customized voice command control instructions, etc.

[0082] Specifically, the keywords of the voice command control may be keywords pre-set in the smart fitness mirror, or keywords recorded by the user. The control instructions executed by the voice command control may be instructions pre-set in the smart fitness mirror, or automated instructions set by the user. The keywords of the voice command control and the executed control instructions may be pre-set corresponding relationships, or may be corresponding relationships set by the user, and may be one-to-one relationships, or may be many-to-one relationships.

[0083] Specifically, the voice command control function can be turned on or off by a specific graphical user interface (GUI) or exercise page, such as the voice command control function is turned on by default on the main page of the graphical user interface (GUI) and the motion command control function is turned off by default on other pages; the voice command control function can also be turned on by recognizing wake-up words; the voice command control function can also be turned on or off based on data detected by other sensors; the voice command control function can also be turned on / off for all functions or on / off for some functions.

[0084] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) based on user actions collected by an image recognition sensor, the smart fitness mirror first displays the first page of the graphical user interface (GUI) through the display 120, the user turns on the voice command control function through a wake-up word, and an icon representing the voice command control function is displayed on the fourth page. The user then says the command "enter the switching page" or "page switch", and the smart fitness mirror displays a page switching page for switching page display through the display 120. In addition to the highlighted first page, the switching page also includes the previous page "second page" of the first page and the next page "third page" of the first page. The switching page also displays an icon representing the voice command control function is displayed. The user then says the command "previous page" or "turn forward one page", and the switching page changes to highlight the second page. At this time, the user says the command "enter" or "confirm", and the smart fitness mirror displays the second page through the display 120.

[0085] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) based on user actions captured by an image recognition sensor is that the smart fitness mirror first displays the fourth page of the graphical user interface (GUI) through the display 120. The user turns on the voice command control function through a wake-up word. An icon representing the voice command control function is displayed on the fourth page. At this time, the user says the command "next page" or "turn to the previous page", and the smart fitness mirror displays the transition from the fourth page to the fifth page through the display 120.

[0086] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) based on user actions captured by an image recognition sensor. The smart fitness mirror first displays the sixth page of the graphical user interface (GUI) through the display 120. The user turns on the voice command control function through action command control. An icon representing the voice command control function is displayed on the fourth page. The user then says the command "start training" or "OK", and the smart fitness mirror displays the display 120 to switch from the sixth page to the seventh page.

[0087] The first page, the second page, the third page, the fourth page and the fifth page may be a wake-up page, a main page, a workout page or other pages respectively; the sixth page is a page for displaying workout cards; and the seventh page is a workout page.

[0088] Touch command control

[0089] The smart fitness mirror can control the graphical user interface (GUI) according to the user touch instructions collected by the touch screen.

[0090] Multimodal command control

[0091] Multimodal command control includes combined control, enhanced control and conflict control.

[0092] The combination control includes but is not limited to action combination control of multiple actions and voice combination control of multiple keywords. More complex control instructions (such as selecting and collecting, selecting and recommending to friends, taking screenshots and sharing, etc.) can be achieved through multiple consecutive actions or keywords (the interval time does not exceed the threshold).

[0093] The combined control includes but is not limited to enhancing and optimizing the user's motion command control or voice command control through facial recognition or scene recognition, and further includes but is not limited to recognizing that the user's facial expression is unhappy when the user's motion command control or voice command control is playing music, presenting the user with soothing recommendation results while showing care for the user; recognizing that the scene temperature is low when the user's motion command control or voice command control is recommending courses, reducing recommendations of meditation courses.

[0094] The conflict control includes but is not limited to when a user uses multiple different control methods to control the smart fitness mirror at the same time, multiple conflicting control instructions appear, and the control instructions that the user wants to execute are determined and executed by the smart fitness mirror by making priority judgments on different control methods.

[0095] Control instructions

[0096] The control instructions include, but are not limited to, system wake-up instructions, page switching instructions, page interaction instructions, and course control instructions, and users can control the smart fitness mirror through these control instructions.

[0097] The system wake-up instructions include instructions for waking up the smart fitness mirror from a standby state to a wake-up page, a main page, an exercise page, or other pages, and instructions for switching the smart fitness mirror from a wake-up page to a main page, an exercise page, or other pages.

[0098] The page switching instruction refers to an instruction for controlling the smart fitness mirror to switch between multiple different pages, and the different pages may be pages of different types or pages of different contents. Further page switching instructions include but are not limited to page turning instructions (next page / previous page) and instructions for controlling the smart fitness mirror to enter a page switching page.

[0099] Among them, page interaction instructions refer to instructions used to control the smart fitness mirror to perform interaction or input, and the page interaction instructions include but are not limited to control interaction instructions and input interaction instructions. Further, the control interaction instructions refer to instructions for controlling the graphical user interface (GUI) or the page objects or interactive objects in the graphical user interface (GUI) to realize their interactive functions, such as confirmation / return, adding to favorites, adding to a plan, waking up a voice assistant, volume control, switching accounts, switching modes, opening a contextual menu, etc. Further, the input interaction instructions refer to instructions for inputting information in the graphical user interface (GUI) or the page objects or interactive objects in the graphical user interface (GUI), such as adding a heart for each instruction entered when evaluating a course; or switching a preset expression / reply for each instruction entered when chatting with other users through a smart fitness mirror, etc.

[0100] The course control instructions are used to control the smart fitness mirror to play exercise videos on the exercise page, and are mainly used to control the playback of exercise videos. Further course control instructions include but are not limited to pause / play, previous / next section, background volume control, coach volume control, course evaluation, etc.

[0101] Data Management

[0102] The data sources of the smart fitness mirror include but are not limited to sensor data and background data, and the sensor data and background data are used for inputting user control instructions and displaying page information.

[0103] The sensor data includes but is not limited to the on-mirror sensor data and the external sensor data. The on-mirror sensor data includes but is not limited to the data detected by sensors installed on the smart fitness mirror, such as image recognition sensors, infrared recognition sensors and voice recognition sensors. More specifically, the data detected by the image recognition sensor includes but is not limited to the image detected by the image recognition sensor and the data obtained after the image processing and recognition, such as user action data, user posture data, facial recognition data, user expression data and environmental image data. More specifically, the data detected by the infrared recognition sensor includes but is not limited to the thermal imaging image detected by the infrared recognition sensor and the data obtained after the thermal imaging image processing and recognition, such as user thermal imaging data, user exercise part data, environmental thermal imaging data, etc. More specifically, the data detected by the voice recognition sensor includes but is not limited to the voice data detected by the sound recognition sensor and the data obtained after the voice data processing and recognition, such as voice control instructions, keyword instructions, wake-up word instructions, etc. The on-mirror sensor data includes but is not limited to inertial sensor data, physiological data sensor data (such as heart rate belt data, body fat scale data, etc.), and other data detected by sensors connected to the smart fitness mirror wirelessly. More specifically, the data detected by the inertial sensor includes but is not limited to the speed / acceleration data detected by the inertial sensor and the data obtained after processing and identifying the speed / acceleration data, such as user motion data, etc. More specifically, the data detected by the physiological data sensor includes but is not limited to the physiological data detected by the physiological data sensor and the data obtained after processing and identifying the physiological data, such as heart rate data, blood pressure data, blood oxygen saturation data, etc.

[0104] The background data includes but is not limited to background user data and other background data. The background user data is data directly associated with the user stored in the smart fitness mirror and / or the server connected to the smart fitness mirror. The background user data includes but is not limited to user physiological data, user social data, user calendar data and usage history data. The usage history data includes but is not limited to the user's completed course data, collection of course data, historical best performance data, physiological data and performance data during the course, class frequency data, etc. The user physiological data includes but is not limited to the user's height, weight, gender, age, heart rate, blood pressure, injury history, etc. The user social data includes but is not limited to data related to the people / friends followed by the user and data related to the groups / interest groups joined by the user. This information can specifically be class invitation data for inviting users to complete designated courses, invitation data for inviting users to one-on-one or multi-person challenges, rankings between users, and dynamic data uploaded by users themselves or other users. The dynamics can be synchronized to other social platforms (such as WeChat, Weibo, Facebook, Twitter, etc.). The user calendar data includes but is not limited to the user's schedule information data, to-do data, course plan data, etc. The other background data is data with low correlation with users stored in the smart fitness mirror and / or the server connected to the smart fitness mirror, and the other background data includes but is not limited to IP data, environmental data, course data and third-party data. Among them, IP data includes but is not limited to the IP address data and Mac address data of the smart fitness mirror. Among them, environmental data includes but is not limited to the time data, positioning data, weather data, temperature data, clothing index data, etc. of the area where the IP of the smart fitness mirror is located. The course data includes but is not limited to exercise video data, coach name, course type, course duration, course difficulty, course label and / or expected consumption, etc. The course type includes but is not limited to showing HIIT, aerobic dance, yoga, strength shaping, combat training, Barre, Pilates, stretching, dance, meditation, challenge, etc. to users through the smart fitness mirror. The course label includes but is not limited to showing users through the smart fitness mirror whether it is an AI recognition course, whether it is a parent-child course, the main body parts involved in the course (such as the whole body, arms, waist and abdomen, legs, etc.) and the props needed in the course (such as no equipment, yoga mat, elastic circle, elastic band, dumbbell, etc.). The third-party data includes but is not limited to data of third-party functions installed on the smart fitness mirror.

[0105] Data storage

[0106] The smart fitness mirror can also store user information locally on the smart fitness mirror and / or a remote storage device (e.g., a cloud service) depending on the amount of storage space used. For example, user information that uses very little storage space can be stored locally on the smart fitness mirror, including but not limited to the user's name, age, height, weight, and gender. In addition, course data can also be stored in the smart fitness mirror to reduce the impact of network latency that may affect the quality of the video stream. The amount of video content stored may be limited by the storage capacity of the smart fitness mirror. In some configurations, video content may be temporarily stored only daily or weekly, or depending on the percentage of the capacity of the smart fitness mirror used. Background user data that uses a large amount of storage space can be stored on a remote storage device, including but not limited to physiological data, such as the user's heart rate and calories consumed, and videos or skeletal point data of the user taken during exercise. The smart fitness mirror can retrieve this information for subsequent analysis and display.

[0107] Data transmission between the smart fitness mirror and the remote storage device can be protected (e.g., encrypted) in various ways to prevent unwanted loss or theft of user information. For example, the low-power Bluetooth protocol includes built-in security features that can be used by devices that utilize the protocol. However, these security features can only be used if the Bluetooth binding step is completed before the connection is established with encryption. In some cases, various security mechanisms may not be implemented or various security mechanisms may fail, and application-level security can be implemented in conjunction with the block specification of the above data. For example, Advanced Encryption Standard (AES) encryption of the message can be applied before the preamble of the preamble message. In some aspects, the low-power Bluetooth protocol performs a similar process via built-in security features at the firmware level, and similar protection can be provided to prevent personnel from reading communications between the client and the server.

[0108] When a client disconnects from a server, the GATT service added for the client to read / notify messages can be removed from the service record on the server device. This ensures that no connections are left open and the system does not accidentally leak information to nefarious snoopers. This connection termination can be triggered by either the server or the client and relies on the Bluetooth low energy stack to provide notification to both parties that the connection has been closed. If Bluetooth bonding is used to provide firmware-level cryptographic security during the initial connection setup, the bonding information can be stored on each device so that bonding does not need to be repeated after subsequent connections between the client and server.

[0109] Feature Management

[0110] The smart fitness mirror can also realize the following functions based on the received data: user identification, mode switching, attention judgment, physical / emotional assessment, and clothing recommendations.

[0111] The user identification function allows the smart fitness mirror to identify the user who is using the smart fitness mirror through the input data. This function is mainly implemented by the following method: the user inputs the user's characteristic data through the data input device, and the smart fitness mirror compares the characteristic data input by the user with the characteristic data in the database, determines the user who is using the smart fitness mirror according to the comparison result, and allows the smart fitness mirror to adjust the graphical user interface (GUI) according to the identification result, such as adjusting the font size, page information preference, etc. The data input device includes but is not limited to the sensor data on the mirror and the external sensor, and the user's characteristic data includes but is not limited to the user's bone points, voiceprint, weight, resting heart rate and other mirror sensor data and the data collected or processed by the external sensor after collection, and the mapping relationship between the user and the characteristic data is recorded in the database. Further, the user identification function can be used for the identification and switching of multiple sub-accounts under a main account, and can also be used for the identification and switching between multiple main accounts. In another example, the user identification function allows the smart fitness mirror to identify the characteristics of the user who is using the smart fitness mirror through the input data, such as male users, female users, elderly people, children, and allows the smart fitness mirror to adjust the graphical user interface (GUI) according to the identification result, such as adjusting the font size, page information, etc.

[0112] Among them, the mode switching function allows the smart fitness mirror to switch between multiple working modes to adapt to different working scenarios. For example, the smart fitness mirror is allowed to switch to the youth mode. In the youth mode, the youth account is bound to the parent account or the main account of the device. In the youth mode, the page information displayed by the smart fitness mirror will be adaptively adjusted, such as the course recommendation will be adjusted to give priority to the presentation of content suitable for teenagers, such as skipping, somatosensory games, etc. At the same time, the real-time situation and motion data in this mode can be synchronized to the smart terminal of the parent account or the main account of the device. In another example, the smart fitness mirror is allowed to switch to the guest mode. In the guest mode, the smart fitness mirror can work without being bound to the account. The page information displayed by the smart fitness mirror can also be adaptively adjusted, such as the course recommendation will be adjusted to give priority to the presentation of content suitable for novices, such as content with a low difficulty label, etc. At the same time, even if the smart fitness mirror is bound to the account, the motion data in this mode will not be counted in the bound account, and will not affect the training plan or completion of the bound account.

[0113] Among them, attention judgment allows the smart fitness mirror to determine whether the user's attention is on the smart fitness mirror when the device is working, so that the smart fitness mirror can judge the user's intention to interact with the mirror and activate or shut down sensors, other digital input devices and functions. Attention judgment is mainly based on the user's performance to determine whether the user wants to use the smart fitness mirror. It can be judged by but not limited to the following features: distance judgment (the distance between the user and the smart fitness mirror does not exceed the threshold), posture judgment (whether the user is facing the smart fitness mirror), action judgment (whether the user has made a specified action or a common exercise action), semantic judgment (whether the recognition result of the user's voice is related to a specific keyword), etc.

[0114] The physical / emotional assessment allows the smart fitness mirror to determine the physical / emotional condition of the user when using the device, so that the smart fitness mirror can adjust the page information or other feedback according to the user's physical / emotional condition. The user's physical / emotional condition can be identified by physiological data sensors and image recognition sensors (facial recognition). When the user's mood is identified as unhappy, the user is presented with soothing page information recommendation results and the user is cared for during voice interaction; in a further example, when the user wakes up the voice function of the smart fitness mirror through the wake-up word, the smart fitness mirror recognizes that the user's expression is relatively depressed through the image recognition sensor (facial recognition), and the smart fitness mirror initiates the first round of dialogue "Hey, what's wrong? You don't look very happy, do you want to listen to a song to relax?" The identification of the physical / emotional condition can be determined by identifying the user's age, body posture, expression and other information to determine the user's physical / emotional state such as emotion, stress, fatigue, etc.; it can also be identified through blood pressure and heart rate data detected by the physiological data sensor.

[0115] Among them, clothing recommendation allows the smart fitness mirror to initiate active interaction with the user based on specific conditions. The smart fitness mirror determines whether the user needs the smart fitness mirror to initiate active interaction based on the collected time data and positioning data, combined with user information and the user's actions in front of the smart fitness mirror. For example, when the user first appears in front of the smart fitness mirror in the morning and turns around, the smart fitness mirror turns on the clothing recommendation function and recommends to the user clothes that are suitable for today's wear at the current location. Furthermore, when the user appears in front of the smart fitness mirror on the weekend and turns around, the smart fitness mirror turns on the clothing recommendation function and outputs the user's current clothing matching suggestions. The above functions can also be used for other functions of the smart fitness mirror that actively interact with users and external information.

[0116] Figure 1An exemplary representation of a smart fitness mirror is shown. The smart fitness mirror may include a processor 110 for partially controlling the operation of various subcomponents in the smart fitness mirror and managing data flows (e.g., video content, audio from a coach or user, biometric feedback analysis) flowing into / out of the smart fitness mirror. The smart fitness mirror may include a display 120 for displaying video content, a graphical user interface (GUI) with which a user can interact and control the smart fitness mirror, biometric feedback data, and / or other visual content. A sensor 130 may be coupled to the processor 110 to collect user-related data. An antenna 140 may be coupled to the processor 110 to provide data transmission between the smart fitness mirror and another device (e.g., a remote control device, a biometric sensor, a wireless router). The antenna 140 may include a plurality of transmitters and receivers, each of which is customized for a specific frequency and / or wireless standard (e.g., Bluetooth, 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 2G, 3G, 4G, 4G LTE, 5G). The amplifier 150 may be coupled to the processor 110 to receive audio signals from the processor 110 to output subsequent sounds through the left speaker 152 and / or the right speaker 154 .

[0117] Smart fitness mirrors can also include Figure 1 Additional components not shown in the figure. For example, the smart fitness mirror may include a switch mode power supply (SMPS), a switch, and onboard memory and storage devices (non-volatile and / or volatile memory), including but not limited to a hard disk drive (HDD), a solid state drive (SDD), a flash memory, a random access memory (RAM), and a secure digital (SD) card. The onboard memory and / or storage device can be used to store firmware and / or software for the operation of the smart fitness mirror. As described above, the onboard memory and / or storage device can also be used to (temporarily and / or permanently) store other data, including but not limited to video content, audio, user video, biometric feedback data, and user settings. In another example, the smart fitness mirror may include various components for mounting and supporting the smart fitness mirror.

[0118] Antenna 140 may include multiple antennas, each antenna being used as a receiver and / or transmitter to communicate with various external devices, such as a user's smart terminal (e.g., computer, smartphone, tablet), an external sensor (e.g., heart rate belt, inertial sensor, body fat scale) and / or a remote server or cloud server for streaming or playing video content. Once again, antenna 140 may comply with various wireless standards, including but not limited to Bluetooth, 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 2G, 3G, 4G, 4G LTE and 5G standards.

[0119] The sensor 130 may include a plurality of sensors, and the plurality of sensors may be one or more of an image recognition sensor (camera), an infrared recognition sensor, or a voice recognition sensor (microphone).

[0120] The image recognition sensor (camera) in the smart fitness mirror can be used to obtain a video and / or still image of the user while the user is performing an activity (e.g., exercising). The user's video can then be shared with a coach to allow the coach to observe and provide guidance to the user during exercise. The video can also be shared with other users of other smart fitness mirrors for comparison or competition. The user's video can also be displayed in real time on the display 120 or stored for later playback. For example, by providing a visual comparison between the user and the coach, the user's video can be used for self-assessment during or after exercise. The stored video can also allow users to evaluate their progress or improvement over time when performing similar fitness.

[0121] Graphical User Interface (GUI)

[0122] The smart fitness mirror may display a graphical user interface (GUI) through the display 120 to facilitate user interaction with the smart fitness mirror.

[0123] The graphical user interface (GUI) includes but is not limited to a wake-up page, a main page, a workout page, and other pages.

[0124] The wake-up page refers to the initial page when the smart fitness mirror is awakened, which is used to display some basic page information. Only when it detects that the user performs a specified operation, the smart fitness mirror is completely awakened and enters the main page, exercise page or other pages.

[0125] The main page displays at least one page information, and the page information is used to display data, operable interactive objects or interactive guidance to the user. The main page is used to display preset page information or page information set by the user to the user, and the user can control the smart fitness mirror by performing specified operations on the main page, and the control includes but is not limited to page switching, function switching or entering third-party access applications.

[0126] The exercise page displays a video playback window for playing exercise videos to guide users to exercise, and can also display at least one of course playback information (such as total course duration, played duration, etc.), accessory connection information (such as heart rate belt connection information, etc.) and user physiological information (such as heart rate, blood pressure, calories consumed, etc.). The exercise page can be entered from the main page or the wake-up page and then returned to the main page or wake-up page, or can be directly entered when the smart fitness mirror is turned on.

[0127] Among them, other pages display pre-installed function pages (such as multimedia player page, weather display page, food recommendation page, etc.) or third-party access application pages (such as WeChat, Tik Tok, Himalaya, etc.) entered through the main page. Other pages are generally entered from the main page or the wake-up page and return to the main page or the wake-up page.

[0128] Page Information

[0129] A graphical user interface (GUI) can display page information through page objects. Each page object is used to display at least one or a group of page information. The displayed page information may be information stored in a local storage device of the smart fitness mirror, or information received by the smart fitness mirror from other devices through a communication interface, including but not limited to course preview information, user information, environmental information, multimedia information, external device information, etc. that can be switched for display.

[0130] The course preview information includes but is not limited to the name of the coach, course type, course duration, course difficulty, course label and / or estimated consumption, etc. The course type includes but is not limited to displaying HIIT, aerobics, yoga, strength shaping, combat training, Barre, Pilates, stretching, dance, meditation, challenges, etc. to users through a smart fitness mirror. The course label includes but is not limited to displaying to users through a smart fitness mirror whether it is an AI-recognized course, whether it is a parent-child course, the main body parts involved in the course (such as the whole body, arms, waist and abdomen, legs, etc.) and the props needed in the course (such as no equipment, yoga mats, elastic circles, elastic bands, dumbbells, etc.). The page object that displays the course preview information is also called a course card.

[0131] The user information includes but is not limited to usage history information, physiological information, social information, calendar information, etc. The usage history information includes but is not limited to displaying completed courses, favorite courses, historical best scores, frequency of classes, etc. to users through smart fitness mirrors. The physiological information includes but is not limited to displaying height, weight, gender, age, heart rate, blood pressure, injury history, etc. to users through smart fitness mirrors. The social information includes but is not limited to displaying information related to people / friends that users follow to users through smart fitness mirrors, and can also be information related to groups / interest groups that users join. This information can specifically be an invitation to invite users to complete a specified course, an invitation to invite users to a one-to-one or multi-person challenge, rankings between users, and dynamics uploaded by users themselves or other users. The dynamics can be synchronized to other social platforms (such as WeChat, Weibo, Facebook, Twitter, etc.). The calendar information includes but is not limited to displaying the user's schedule information, to-do items, course plans, etc. to users through smart fitness mirrors. The page object that displays user information is also called a user card.

[0132] The environmental information includes but is not limited to weather information. The weather information includes but is not limited to showing the weather forecast, temperature, humidity, clothing recommendations, etc. to the user through the smart fitness mirror. The page object showing weather information is also called a weather card.

[0133] The multimedia information includes but is not limited to music and video information played by the smart fitness mirror. The page object displaying the multimedia information is also called a multimedia card.

[0134] The external device information includes, but is not limited to, information of other fitness devices connected to the smart fitness mirror and other smart furniture (such as sweeping robots, smart speakers, smart gateways, etc.) displayed through the smart fitness mirror. The page object displaying the external device information is also called a device card.

[0135] The GUI can also display page information through interactive objects, which can be icons, text, highlighted selections, other forms that can prompt users to interact, or a combination thereof. The interactive objects are used to display to users the controls or operations that users can perform (such as page switching, selection, pause / play of courses, etc.). Furthermore, the interactive objects can also be combined with the pages or page objects of the GUI to display different interactive functions.

[0136] In addition to being fixed on each page, the page objects and interactive objects can also be displayed in the form of pop-up windows or scrolling playback on each page.

[0137] Some sample pages

[0138] The same plan Figure 2 , 3 , 4, 5-a, 5-b, where Figure 2 It is the first main page. Figure 3 It is a page switching page. Figure 4 , 5-a, 5-b are the second main page, the third main page and the fourth main page respectively. The first main page displays multiple page information through multiple page objects, including weather cards, multimedia cards and recommended course cards, etc. At the same time, the first main page also includes an interactive object for entering the page switching page and an interactive object combined with the course card displayed in the upper right corner. The user can enter the second main page by performing an interaction with the interactive object combined with the course card, or enter the page switching page by performing an interaction with the interactive object for entering the page switching page. The page switching page includes preview images of the first main page, the second main page and the third main page and highlights one of the pages. At the same time, the interactive objects for returning and confirming to enter the highlighted page are also displayed above the page switching page, and the interactive objects for switching the highlighted page are also displayed below the page switching page. The user can switch the highlighted page by performing an interaction with the interactive object for switching the highlighted page, and enter the selected page by confirming the interaction with the interactive object for entering the highlighted page. When the user selects and enters the second main page, the second main page includes multiple course cards and highlights one of the course cards. At the same time, the second main page also displays the interactive objects for returning and confirming to enter the course of the highlighted course card at the top and the interactive objects for switching to the highlighted course card at the bottom. The user can switch the highlighted course card by executing the interaction of switching the interactive objects of the highlighted course card, and enter the exercise page of the selected course card by executing the interaction of confirming to enter the interactive objects of the highlighted course card. When the user selects and enters the third main page, the third main page includes multiple page objects displayed in parallel. At the same time, the third main page also displays the selected page object and the interactive objects for entering the selected page object, and the interactive objects for switching page objects are also displayed at the bottom of the third main page. The user can switch the displayed page object by the interaction of the interactive objects of the switching page object, and enter the page corresponding to the selected page object by executing the interaction of entering the interactive objects of the selected page object.

[0139] Option 2 Figure 6 , 7 , 8, and 9, where Figure 6 It is the fifth main page. Figure 7 It is the sixth main page. Figure 8 , 9They are the seventh and eighth main pages respectively. The fifth main page displays multiple page information through multiple page objects, including weather cards, multimedia cards, and recommended course cards, etc. At the same time, the fifth main page also includes interactive objects for switching main pages and interactive objects combined with course cards displayed below. Users can enter the eighth main page by performing interactions with interactive objects combined with course cards, or switch the displayed main page by performing interactions with interactive objects for switching main pages. When the user chooses to switch the displayed main page to the sixth main page, the sixth main page includes multiple page objects displayed in parallel. At the same time, the sixth main page also displays the selected page object and the interactive object for entering the selected page object, and the interactive object for switching the main page is also displayed below the sixth main page. Users can switch the displayed main page by performing interactions with the interactive objects for switching main pages, and enter the page corresponding to the selected page object by performing interactions with the interactive objects for entering the selected page object. When the user chooses to switch the displayed main page to the seventh main page, the seventh main page includes a plurality of page objects displayed in parallel, and the seventh main page also displays the selected page object and the interactive object for entering the selected page object, and the interactive object for switching the main page is also displayed below the seventh main page. The user can switch the displayed main page by executing the interaction of the interactive object for switching the main page, and enter the page corresponding to the selected page object by executing the interaction of the interactive object for entering the selected page object. When the user chooses to enter the page corresponding to the selected page object to be the eighth main page, the eighth main page includes a plurality of course cards displayed in parallel, and the eighth main page also displays the selected course card and the interactive object for entering the selected course card, and the interactive object for switching the course card is also displayed below the eighth main page. The user can switch the displayed course card by executing the interaction of the interactive object for switching the course card, and enter the exercise page corresponding to the selected course card by executing the interaction of the interactive object for entering the selected course card.

[0140] The page switching page includes preview images of the first main page, the second main page, and the third main page, and highlights one of the pages. At the same time, the interactive objects for returning and confirming to enter the highlighted page are displayed at the top of the page switching page, and the interactive objects for switching the highlighted page are also displayed below the page switching page. The user can switch the highlighted page by executing the interaction of the interactive object for switching the highlighted page, and enter the selected page by confirming the interaction of the interactive object for entering the highlighted page. When the user selects and enters the second main page, the second main page includes multiple course cards and highlights one of the course cards. At the same time, the interactive objects for returning and confirming to enter the course of the highlighted course card are also displayed at the top of the second main page, and the interactive objects for switching the highlighted course card are also displayed below the page switching page. The user can switch the highlighted course card by executing the interaction of the interactive object for switching the highlighted course card, and enter the exercise page of the selected course card by executing the interaction of the interactive object for confirming to enter the course of the highlighted course card. When the user selects and enters the third main page, the third main page includes multiple page objects displayed in parallel. The third main page also displays the selected page object and the interactive object for entering the selected page object, and the interactive object for switching page objects is also displayed below the third main page. The user can switch the displayed page object by interacting with the interactive object for switching page objects, and enter the page corresponding to the selected page object by executing the interaction of the interactive object for entering the selected page object.

[0141] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the multimodal interactive fitness mirror are implemented.

[0142] The processor may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0143] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the multimodal interactive fitness mirror device in the invention by running or executing the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0144] Video streaming method

[0145] The smart fitness mirror is configured to display video content from a video production factory on the display surface 120. The video content can be streamed as live content or pre-recorded recorded content. The live content can also be recorded and stored in a cloud server so that users can request and play the video content later, thus becoming recorded content. When streaming the video, the smart fitness mirror first sends a request for a specified course to the API server. The data returned by the API server is divided into two parts. The first part is an overview of the specified course (including course pictures and course introduction); the second part is the address (URL) of the course video, and then the smart fitness mirror requests the corresponding course video on the OSS server according to the received URL. When the course is recorded content, the smart fitness mirror uses the HLS protocol for video playback, that is, the address (URL) of the course video is the M3U8 file of the corresponding course, and then the player of the smart fitness mirror downloads the corresponding course video slice from the OSS server according to the record of the M3U8 file. When the course is live content, the smart fitness mirror uses the RTMP protocol to play the video, that is, the address (URL) of the course video is the RTMP live address of the corresponding course, and then the player of the smart fitness mirror downloads the corresponding course video live data from the OSS server according to the RTMP live address.

[0146] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the multimodal interactive fitness mirror are implemented.

[0147] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, but is not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0148] A method for action recognition,

[0149] like Fig.10 As shown, based on the above embodiment, the device specifically includes the following steps in the interaction process:

[0150] S121 The fitness mirror collects user images in the area and obtains user action images;

[0151] S122 generates skeleton point information of the user according to the action image, and establishes a coordinate system according to the skeleton point information;

[0152] S122.1 Skeletal point information includes the top of the head skeleton points, pelvic skeleton points, left hand skeleton points and right hand skeleton points;

[0153] S122.2 obtaining a straight-line distance between a head bone point and a pelvic bone point, and obtaining a unit length of a coordinate system according to the straight-line distance;

[0154] S122.21 Obtain the straight-line distance between the top bone point and the pelvic bone point, and use one sixth of the straight-line distance as the unit length of the coordinate system, that is, the size of the unit length 1 of the coordinate system is one sixth of the straight-line distance;

[0155] S122.3 Take the straight line between the top bone point and the pelvic bone point as the ordinate, the pelvic bone point as the origin, and construct a rectangular coordinate system with the obtained unit length;

[0156] S123 establishes a plurality of interactive areas according to the coordinate system;

[0157] In this embodiment, there are five interaction areas, namely, a first interaction area, a second interaction area, a third interaction area, a fourth interaction area and a fifth interaction area. Figure 4 As shown, the first interactive area is Figure 3 The A area in the second interaction area is Figure 4 The third interactive area is Figure 4 The C region, the fourth interaction region is Figure 4 The D region, the fifth interaction region is Figure 4 The E area.

[0158] The first interactive area is centered on the top bone point, with a length of 5 units in the vertical direction and 6 units in the horizontal direction, that is, the length of the first interactive area is 12 units in length and the height is 10 units in length. The first interactive area is centered on the top bone point, with a length of 5 units in the vertical direction upward as the top edge and a length of 5 units in the vertical direction downward as the bottom edge; centered on the top bone point, with a length of 6 units in the horizontal direction to the left as the left edge and a length of 6 units in the horizontal direction to the right as the right edge. The preset control instruction for the first interactive area is to determine A, that is, when the interactive posture is in the first interactive area, the control instruction corresponding to the first interactive area is output, that is, the control instruction for determining A is output.

[0159] The control instruction of the second interactive area is left, and the control instruction of the third interactive area is right. The second interactive area takes the pelvic bone point as the starting point, 2 unit lengths vertically upward as the upper side, 10 unit lengths vertically downward as the lower side, takes the top of the head bone point as the starting point, 3 unit lengths horizontally to the left as the right side, and 16 unit lengths horizontally to the left as the left side, forming an area with a length of 13 units and a height of 12 units.

[0160] The third interactive area is an area with the pelvic bone point as the starting point, 2 units of length vertically upward as the upper side, 10 units of length vertically downward as the lower side, and with the top of the head bone point as the starting point, 3 units of length horizontally to the right as the left side, and 16 units of length horizontally to the right as the right side, forming an area with a length of 13 units and a height of 12 units.

[0161] The control instruction of the fourth interactive area is to determine D, and the control instruction of the fifth interactive area is to determine E. The fourth interactive area takes the top bone point as the starting point, 4 unit lengths vertically upward as the upper side, 5 unit lengths vertically downward as the lower side, takes the top bone point as the starting point, 7 unit lengths horizontally to the left as the right side, and 16 unit lengths horizontally to the left as the left side, forming an area with a length of 11 units and a height of 9 units.

[0162] The fifth interactive area is an area with the top of the head bone point as the starting point, 4 units of length vertically upward as the upper side, 5 units of length vertically downward as the lower side, and 11 units of length with the top of the head bone point as the starting point, 7 units of length horizontally to the right as the left side, and 16 units of length horizontally to the right as the right side, and a height of 9 units of length.

[0163] S124 identifies the interaction posture according to the skeleton point information;

[0164] S124.1 obtain the distance between the left hand skeleton point and the right hand skeleton point;

[0165] S124.2 compares the distance between the left hand skeleton point and the right hand skeleton point with the threshold value T. If the distance between the left hand skeleton point and the right hand skeleton point is less than or equal to the threshold value T, the interaction gesture is recognized. In this embodiment, the threshold value T is 15 cm. If the distance between the left hand skeleton point and the right hand skeleton point is less than or equal to the threshold value T, it is judged that the user makes an interaction gesture and the interaction gesture is recognized. For example, in this embodiment, the interaction gesture is clapping, that is, the user makes a clapping action; if the distance between the left hand skeleton point and the right hand skeleton point is greater than the threshold value T, the user does not make a clapping action; in this embodiment, the specific interaction gesture is only used as an example and is not limited.

[0166] S124.3 When the user makes an interactive gesture, the coordinates of the middle point between the left hand bone point and the right hand bone point are obtained;

[0167] S125 determines whether the coordinates of the middle point are located in the interactive area;

[0168] S125.1 Determine whether the coordinates of the middle point are located in the interactive area;

[0169] S125.11 If the coordinates of the middle point are located in one of the interactive areas, the interactive area is activated, and the control instruction corresponding to the interactive area is output; the other interactive areas are in an inactive state;

[0170] S125.12 When the control instruction corresponding to the interactive area is output, the interactive area is activated and the other interactive areas are inactive; the area of ​​the interactive area in the activated state increases, and the area of ​​the interactive area in the inactive state decreases.

[0171] In this embodiment, the length of the interactive area in the activated state increases by 1.5 unit lengths, and the height increases by 0.5 unit lengths. Specifically, in this embodiment, the length of the interactive area in the activated state increases to the left and right sides successively, that is, expands outward by 0.75 unit lengths, and the height increases to the upper and lower ends successively, that is, expands outward by 0.25 unit lengths. The length of the interactive area in the inactivated state decreases by 2 unit lengths, and the height decreases by 1 unit length. Specifically, the length of the interactive area in the inactivated state decreases to the left and right sides successively, that is, shrinks inward by 1 unit length, and the height decreases to the upper and lower ends, that is, shrinks inward by 0.5 unit length.

[0172] When the control instruction corresponding to the interactive area is not output, the area of ​​the interactive area does not change. After the control instruction is output, the area of ​​the interactive area is adjusted. After the control instruction is executed, the area of ​​the interactive area is restored to the original state.

[0173] When the user makes an interactive gesture in the first interactive area, the fitness mirror recognizes the interactive gesture and outputs a control instruction corresponding to the first interactive area. The area of ​​the first interactive area expands, that is, the upper side increases by 0.25 unit length vertically upward, the lower side increases by 0.25 unit length vertically downward, the left side increases by 0.75 unit length horizontally to the left, and the right side increases by 0.75 unit length horizontally to the right.

[0174] All parameters, dimensions, materials and configurations described herein are intended to be exemplary, and actual parameters, dimensions, materials and / or configurations will depend on one or more specific applications for which the present invention is used. It should be understood that the foregoing embodiments are presented primarily by way of example, and within the scope of the appended claims and their equivalents, embodiments of the present invention may be practiced in a manner other than as specifically described and claimed. Inventive embodiments of the present disclosure relate to each individual feature, system, article, material, kit and / or method described herein.

[0175] In addition, if such features, systems, articles, materials, kits and / or methods are not mutually contradictory, any combination of two or more such features, systems, articles, materials, kits and / or methods is included in the inventive scope of the present disclosure. Without departing from the spirit of the present disclosure, other substitutions, modifications, changes and omissions may be made in the design, operating conditions and arrangement of the corresponding elements of the exemplary embodiments. The use of numerical ranges does not exclude equivalents that fall outside the range and perform the same function in the same manner to produce the same results.

[0176] The above embodiments may be implemented in a variety of ways. For example, the embodiments may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be executed on a suitable processor or set of processors, whether provided in a single computer or distributed among multiple computers.

[0177] Furthermore, the computer may be embodied in any of a variety of forms, such as a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer. In addition, the computer may be embedded in a device not generally regarded as a computer but having suitable processing capabilities, including a personal digital assistant (PDA), a smart phone, or any other suitable portable or fixed electronic device.

[0178] In addition, the computer may have one or more input and output devices. Among other things, these devices may also be used to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visual presentation of the output and a speaker or other sound-generating device for audible presentation of the output. Examples of input devices that may be used for a user interface include a keyboard and a pointing device, such as a mouse, a touch pad, and a digitized tablet computer. As another example, a computer may receive input information through speech recognition or other audible formats.

[0179] Such computers may be interconnected in a suitable form through one or more networks, including local area networks or wide area networks, such as enterprise networks, intelligent networks (IN) or the Internet. Such networks may be based on suitable technologies, may operate according to suitable protocols, and may include wireless networks, wired networks or fiber optic networks.

[0180] The various methods or processes outlined herein may be encoded as software that can be executed on one or more processors using any of a variety of operating systems or platforms. In addition, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may also be compiled into executable machine language code or intermediate code that is executed on a framework or virtual machine. Some embodiments may specifically employ one or more of a specific operating system or platform and a specific programming language and / or scripting tool to facilitate execution.

[0181] In addition, various inventive concepts can be embodied as one or more methods, wherein at least one example of these methods has been provided. The actions performed as a part of the method can be sorted in different ways in some cases. Therefore, in some embodiments of the invention, the corresponding actions of a given method can be performed in an order different from the order specifically shown, which can include performing some actions simultaneously (even if such actions are shown as sequential actions in exemplary embodiments).

[0182] All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety.

[0183] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0184] Unless clearly indicated to the contrary, the indefinite articles "a" and "an" as used in this specification and the claims should be understood to mean "at least one".

[0185] The phrase "and / or" used in this specification and claims should be understood to mean "one or two" elements so combined, i.e., elements that exist in combination in some cases and separately in other cases. Multiple elements listed with "and / or" should be interpreted in the same manner, i.e., "one or more" elements are so combined. In addition to the elements specifically represented by the "and / or" clause, other elements may optionally be present, whether related or unrelated to those specifically represented. Thus, as a non-limiting example, when used in conjunction with open language such as "comprising", a reference to "A and / or B" may refer to only A (optionally including elements other than B) in one embodiment; to only B (optionally including elements other than A) in another embodiment; to both A and B (optionally including other elements) in yet another embodiment; and so on.

[0186] As used in this specification and claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as inclusive, that is, including at least one, but also including many elements or more than one element in an element list and (optionally) additional unlisted items. Only when it is clearly indicated that they are contradictory, otherwise, such as "only one" or "exactly one" or "consisting of..." when used in the claims will refer to an element that just includes many elements or an element list. In general, when the term "or" as used herein is followed by exclusive terms such as "one of the two", "one in", "only one of" or "exactly one of", it should only be interpreted as indicating exclusive alternatives (that is, "one or the other but not two"). Its ordinary meaning when "consisting essentially of..." is used in the claims is as used in the field of patent law.

[0187] As used in this specification and claims, with respect to a list of one or more elements, the phrase "at least one" should be understood to mean at least one element selected from any one or more elements in the list of elements, but does not necessarily include at least one element of each element specifically listed in the list of elements, and does not exclude any combination of elements in the list of elements. This definition also allows for the optional presence of elements other than the elements specifically represented in the list of elements to which the phrase "at least one" refers, whether related or unrelated to those specifically represented. Thus, as a non-limiting example, "at least one of A and B" (or equivalently, "at least one of A or B", or equivalently "at least one of A and / or B") may refer to at least one A, optionally including more than one A, without the presence of B (and optionally including elements other than B) in one embodiment; to at least one B, optionally including more than one B, without the presence of A (and optionally including elements other than A) in another embodiment; to at least one A, optionally including more than one A and at least one B, optionally including more than one B (and optionally including other elements) in yet another embodiment; and so on.

[0188] In the claims and the description above, all transitional phrases such as "includes," "comprising," "carrying," "having," "containing," "involving," "holding," "consisting of," etc. should be understood as open-ended, i.e., including but not limited to.

[0189] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0190] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An interactive intelligent fitness mirror device, characterized in that: include A display, for displaying a user interface, wherein the user interface has at least one interactive object; a mirror, the mirror being used to display the image of the user on the mirror, and the display showing the user interface to the user through the mirror; A sensor for collecting user motion data of the user; A processor, coupled to the display and the sensor, generates a control instruction based on the user action data collected by the sensor, wherein the control instruction controls the user interface to complete a specified operation; The processor is also used to obtain the motion trajectory of the user action based on the user action data collected by the sensor, and control the floating icon to be displayed on the user interface according to the motion trajectory, and the floating icon is used to indicate the position indicated by the user action in the user interface.

2. The interactive intelligent fitness mirror device according to claim 1, characterized in that: The interactive object of the user interface includes a floating icon wake-up interaction. When the processor determines that the action indicated by the user action data collected by the sensor matches the floating icon wake-up action corresponding to the preset floating icon wake-up interaction, the floating icon is enabled so that the floating icon moves with the user's action.

3. The interactive intelligent fitness mirror device according to claim 1 or 2, characterized in that: When the processor determines that the user has completed the selection operation in the current user interface, the processor turns off the floating icon function so that the floating icon no longer moves following the user's action.

4. The interactive intelligent fitness mirror device according to claim 3, characterized in that: When the processor determines that the floating icon moves to the position of the user interface object in the user interface according to the user action data, and determines that the user performs the selection operation at the position of the user interface object, the processor executes the operation corresponding to the user interface object.

5. The interactive intelligent fitness mirror device according to claim 4, characterized in that: The selection operation includes any one of gesture pausing, gesture grabbing, pushing the palm forward, pushing the palm backward, waving the hand, and making a fist.

6. The interactive intelligent fitness mirror device according to claim 1 or 2, characterized in that: The user interface also includes page animations, which include page switching animations and page interaction animations. The page switching animations are used to produce dynamic effects when the page is operated or changed, and the page interaction effects are used to provide guidance to the user when interacting.

7. The interactive intelligent fitness mirror device according to claim 1, characterized in that: The processor generates the user's skeleton point coordinates based on the user action data collected by the sensor, obtains the coordinates corresponding to the floating icon on the user interface according to the skeleton point coordinates, calls the motion trajectory corresponding to the coordinates, and controls the floating icon to be displayed on the user interface according to the motion trajectory.

8. The interactive intelligent fitness mirror device according to claim 1, characterized in that: The user interface also includes a wake-up page, and the wake-up page is used to switch to other user interfaces when the sensor collects user action data that matches a preset wake-up action.

9. The interactive intelligent fitness mirror device according to claim 1, characterized in that: The user interface also includes at least one page object, which is used to display at least one page information, and the page information includes exercise course videos, visual data, operable interactive objects, and interactive guides.

10. The interactive intelligent fitness mirror device according to claim 1, characterized in that: The user actions collected by the sensor include static actions and dynamic actions. The static action is that the user makes a predetermined posture with a predetermined body part and maintains it for a specified time, and the dynamic action is that the user makes a predetermined action with a predetermined body part.

11. The interactive intelligent fitness mirror device according to claim 10, characterized in that: Static movements include raising the left / right hand and holding it for a preset time, raising the left / right leg and holding it for a preset time, or raising both hands above the head and making a heart shape and holding it for a preset time.

12. The interactive intelligent fitness mirror device according to claim 10, characterized in that: Dynamic movements include sliding the left / right hand horizontally, sliding the left / right hand up and down, clapping hands, jumping up or squatting.

13. The interactive intelligent fitness mirror device according to claim 1, characterized in that: The sensor is also used to collect characteristic data of the user. The processor compares the characteristic data input by the user with the characteristic data in the database, determines the user who is using the mirror device based on the comparison result, and allows the mirror device to adjust the graphical user interface based on the recognition result. The characteristic data of the user includes the user's bone points, voiceprint, weight or resting heart rate data. The database records the mapping relationship between the user and the characteristic data.

Images