Interactive fitness mirror device and method

Abstract

This invention discloses an interactive fitness mirror device and method, comprising: a communication interface for receiving video images; a display for displaying a user interface or exercise interface, the user interface having at least one interactive object providing control actions for a user to perform; the exercise interface including the video images received through the communication interface; a mirror for displaying a user's reflection on the mirror, the display showing the user interface to the user through the mirror; a sensor for collecting the user's motion data; and a processor coupled to the display and the sensor for determining an operation to be performed based on the user's motion data, based at least one of the actions displayed in the video images of the exercise interface and the control actions provided by the interactive object for the user to perform. This invention effectively avoids misinterpretation of motion commands caused by following movements during exercise.

Description

Technical Field

[0001] This invention relates to the fields of smart home and smart fitness, specifically to an interactive fitness mirror device and method. Background Technology

[0002] With rapid economic development and an accelerated pace of life, the high-speed lifestyle and heavy workload have led to various health problems among urban dwellers, making fitness a hot topic. Smart fitness mirrors are a new type of fitness product that integrates artificial intelligence, hardware, and content services.

[0003] Currently, most people who exercise at home use smart fitness mirrors to watch videos and learn training movements. These mirrors function as both a mirror and a high-definition image of the instructor. However, existing smart fitness mirrors have cumbersome interaction methods, generally requiring control via external smart devices (such as smartphones) or touchscreens. Controlling via external devices is complex and requires pairing the device with the mirror. Touchscreen controls are prone to fingerprints and sweat, affecting the screen's display. Furthermore, current fitness mirrors are bulky, taking up considerable home space, and only display fitness content and related data, resulting in low usage frequency in daily life. Therefore, the inventors have optimized the interaction between the smart fitness mirror and the user by adding a personalized interface and new interaction methods, thereby increasing the frequency of use of the smart fitness mirror in daily life.

[0004] As research and development progressed, the inventors discovered that since one of the main functions of smart fitness mirrors is to collect users' movement movements, and the user's control actions during exercise are easily confused with the movement movements, the fitness mirror may perform operations incorrectly, thereby affecting the user's interactive experience and usage efficiency. Summary of the Invention

[0005] One object of the present invention is to provide an interactive fitness mirror device and method, which avoids misjudgment by comparing the motion data collected by the sensor with the motion displayed in the video image of the exercise interface or the control action provided by the interactive object.

[0006] This objective is achieved using the following technical solution:

[0007] An example of a smart fitness mirror includes a communication interface for receiving video images from a fitness instructor, a display operatively coupled to the communication interface to display the video images from the fitness instructor, and a mirror positioned in front of the display to reflect the image of a person opposite the display. The mirror displays a user's image on the mirror, the display shows the user interface to the user through the mirror, and the fitness instructor's video images are transmitted to the person opposite the display such that the fitness instructor's video images appear to be superimposed on a portion of the person's image.

[0008] An example of an interactive fitness method includes the following steps: (1) streaming fitness content to an interactive video system, the interactive video system including a mirror for displaying a user’s reflection on the mirror and a display set on the mirror; (2) displaying the fitness content to the user via the display and the mirror; and (3) using the mirror to reflect the user’s image so that the user’s image is at least partially superimposed on the fitness content displayed via the partial reflection of the display and the mirror.

[0009] An example of a method for using a smart fitness mirror includes the following steps: when displaying fitness content to a user on a video display behind a partially transmissive mirror: (1) reflecting the user's image using the partially transmissive mirror; (2) measuring the user's heart rate using a heart rate monitor attached to the user; (3) transmitting the heart rate from the heart rate monitor to an antenna operatively coupled to the video display; (4) displaying the user's heart rate on the video display; and (5) displaying the user's target heart rate on the video display.

[0010] To add personalized interfaces and new interaction methods to smart fitness mirrors, optimize the interaction between smart fitness mirrors and users, and increase the frequency of use of smart fitness mirrors in daily life, this invention designs an operating system Launcher for smart fitness mirrors, specifically including: (1) GUI interface; (2) system control method for motion control; (3) system control method for voice control; (4) multimodal fusion control method; (5) executable control commands; (6) data acquisition and management method; and (7) some specific function management methods. A system and operation method are customized for smart fitness mirrors, so that the function of smart fitness mirrors is not limited to watching fitness exercise videos, but can also be an important part of smart homes, making it easier for users to view more information and improve usage efficiency. At the same time, the customized operation method overcomes the drawback of traditional smart fitness mirrors that require touch control and are prone to leaving fingerprints on some reflective surfaces. Since the core function of smart fitness mirrors is to allow users to compare their own movements when watching fitness exercise videos, leaving fingerprints on some reflective surfaces has a significant impact on the function of smart fitness mirrors, causing most users to be unwilling to use touch interaction. However, using smart terminals to control smart fitness mirrors is not intuitive and is quite troublesome. Motion control avoids these problems. While gesture control is used in other electronic products, its application in smart fitness mirrors represents a breakthrough. Because smart fitness mirrors include reflective surfaces, users can clearly see their movements, reducing the possibility of accidental control or misoperation. Furthermore, since motion recognition is a core function of smart fitness mirrors, they can achieve higher precision and more complex motion recognition without additional costs, resulting in a much wider range of control commands compared to existing gesture control. Finally, because the primary function of smart fitness mirrors is exercise, applying motion control to them also requires preventing accidental touches during exercise movements, a significant difference from traditional gesture control.

[0011] However, the primary function of the fitness mirror of this invention is exercise and fitness. Users following or randomly performing movements in exercise videos may lead to accidental control activation. Therefore, this invention optimizes the aforementioned interaction method by determining the operation to be performed based on the user's movement data, at least based on at least one of the movements displayed in the video image on the exercise interface and the control actions provided by the interactive object for the user to execute. This helps determine whether the user is performing motion control or simply following the exercise video, avoiding misjudgments that could affect user comfort and fitness efficiency.

[0012] Preferably, the control actions and control instructions preset in the mirror device are matched. The control instructions are used to control the user interface to complete a specified operation. When the processor determines, based on the action data, that the action indicated by the action data matches the control action, the processor executes the control instruction corresponding to the control action.

[0013] Preferably, to avoid misjudgment, when the processor determines, based on the motion data, that the motion data indicates a motion that matches the motion displayed in the video image of the exercise interface, the processor does not execute the operation of the control instruction.

[0014] Preferably, the processor determines the operation matching the action indicated by the action data based on the control action and the action displayed in the video image.

[0015] The processor can compare the motion data, the control action, and the actions displayed in the video image to determine which one matches; alternatively, it can first compare the motion data with the control action, and if they match, then it will not compare it with the actions displayed in the video image; or it can first compare it with the actions displayed in the video image, and if they match, then it will not compare it with the control action.

[0016] All combinations of the foregoing and additional concepts discussed in more detail below (assuming such concepts are not contradictory) can be contemplated as part of the inventive subject matter disclosed herein. Specifically, all combinations of the claimed subject matter appearing at the end of this disclosure can be contemplated as part of the inventive subject matter disclosed herein. Terms expressly adopted herein and that may also appear in any disclosure incorporated by reference should be given the meaning most consistent with the specific concepts disclosed herein.

[0017] The present invention provides one or more technical solutions, which have at least the following technical effects or advantages: The present invention determines the operation to be performed based on the user's action data by at least one of the actions displayed in the video image of the exercise interface and the control actions provided by the interactive object for the user to perform, thereby avoiding misjudgment of action commands caused by following actions during exercise.

[0018] Meanwhile, the fitness mirror features one or more user interface objects that can be switched, enriching the mirror's functionality, enhancing the user experience of multiple user interface objects, and increasing users' motivation and interest in exercising. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0020] Figure 1 A block diagram of an exemplary smart fitness mirror;

[0021] Figure 2 This is an example GUI page on a smart fitness mirror;

[0022] Figure 3 This is an example GUI page on a smart fitness mirror;

[0023] Figure 4 This is an example GUI page on a smart fitness mirror;

[0024] Figure 5-a This is an example GUI page on a smart fitness mirror;

[0025] Figure 5-b This is an example GUI page on a smart fitness mirror;

[0026] Figure 6 This is an example GUI page on a smart fitness mirror;

[0027] Figure 7 This is an example GUI page on a smart fitness mirror;

[0028] Figure 8 This is an example GUI page on a smart fitness mirror;

[0029] Figure 9 This is an example GUI page on a smart fitness mirror;

[0030] Figure 10 This is a schematic diagram of motion recognition for a smart fitness mirror. Detailed Implementation

[0031] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, where there is no conflict, the embodiments of the present invention and the features thereof can be combined with each other.

[0032] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0033] Those skilled in the art should understand that, in the disclosure of this invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting this invention.

[0034] Therefore, this patent relates to an interactive fitness mirror device (also referred to as a "smart fitness mirror" and an "interactive training system") and a method of using the interactive training device. The smart fitness mirror includes a display configured to display exercise content (pre-recorded video or live stream) and an interface that allows users to personalize their workouts. Furthermore, the smart fitness mirror can allow users and / or instructors to interact with each other during workouts in a manner similar to a regular workout conducted in a gym or small fitness studio where the user and instructor are in the same room (e.g., providing feedback to the instructor on the pace of the workout, correcting the user's form during a specific fitness program).

[0035] An example of a smart fitness mirror.

[0036] Image recognition sensors (e.g., cameras) in smart fitness mirrors can be used to acquire video and / or still images of the user while they are engaged in activities (e.g., exercising). The user's video can then be shared with a coach to allow the coach to observe and provide guidance during the workout. Videos can also be shared with other users on other smart fitness mirrors for comparison or competition. The user's video can also be displayed in real-time on a display or stored for later playback. For example, by providing a visual comparison between the user and a coach, the user's video can be used for self-assessment during or after a workout. Stored videos can also allow users to evaluate their progress or improvement over time when performing similar workouts.

[0037] The sensor can be configured to acquire dynamic and / or static images of the user when the user is performing an activity (e.g., exercise), and determine whether the user's actions match the standard actions based on the dynamic and / or static images and output the determination result. If the user's actions do not match the standard actions, action correction information is output.

[0038] While determining whether the user's action matches the standard action, the system also determines whether the user's action matches the action displayed in the video on the exercise interface based on dynamic and / or static images. If the user's action does not match the action displayed in the video on the exercise interface, no accidental touch command will be output. At the same time, if the user's action matches the standard action, the system will control the user interface to complete the specified operation based on the control command generated by matching the action data with the standard action.

[0039] If the user's action matches the action displayed in the video on the exercise interface, an accidental touch command is output. Regardless of whether the user's action matches the standard action, the accidental touch command will prevent the exercise interface from completing the specified operation.

[0040] In this embodiment, the comparison of whether the motion data matches the motion displayed in the video on the exercise interface, and the comparison of whether the motion data matches the standard motion are performed simultaneously, to ensure that the user can react quickly after making a motion.

[0041] The processor determines whether the action indicated by the user data matches the action displayed in the video. This can be understood as a strong correlation. The user's action and the action displayed in the video do not need to be completely identical. It is sufficient to determine that the user's action is following or imitating the action displayed in the video, or that there is a high degree of consistency.

[0042] Furthermore, control actions correspond to control instructions. Matching a control action means that the processor will execute the control instruction that matches that control action.

[0043] The operation to be performed based on the user's action data is determined based on at least one of the actions displayed in the video footage of the exercise interface and the control actions provided by the interactive object for the user to perform. The performed operation is, when it matches a control action, generating a control command based on the control action and executing the operation corresponding to the control command. When it matches an action displayed in the video footage, the performed operation can be an evaluation based on standard actions within the video content. An example smart fitness mirror.

[0044] Based on the above embodiments, the sensor can be configured to acquire dynamic and / or static images of the user when the user performs an activity (e.g., exercise). After acquiring the dynamic and / or static images, the sensor first compares them with standard movements. If the user's movements do not match the standard movements, movement correction information is output. If the user's movements match the standard movements, the sensor compares the dynamic and / or static images with the movements displayed in the video on the exercise interface. If the user's movements do not match the movements displayed in the video on the exercise interface, a control command generated based on the matching of the movement data with the standard movements is used to control the user interface to complete the specified operation. If the user's movements match the movements displayed in the video on the exercise interface, movement correction information is output.

[0045] In this embodiment, dynamic and / or static images are first compared with standard actions. Based on the comparison results, it is selected whether to compare the actions displayed in the video images of the exercise interface. If the dynamic and / or static images do not match the standard actions, there is no need to compare the actions displayed in the video images of the exercise interface. If the dynamic and / or static images match the standard actions, further comparison of the actions displayed in the video images of the exercise interface is required, which simplifies the operation and improves the interaction efficiency.

[0046] An example of a smart fitness mirror.

[0047] Based on the above embodiments, the device can be configured to acquire dynamic and / or static images of the user when the user is engaged in an activity (e.g., exercise). After acquiring the dynamic and / or static images, the device first compares the dynamic and / or static images with the actions displayed in the video on the exercise interface. If the user's actions match the actions displayed in the video on the exercise interface, a mis-touch command is output, without needing to compare the dynamic and / or static images with standard actions. If the user's actions do not match the actions displayed in the video on the exercise interface, the device continues to compare the action data with standard actions. If they match, a control command generated based on the matching of the action data and standard actions controls the device to complete the specified operation; otherwise, if they do not match, action correction information is output.

[0048] In this embodiment, the dynamic and / or static images are first compared with the actions displayed in the video images of the exercise interface. Based on the comparison results, it is selected whether to compare the dynamic and / or static images with the standard actions. If the dynamic and / or static images match the actions displayed in the video images of the exercise interface, it is not necessary to compare them with the standard actions.

[0049] An example of a smart fitness mirror.

[0050] Based on the above embodiments, if this device is on the user interface, it is only necessary to compare dynamic and / or static images with standard actions, and it is not necessary to compare user actions with the actions displayed in the video images of the exercise interface.

[0051] If this device is on the exercise interface, it can be one or more of the comparison methods described in the above embodiments.

[0052] In one or more embodiments, the sensor may also be an image recognition sensor (camera), which acquires dynamic and / or static images of the user, processes the acquired dynamic and / or static images into skeletal point images, and determines whether the user's actions match the actions displayed in the standard movements or video images of the exercise interface based on the skeletal point images and outputs the determination results. It may also process dynamic and / or static images in real time when the user is using the smart fitness mirror or after the exercise is completed, so as to control the smart fitness mirror or export the user's biometric data based on the user's movement and actions.

[0053] The sensor can also be an infrared recognition sensor in a smart fitness mirror. This infrared sensor can acquire thermal images of the user during activity (e.g., exercise), and based on the thermal images, determine the body parts being exercised, the muscles being used, and the calories burned. It can also determine whether the user's movements match standard movements based on the exercised and used body parts, and output the result. If the user's movements do not match the standard movements, it outputs movement correction information.

[0054] The sensor can also be a voice recognition sensor (microphone) in a smart fitness mirror. The voice recognition sensor (microphone) can be used to acquire and process the user's voice information in real time when the user is using the smart fitness mirror, and to control the smart fitness mirror or have a conversation with the user based on the user's voice information.

[0055] An example of a smart fitness mirror.

[0056] Figure 1An exemplary representation of a smart fitness mirror is shown. The smart fitness mirror may include a processor 110 for partially controlling the operation of various sub-components within the smart fitness mirror and managing data flows into / out of the smart fitness mirror (e.g., video content, audio from a coach or user, biometric feedback analysis). The smart fitness mirror may include a display 120 for displaying video content, a graphical user interface (GUI) with which the user can interact and control the smart fitness mirror, biometric feedback data, and / or other visual content. A sensor 130 may be coupled to the processor 110 to acquire user-related data. An antenna 140 may be coupled to the processor 110 to provide data transmission between the smart fitness mirror and another device (e.g., a remote control, a biometric sensor, a wireless router). The antenna 140 may include multiple transmitters and receivers, each tailored for a specific frequency and / or wireless standard (e.g., Bluetooth, 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 2G, 3G, 4G, 4G LTE, 5G). Amplifier 150 can be coupled to processor 110 to receive audio signals from processor 110 for outputting subsequent sound through left speaker 152 and / or right speaker 154.

[0057] Smart fitness mirrors can also include Figure 1 Additional components not shown. For example, a smart fitness mirror may include a switch-mode power supply (SMPS), a switch, and onboard memory and storage devices (non-volatile and / or volatile memory), including but not limited to hard disk drives (HDDs), solid-state drives (SDDs), flash memory, random access memory (RAM), and secure digital cards (SD cards). This onboard memory and / or storage device may be used to store firmware and / or software for the operation of the smart fitness mirror. As mentioned above, the onboard memory and / or storage device may also be used to (temporarily and / or permanently) store other data, including but not limited to video content, audio, user video, biometric feedback data, and user settings. In another example, the smart fitness mirror may include various components for mounting and supporting the smart fitness mirror.

[0058] Antenna 140 may include multiple antennas, each acting as a receiver and / or transmitter to communicate with various external devices, such as a user's smart terminal (e.g., a computer, smartphone, tablet), external sensors (e.g., a heart rate monitor, inertial sensor, body fat scale), and / or a remote server or cloud server for streaming or playing video content. Again, antenna 140 may comply with various wireless standards, including but not limited to Bluetooth, 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 2G, 3G, 4G, 4G LTE, and 5G standards.

[0059] Sensor 130 may include multiple sensors, which may be one or more of an image recognition sensor (camera), an infrared recognition sensor, or a voice recognition sensor (microphone).

[0060] Image recognition sensors (cameras) in smart fitness mirrors can be used to acquire video and / or still images of users while they are engaged in activities such as exercise. These videos can then be shared with a coach to allow the coach to observe and provide guidance during the workout. Videos can also be shared with other users on other smart fitness mirrors for comparison or competition. User videos can also be displayed in real-time on a display or stored for later playback. For example, by providing a visual comparison between the user and a coach, the user's video can be used for self-assessment during or after a workout. Stored videos can also allow users to evaluate their progress or improvement over time when performing similar workouts.

[0061] Graphical User Interface (GUI)

[0062] The smart fitness mirror can display a graphical user interface (GUI) on the display 120 to facilitate user interaction with the smart fitness mirror.

[0063] Graphical user interface (GUI) includes, but is not limited to, wake-up page, main page, workout page, and other pages.

[0064] The wake-up page refers to the initial page displayed when the smart fitness mirror is activated. It is used to display some basic page information. Only when the user performs a specified operation will the smart fitness mirror be fully activated and enter the main page, exercise page, or other pages.

[0065] The main page displays at least one piece of page information, which is used to show users data, interactive objects, or interactive guidance. The main page displays preset page information or user-defined page information. Users can also control the smart fitness mirror by performing specific operations on the main page, including but not limited to page switching, function switching, or accessing third-party applications.

[0066] The workout page displays a video playback window to guide users through exercises. It can also display at least one of the following: course playback information (such as total course duration and played duration), accessory connection information (such as heart rate monitor connection information), and user physiological information (such as heart rate, blood pressure, and calories burned). The workout page can be accessed from the main page, or by waking up the smart fitness mirror and then returning to it. It can also be accessed directly when the smart fitness mirror is turned on.

[0067] Other pages display pre-installed function pages (such as multimedia player pages, weather display pages, food recommendation pages, etc.) or third-party integrated application pages (such as WeChat, Douyin, Himalaya, etc.) accessible through the main page. Other pages are generally accessed from the main page or through a wake-up page and then returned to the main page or wake-up page.

[0068] Page Information

[0069] The graphical user interface (GUI) can display page information through page objects. Each page object is used to display at least one or a set of page information. The displayed page information may be information stored in the local storage device of the smart fitness mirror, or information received by the smart fitness mirror from other devices through a communication interface, including but not limited to switchable course preview information, user information, environmental information, multimedia information, external device information, etc.

[0070] The course preview information includes, but is not limited to, instructor name, course type, course duration, course difficulty, course tags, and / or estimated calorie expenditure. The course type includes, but is not limited to, HIIT, aerobics, yoga, strength training, combat training, Barre, Pilates, stretching, dance, meditation, and challenges, all displayed to users via a smart fitness mirror. Course tags include, but are not limited to, whether the course is AI-recognized, whether it is a parent-child course, the main body parts involved (e.g., full body, arms, waist, abdomen, legs), and the equipment required (e.g., no equipment, yoga mat, resistance band, resistance belt, dumbbells, etc.). The page object displaying the course preview information is also called a course card.

[0071] The user information includes, but is not limited to, usage history, physiological information, social information, and calendar information. Usage history includes, but is not limited to, displaying completed courses, saved courses, best historical scores, and class frequency to the user via the smart fitness mirror. Physiological information includes, but is not limited to, displaying height, weight, gender, age, heart rate, blood pressure, and injury history to the user via the smart fitness mirror. Social information includes, but is not limited to, displaying information related to people / friends followed by the user via the smart fitness mirror, or information related to groups / interest groups joined by the user. This information may specifically include invitations to complete designated courses, invitations to one-on-one or group challenges, user rankings, and updates uploaded by the user or other users. These updates can be synchronized to other social platforms (such as WeChat, Weibo, Facebook, Twitter, etc.). Calendar information includes, but is not limited to, displaying the user's schedule, to-do items, and course plans via the smart fitness mirror. The page object displaying user information is also called a user card.

[0072] The environmental information includes, but is not limited to, weather information. This weather information includes, but is not limited to, displaying weather forecasts, temperature, humidity, and clothing recommendations to users via a smart fitness mirror. The page object displaying the weather information is also called a weather card.

[0073] The multimedia information includes, but is not limited to, music and video information played through the smart fitness mirror. The page object displaying the multimedia information is also called a multimedia card.

[0074] The external device information includes, but is not limited to, information about other fitness devices connected to the smart fitness mirror, and information about other smart furniture (such as robot vacuums, smart speakers, smart gateways, etc.). The page object displaying the external device information is also called a device card.

[0075] A graphical user interface (GUI) can also display page information through interactive objects, which can be icons, text, highlighted selections, other forms that prompt the user to interact, or combinations thereof. These interactive objects are used to demonstrate controls or operations that the user can perform (such as page switching, selection, pausing / playing a course, etc.). Furthermore, these interactive objects can be combined with pages or page objects of the GUI to display different interactive functions.

[0076] The page objects and interactive objects can be fixed on each page, or they can be displayed in pop-up windows or scrolling playback on each page.

[0077] Some example pages

[0078] Option 1 Figure 2 , 3 As shown in 4, 5-a, and 5-b, where Figure 2 This is the first main page. Figure 3 It is a page switching process. Figure 4 , 5-a5-b represent the second, third, and fourth main pages, respectively. The first main page displays multiple page information through various page objects, including weather cards, multimedia cards, and recommended course cards. It also includes an interactive object in the upper right corner for navigating to the next page, as well as an interactive object combined with the course cards. Users can enter the second main page by interacting with the interactive object combined with the course cards, or they can enter the page switching page by interacting with the interactive object used to enter the next page. The page switching page includes preview images of the first, second, and third main pages, highlighting one of them. The top of the page switching page displays interactive objects for returning and confirming entry to the highlighted page, and the bottom displays an interactive object for switching the highlighted page. Users can switch the highlighted page by interacting with the interactive object for switching the highlighted page, and enter the selected page by confirming entry to the highlighted page. When a user selects and enters the second main page, it includes multiple course cards with one highlighted. The top of the second main page displays interactive objects for returning to and confirming entry into the highlighted course card, while the bottom displays an interactive object for toggling the highlighted course card. Users can switch the highlighted course card by using the toggling interactive object and enter the selected course card's workout page by using the confirm entry interactive object. When a user selects and enters the third main page, it includes multiple page objects displayed side-by-side. The third main page also displays the selected page object and an interactive object for entering it. The bottom of the third main page displays an interactive object for toggling the page object. Users can switch the displayed page object by using the toggling interactive object and enter the page corresponding to the selected page object by using the enter interactive object.

[0079] Option 2 Figure 6 , 7 As shown in Figures 8 and 9, among which Figure 6 It's the fifth main page. Figure 7 It is the sixth main page. Figure 8 , 9These are the seventh and eighth main pages. The fifth main page displays multiple page information through various page objects, including weather cards, multimedia cards, and recommended course cards. It also includes interactive objects at the bottom for switching main pages and interactive objects that can be combined with course cards. Users can enter the eighth main page by interacting with the interactive object combined with course cards, or switch the displayed main page by interacting with the main page switching object. When the user selects the sixth main page, it includes multiple page objects displayed side-by-side. The sixth main page also displays the selected page object and the interactive object for entering the selected page object. At the bottom of the sixth main page, interactive objects for switching main pages are also displayed. Users can switch the displayed main page by interacting with the main page switching object, and enter the page corresponding to the selected page object by interacting with the interactive object that enters the selected page object. When a user selects the seventh main page as the displayed main page, the seventh main page includes multiple page objects displayed side-by-side. It also displays the selected page object and an interactive object for entering the selected page object. Furthermore, an interactive object for switching main pages is displayed at the bottom of the seventh main page. Users can switch the displayed main page by interacting with the main page switching object, and enter the page corresponding to the selected page object by interacting with the page entering the selected page object. When a user selects the page corresponding to the selected page object as the eighth main page, the eighth main page includes multiple course cards displayed side-by-side. It also displays the selected course card and an interactive object for entering the selected course card. Furthermore, an interactive object for switching course cards is displayed at the bottom of the eighth main page. Users can switch the displayed course cards by interacting with the course card switching object, and enter the exercise page corresponding to the selected course card by interacting with the page entering the selected course card.

[0080] The page transition page includes preview images of the first, second, and third main pages, highlighting one of them. The top of the page transition page displays interactive objects for returning to and confirming entry into the highlighted page, and the bottom of the page transition page also displays an interactive object for switching the highlighted page. Users can switch the highlighted page by interacting with the interactive object for switching the highlighted page, and enter the selected page by confirming entry into the highlighted page. When the user selects and enters the second main page, the second main page includes multiple course cards, highlighting one of them. The top of the second main page also displays interactive objects for returning to and confirming entry into the highlighted course card, and the bottom of the page transition page also displays an interactive object for switching the highlighted course card. Users can switch the highlighted course card by interacting with the interactive object for switching the highlighted course card, and enter the exercise page of the selected course card by confirming entry into the highlighted course card. When a user selects and enters the third main page, the third main page includes multiple page objects displayed side by side. The third main page also displays the selected page object and the interactive object for entering the selected page object. At the bottom of the third main page, there is also an interactive object for switching page objects. Users can switch the displayed page objects by interacting with the interactive object for switching page objects, and enter the page corresponding to the selected page object by executing the interactive object for entering the selected page object.

[0081] Page animation

[0082] The graphical user interface (GUI) also includes page animations, including but not limited to page transition animations and page interaction animations.

[0083] Page transition animations include page turning animations, returning to the previous level, and entering the next level animations. Page transition animations are mainly for page operations and changes, with the aim of providing a better user experience when displaying page operations and changes.

[0084] Page interaction animations include selection confirmation animations and action guidance animations. These animations primarily guide users during interactions, such as indicating the current interaction, its progress, and required actions. More specifically, they include, but are not limited to, highlighting or flashing relevant page information during user interaction, and displaying progress bars. They also include recognizing and displaying prompts or suggested actions when performing action control, and providing completion prompts based on user voice commands during voice control.

[0085] Smart Fitness Mirror Control

[0086] Users can directly interface with the smart fitness mirror to control it (e.g., through motion commands, voice commands, touch commands, etc.). A graphical user interface (GUI) can be provided on the display 120 to facilitate user interaction with the smart fitness mirror.

[0087] Motion command control (including "posture control" and "gesture control")

[0088] Smart fitness mirrors can control graphical user interfaces (GUIs) based on user movements captured by image recognition sensors.

[0089] Specifically, actions controlled by action commands include, but are not limited to, static actions and dynamic actions. Static actions refer to a user assuming a specific posture with a particular body part and maintaining it for a specified time; dynamic actions refer to a user performing a specific movement with a particular body part.

[0090] Specifically, static movements include, but are not limited to, raising the left / right hand, raising the left / right leg, raising both hands overhead to form a heart shape, and various static hand gestures. The duration of these movements is generally 0-5 seconds or longer. Any movement determined by the positional relationship between a specific part of the body and space, the body as a whole, or other specific parts, and maintained for a specified time, should fall within the protection scope of static movements. Further specific examples include: holding the left arm straight at a 15° / 30° angle to the horizontal for 1.5 seconds, holding the left and right arms at a right angle for 1 second, and extending the left arm forward for 0.1 seconds.

[0091] Specifically, dynamic movements include, but are not limited to, horizontal sliding of the left / right hand, vertical sliding of the left / right hand, clapping, jumping, squatting, and various dynamic gestures. Any movement determined by the trajectory of a specific part of the body relative to space, the entire body, or other specific parts should fall within the scope of protection for dynamic movements. Further specific examples include: horizontally sliding the left hand from the left side of the body to the right side; horizontally sliding the left hand from the right side of the body to the left side; horizontally sliding the right hand from the right side of the body to the left side; vertically sliding the left hand from above the shoulder to below the waist; vertically sliding the left hand from above the upper 20% to below the upper 40% of the body; clapping both hands above the head; and clapping both hands above the left shoulder.

[0092] Specifically, the control instructions executed by the action command control include, but are not limited to, control instructions for non-exercise pages (wake-up page, main page, other pages) and control instructions for exercise pages. Control instructions for non-exercise pages mainly involve user interaction with these pages. Control instructions for exercise pages mainly involve user control instructions for playing exercise courses.

[0093] Specifically, control commands for non-exercise pages include, but are not limited to, switching between non-exercise pages, previous page, next page, confirm, return, selecting courses, adding courses to favorites, adding to plans, entering page switching interface, activating voice assistant, and multimedia volume control.

[0094] Specifically, the control commands for the exercise page include, but are not limited to, pausing the course, playing the course, going to the previous section, going to the next section, controlling the volume of the course background music, controlling the volume of the instructor, and evaluating the course.

[0095] Specifically, users can achieve different functions through combinations of multiple actions. This includes, but is not limited to, using a first action to enter the first-level control command menu, and then using a second action to select commands or enter the second-level control command menu within the first-level menu. These control command menus can be displayed to the user on the monitor. Further specific examples include, but are not limited to, the first-level control command menu being a course selection control command menu, including commands to play a course, save a course, display course details, book a course, and enter the second-level control command menu. The second-level control command menu can be a course evaluation menu, including commands for positive, neutral, and negative reviews.

[0096] Specifically, this also includes providing prompts for user actions and commands within the graphical user interface (GUI). This includes, but is not limited to, displaying icons of operable actions in the GUI, marking or highlighting control instructions for static user actions in the GUI, displaying a progress bar for the duration of static user actions in the GUI, and guiding user actions in dynamic user actions (when the completion rate of a dynamic user action exceeds a specified percentage).

[0097] Specifically, this also includes the use of other sensors to assist in the judgment of the graphical user interface (GUI) when the smart fitness mirror can control the user's actions based on the image recognition sensor. This includes, but is not limited to, using a voice recognition sensor to enable / disable / confirm action command control, using a facial recognition sensor to determine whether the user is facing forward or backward (only enabling action command control when the user is facing the smart fitness mirror), using a facial recognition sensor to distinguish different users and load custom action command control instructions, and using an IMU to optimize the accuracy of action / trajectory recognition.

[0098] Specifically, the actions controlled by motion commands can be pre-set in the smart fitness mirror or actions recorded by the user. The control instructions executed by motion commands can be pre-set in the smart fitness mirror or automated instructions set by the user. The correspondence between actions and executed control instructions can be pre-set or user-defined; it can be a one-to-one relationship or a many-to-one relationship.

[0099] Specifically, the activation or deactivation of the motion command control function can be determined by a specific graphical user interface (GUI) or exercise page. For example, the motion command control function may be enabled by default on the main page of the GUI and disabled by default on other pages. The activation or deactivation of the motion command control function may also be determined based on data detected by other sensors. The activation or deactivation of the motion command control function may also be a full activation / deactivation or a partial activation / deactivation.

[0100] A specific example of a smart fitness mirror controlling a graphical user interface (GUI) based on user actions captured by an image recognition sensor: The smart fitness mirror first displays the first page of the GUI on the display 120. The first page shows a prompt icon indicating that raising the left hand enters a page switching mode. When the user raises their left hand, the prompt icon is highlighted and a progress bar starts. After 1.5 seconds, the progress bar completes, and the smart fitness mirror displays a page switching page on the display 120. In addition to the highlighted first page, the switching page also includes a "second page" (the previous page). The page that appears next to the first page, "Page 3," also displays icons for "Previous Page," "Next Page," "Exit," and "Confirm." The "Previous Page" icon represents a horizontal swing of the left hand from left to right, the "Next Page" icon represents a horizontal swing of the right hand from right to left, the "Exit" icon represents raising the left hand, and the "Confirm" icon represents raising the right hand. When the user makes a horizontal swing of the left hand from left to right, the page transitions to highlight Page 2. If the user then raises their right hand, the "Confirm" icon will be highlighted and a progress bar will appear. After 1.5 seconds, the progress bar will complete, and the smart fitness mirror will display Page 2 on the monitor.

[0101] A specific example of a smart fitness mirror controlling a graphical user interface (GUI) based on user actions collected by an image recognition sensor: The smart fitness mirror first displays the fourth page of the GUI on the display 120. The fourth page displays a second prompt icon representing a horizontal swing of the left hand from left to right to enter the previous page and a third prompt icon representing a horizontal swing of the right hand from right to left to enter the next page. When the user makes a horizontal swing of the right hand from right to left, the smart fitness mirror displays a transition from the fourth page to the fifth page on the display 120.

[0102] A specific example of a smart fitness mirror that can control a graphical user interface (GUI) based on user actions collected by an image recognition sensor. The smart fitness mirror first displays, for example, on a display 120. Figure 5-a The third main page includes multiple page objects displayed side-by-side. It also displays the selected page object and an interactive object for entering the selected page object. Below the third main page, there is an interactive object for switching between page objects. Users can switch the displayed page objects using this interactive object and enter the page corresponding to the selected page object by executing the interactive object for entering the selected page object. Users can select a page object by raising their right / left hand diagonally upwards to the left, downwards to the left, upwards to the right, or downwards to the right. After selection, holding the right / left hand in this position and performing a specified static hand gesture, such as spreading the fingers for one second, will enter the page corresponding to the selected page object. Alternatively, users can switch the displayed page objects by making a horizontal waving motion of their right hand from left to right to enter the fourth main page.

[0103] Furthermore, in a specific example of smart fitness mirror hover motion control, the smart fitness mirror first displays, as shown on the monitor 120... Figure 5-aThe third main page includes page objects arranged in a four-grid layout. When the user uses the smart fitness mirror, the selected page object is determined based on the position of the user's right hand. Specifically, when the user's right hand is above the right shoulder, the selected page object is "Weather"; when the right hand is below the right shoulder, the selected page object is "Reminder"; when the right hand is above the left shoulder, the selected page object is "Music"; and when the right hand is below the right shoulder, the selected page object is "Body Fat Scale". When the user's right hand moves, the smart fitness mirror determines the selected page object based on the detected right hand position. After the right hand stops moving and a certain time has elapsed, a preset interaction for the selected page object is executed. This preset interaction could be opening the page object's function or providing a context menu for that page object. The four-grid layout of page objects can also be arranged horizontally, in a six-grid layout, or in a nine-grid layout, etc.

[0104] The actions can be set according to personal preferences (left- or right-handed, disabled, etc.), and have different modes for one-handed and two-handed use, including left / right hand mode and two-handed mode (such as left-handed selection and right-handed screen switching).

[0105] The first page, the second page, the third page, the fourth page, and the fifth page can be the wake-up page, the main page, the exercise page, or other pages, respectively.

[0106] Voice command control

[0107] Smart fitness mirrors can control graphical user interfaces (GUIs) based on user voice collected by voice recognition sensors.

[0108] Specifically, users can use voice commands to control speech recognition results to replace actions controlled by action commands and achieve the same control effect.

[0109] Specifically, the control instructions executed by the action command control include, but are not limited to, control instructions for non-exercise pages (wake-up page, main page, other pages) and control instructions for exercise pages. Control instructions for non-exercise pages mainly involve user interaction with these pages. Control instructions for exercise pages mainly involve user control instructions for playing exercise courses.

[0110] Specifically, control commands for non-exercise pages include, but are not limited to, switching between non-exercise pages, previous page, next page, confirm, return, selecting courses, adding courses to favorites, adding to plans, entering page switching interface, activating voice assistant, and multimedia volume control.

[0111] Specifically, the control commands for the exercise page include, but are not limited to, pausing the course, playing the course, going to the previous section, going to the next section, controlling the volume of the course background music, controlling the volume of the instructor, and evaluating the course.

[0112] Specifically, users can achieve different functions through combinations of multiple actions. This includes, but is not limited to, using a first voice command to enter the first-level control command menu, and then using a second voice command to select commands or enter the second-level control command menu within the first-level menu. These control command menus can be displayed to the user on a monitor. Further specific examples include, but are not limited to, the first-level control command menu being a course selection control command menu, including commands to play a course, save a course, display course details, book a course, and enter the second-level control command menu. The second-level control command menu can be a course evaluation menu, including commands for positive, neutral, and negative reviews.

[0113] Specifically, this also includes providing prompts for user voice commands within the graphical user interface (GUI). This includes, but is not limited to, displaying keywords for available voice commands in the GUI, and completing user voice commands within the GUI (when the user's dynamic action completion rate exceeds a specified percentage).

[0114] Specifically, this also includes smart fitness mirrors relying on other sensors for auxiliary judgment when controlling the graphical user interface (GUI) based on user voice collected by the voice recognition sensor. This includes, but is not limited to, using motion recognition sensors to enable / disable / confirm motion command control, using facial recognition sensors to determine whether the user is facing forward or backward (enabling motion command control only when the user is facing the smart fitness mirror), and using facial recognition sensors to distinguish different users and load custom voice command control instructions.

[0115] Specifically, the keywords for voice command control can be pre-set keywords in the smart fitness mirror or keywords recorded by the user. The control instructions executed by voice command control can be pre-set instructions in the smart fitness mirror or automated instructions set by the user. The correspondence between voice command control keywords and executed control instructions can be pre-set or user-defined; it can be a one-to-one relationship or a many-to-one relationship.

[0116] Specifically, the activation or deactivation of voice command control can be determined by a specific graphical user interface (GUI) or exercise page. For example, voice command control may be enabled by default on the main page of the GUI, while action command control may be disabled by default on other pages. The activation or deactivation of voice command control can also be determined by recognizing a wake word. The activation or deactivation of voice command control can also be determined based on data detected by other sensors. The activation or deactivation of voice command control can also be the activation / deactivation of all functions or the activation / deactivation of some functions.

[0117] A specific example of a smart fitness mirror controlling a graphical user interface (GUI) based on user actions collected by an image recognition sensor: The smart fitness mirror first displays the first page of the GUI on the display 120. The user activates the voice command control function using a wake word. An icon representing the activation of the voice command control function is displayed on the fourth page. When the user says "Enter the switching page" or "Switch pages," the smart fitness mirror displays a page switching page on the display 120. In addition to the highlighted first page, the switching page also includes the previous page "Second Page" and the next page "Third Page." The switching page also displays an icon representing the activation of the voice command control function. When the user says "Previous Page" or "Turn to the previous page," the switching page changes to highlight the second page. When the user says "Enter" or "OK," the smart fitness mirror displays the second page on the display 120.

[0118] A specific example of a smart fitness mirror controlling a graphical user interface (GUI) based on user actions collected by an image recognition sensor: The smart fitness mirror first displays the fourth page of the GUI on the display 120. The user activates the voice command control function by using a wake word. An icon representing the activation of the voice command control function is displayed on the fourth page. When the user says "next page" or "turn to the next page", the smart fitness mirror displays the transition from the fourth page to the fifth page on the display 120.

[0119] A specific example of a smart fitness mirror controlling a graphical user interface (GUI) based on user actions collected by an image recognition sensor: The smart fitness mirror first displays the sixth page of the GUI on the display 120. The user activates the voice command control function through action commands. An icon representing the activation of the voice command control function is displayed on the fourth page. When the user says "Start Training" or "OK", the smart fitness mirror displays a transition from the sixth page to the seventh page on the display 120.

[0120] The first, second, third, fourth, and fifth pages can be wake-up pages, main pages, exercise pages, or other pages, respectively. The sixth page is used to display exercise cards, and the seventh page is the exercise page.

[0121] Touch command control

[0122] The smart fitness mirror can control the graphical user interface (GUI) based on the user's touch commands collected by the touchscreen.

[0123] Multimodal command control

[0124] Multimodal command control includes combinational control, enhancement control, and conflict control.

[0125] The combined control includes, but is not limited to, action combination control of multiple actions and voice combination control of multiple keywords. More complex control commands (such as selecting and favorite, selecting and recommending to friends, taking a screenshot and sharing, etc.) can be achieved through multiple actions or keywords in succession (with an interval of no more than a threshold).

[0126] The combined control includes, but is not limited to, enhancing and optimizing user action command control or voice command control through facial recognition or scene recognition. Further, it includes, but is not limited to, when the user's action command or voice command control is to play music, recognizing that the user's facial expression is unhappy, presenting the user with soothing recommendations and showing concern for the user; when the user's action command or voice command control is to recommend courses, recognizing that the scene temperature is low, reducing the recommendation of meditation courses.

[0127] Conflict control includes, but is not limited to, when multiple conflicting control commands occur when a user uses multiple different control methods to control the smart fitness mirror simultaneously, determining the control command that the user wants to execute by prioritizing the different control methods and having the smart fitness mirror execute it.

[0128] Control commands

[0129] Control commands include, but are not limited to, system wake-up commands, page switching commands, page interaction commands, and course control commands. Users can use these control commands to control the smart fitness mirror.

[0130] The system wake-up commands include commands to wake the smart fitness mirror from standby mode to the wake-up page, main page, exercise page, and other pages, and commands to switch the smart fitness mirror from the wake-up page to the main page, exercise page, and other pages.

[0131] The page switching commands refer to commands used to control the smart fitness mirror to switch between multiple different pages. These different pages can be pages of different types or pages with different content. Further page switching commands include, but are not limited to, page turning commands (next page / previous page) and commands to control the smart fitness mirror to enter a page switching page.

[0132] Page interaction commands refer to instructions used to control the smart fitness mirror to perform interactions or inputs. These page interaction commands include, but are not limited to, control interaction commands and input interaction commands. Further, control interaction commands refer to commands that control page objects or interactive objects within a graphical user interface (GUI) to perform their interactive functions, such as confirming / returning, adding to favorites, adding to a plan, activating the voice assistant, volume control, switching accounts, switching modes, and opening a context menu. Further, input interaction commands refer to commands that input information into page objects or interactive objects within a graphical user interface (GUI), such as adding a heart with each input command when evaluating a course; or switching to a preset emoticon / reply with each input command when chatting with other users through the smart fitness mirror.

[0133] Course control commands refer to the commands used to control the smart fitness mirror when playing exercise videos on the exercise page, primarily for controlling the playback of exercise videos. Further course control commands include, but are not limited to, pause / play, previous / next lesson, background volume control, coach volume control, and course evaluation.

[0134] Data Management

[0135] The data sources of the smart fitness mirror include, but are not limited to, sensor data and backend data, which are used for the input of user control commands and the display of page information.

[0136] The sensor data includes, but is not limited to, data from on-mirror sensors and data from external sensors. The on-mirror sensor data includes, but is not limited to, data detected by sensors mounted on the smart fitness mirror, such as image recognition sensors, infrared recognition sensors, and voice recognition sensors. More specifically, the data detected by image recognition sensors includes, but is not limited to, images detected by the image recognition sensor and data obtained after image processing and recognition, such as motion data, user posture data, facial recognition data, user expression data, and environmental image data. More specifically, the data detected by infrared recognition sensors includes, but is not limited to, thermal images detected by the infrared recognition sensor and data obtained after processing and recognition of the thermal images, such as user thermal imaging data, user exercise area data, and environmental thermal imaging data. More specifically, the data detected by voice recognition sensors includes, but is not limited to, voice data detected by the voice recognition sensor and data obtained after processing and recognition of the voice data, such as voice control commands, keyword commands, and wake-up word commands. The on-mirror sensor data includes, but is not limited to, data detected by inertial sensor data and physiological data sensor data (such as heart rate monitor data and body fat scale data), and data detected by sensors wirelessly connected to the smart fitness mirror. More specifically, the data detected by inertial sensors includes, but is not limited to, velocity / acceleration data detected by inertial sensors and data obtained after processing and identifying the velocity / acceleration data, such as motion data. More specifically, the data detected by physiological data sensors includes, but is not limited to, physiological data detected by physiological data sensors and data obtained after processing and identifying the physiological data, such as heart rate data, blood pressure data, and blood oxygen saturation data.

[0137] The backend data includes, but is not limited to, backend user data and other backend data. The backend user data refers to data directly related to the user stored on the smart fitness mirror and / or a server connected to the smart fitness mirror. This backend user data includes, but is not limited to, user physiological data, user social data, user calendar data, and usage history data. The usage history data includes, but is not limited to, data on completed courses, saved courses, best historical scores, physiological and performance data during courses, and frequency of classes. The user physiological data includes, but is not limited to, the user's height, weight, gender, age, heart rate, blood pressure, and medical history. The user social data includes, but is not limited to, data related to people / friends followed by the user and data related to groups / interest groups joined by the user. Specifically, this information may include invitations to complete designated courses, invitations to one-on-one or multi-person challenges, user rankings, and dynamic data uploaded by the user or other users. These dynamics can be synchronized to other social platforms (such as WeChat, Weibo, Facebook, Twitter, etc.). The user calendar data includes, but is not limited to, user schedule information, to-do list data, and course plan data. The other background data refers to data with low user relevance stored on the smart fitness mirror and / or the server connected to the smart fitness mirror. This other background data includes, but is not limited to, IP data, environmental data, course data, and third-party data. IP data includes, but is not limited to, the smart fitness mirror's IP address and MAC address data. Environmental data includes, but is not limited to, the time, location, weather, temperature, and clothing index data of the area where the smart fitness mirror's IP is located. Course data includes, but is not limited to, workout video data, instructor name, course type, course duration, course difficulty, course tags, and / or estimated calorie expenditure. Course types include, but are not limited to, HIIT, aerobics, yoga, strength training, combat training, Barre, Pilates, stretching, dance, meditation, and challenges displayed to users via the smart fitness mirror. Course tags include, but are not limited to, whether the course is AI-recognized, whether it is a parent-child course, the main body parts involved (e.g., full body, arms, waist, legs), and the equipment used in the course (e.g., no equipment, yoga mat, resistance band, dumbbells). The third-party data includes, but is not limited to, data from third-party functions installed on the smart fitness mirror.

[0138] Data storage

[0139] Smart fitness mirrors may also store user information locally on the smart fitness mirror and / or a remote storage device (e.g., a cloud service), depending on the amount of storage space available. For example, user information requiring minimal storage space can be stored locally on the smart fitness mirror; this information includes, but is not limited to, the user's name, age, height, weight, and gender. Additionally, class data can be stored in the smart fitness mirror to reduce the impact of network latency that could affect video streaming quality. The amount of video content stored may be limited by the storage capacity of the smart fitness mirror. In some configurations, video content may be temporarily stored only daily or weekly, or depending on a percentage of the smart fitness mirror's capacity. Background user data requiring significant storage space can be stored on a remote storage device; this user information includes, but is not limited to, physiological data such as the user's heart rate and calories burned, as well as video footage or skeletal data of the user captured during exercise. The smart fitness mirror can retrieve this information for subsequent analysis and display.

[0140] Data transmission between smart fitness mirrors and remote storage devices can be protected (e.g., encrypted) in various ways to prevent unwanted loss or theft of user information. For example, the Bluetooth Low Energy (BLE) protocol includes built-in security features that can be used by devices utilizing the protocol. However, these security features are only available when the Bluetooth pairing process is completed before establishing an encrypted connection. In some cases, where various security mechanisms may not be implemented or may malfunction, application-level security can be achieved by combining the aforementioned data chunking specifications. For example, Advanced Encryption Standard (AES) encryption can be applied to the message before the preamble. In some aspects, the BLE protocol performs a similar process via firmware-level built-in security features and can provide similar protection against unauthorized reading of communications between the client and server.

[0141] When the client disconnects from the server, the GATT service added for the client to read / notify messages, which was used in the service recording on the server device, can be removed. This ensures that no connection is open and that the system does not accidentally leak information to malicious snoops. This connection termination can be triggered by either the server or the client and relies on the Bluetooth Low Energy stack to notify both parties that the connection has been closed. If Bluetooth tethering is used in the initial connection setup to provide firmware-level cryptographic security, the tethering information can be stored on each device, eliminating the need for repeated tethering after subsequent connections between the client and server.

[0142] Function Management

[0143] The smart fitness mirror can also perform the following functions based on the received data: user recognition, mode switching, attention judgment, body / emotion assessment, and clothing recommendations.

[0144] The user identification function allows the smart fitness mirror to identify the user using it based on input data. This function is primarily achieved through the following method: the user inputs their characteristic data through a data input device; the smart fitness mirror compares this data with characteristic data in its database; based on the comparison result, it determines the user currently using the mirror; and based on the identification result, the smart fitness mirror can adjust its graphical user interface (GUI), such as adjusting font size and page information preferences. The data input device includes, but is not limited to, on-mirror sensor data and external sensors; the user's characteristic data includes, but is not limited to, data collected by on-mirror sensors and external sensors, such as skeletal points, voiceprints, weight, and resting heart rate, or data collected and processed by external sensors; the database records the mapping relationship between users and characteristic data. Furthermore, the user identification function can be used to identify and switch between multiple sub-accounts under a main account, or between multiple main accounts. In another example, the user identification function allows the smart fitness mirror to identify the characteristics of the user using it, such as male, female, elderly, or child, based on the input data; and based on the identification result, the smart fitness mirror can adjust its graphical user interface (GUI), such as adjusting font size and page information.

[0145] The mode switching function allows the smart fitness mirror to switch between multiple working modes to adapt to different work scenarios. For example, the smart fitness mirror can switch to a youth mode. In youth mode, the youth account is linked to the parent's account or the main device account. In youth mode, the page information displayed by the smart fitness mirror will be adjusted accordingly. For example, course recommendations will be adjusted to prioritize content suitable for teenagers, such as jump rope and motion-sensing games. At the same time, real-time status and exercise data in this mode can be synchronized to the smart terminal of the parent's account or the main device account. In another example, the smart fitness mirror can switch to a guest mode. In guest mode, the smart fitness mirror can work without being linked to an account. The page information displayed by the smart fitness mirror can also be adjusted accordingly. For example, course recommendations will be adjusted to prioritize content suitable for beginners, such as content with low difficulty tags. At the same time, even if the smart fitness mirror is linked to an account, the exercise data in this mode will not be recorded in the linked account and will not affect the training plan or course completion status of the linked account.

[0146] Attention assessment allows smart fitness mirrors to determine whether the user's attention is on the mirror while it is in use. This enables the mirror to determine the user's intention to interact with the mirror and to activate or deactivate sensors, other data input devices, and functions. Attention assessment primarily determines whether the user intends to use the smart fitness mirror based on their behavior. This can be done, but is not limited to, using the following characteristics: distance assessment (the distance between the user and the mirror does not exceed a threshold), posture assessment (whether the user is facing the mirror directly), motion assessment (whether the user has performed a specified motion or a commonly used exercise motion), and semantic assessment (whether the user's voice recognition results are related to specific keywords).

[0147] The body / emotion assessment allows the smart fitness mirror to determine the user's physical / emotional state while using the device, enabling the mirror to adjust page information or other feedback accordingly. The user's physical / emotional state can be identified through physiological data sensors and image recognition sensors (facial recognition). When the mirror detects that the user is unhappy, it presents soothing page recommendations and shows concern for the user during voice interaction. In another example, when a user activates the smart fitness mirror's voice function using a wake word, the mirror, through its facial recognition sensor, detects a depressed expression and initiates the first round of dialogue: "Hey, what's wrong? You don't look too happy. Want to listen to a song to relax?" The identification of the physical / emotional state can be achieved by recognizing the user's age, posture, and facial expressions to determine their emotional state, stress, fatigue, etc.; it can also be done through blood pressure and heart rate data detected by physiological data sensors.

[0148] The clothing recommendation feature allows smart fitness mirrors to initiate proactive interactions with users based on specific conditions. The smart fitness mirror uses collected time and location data, combined with user information and the user's actions in front of the mirror, to determine whether proactive interaction is needed. For example, if a user first appears in front of the smart fitness mirror in the morning and turns around, the mirror can activate the clothing recommendation function, suggesting suitable clothing for the day based on the current location. Furthermore, if a user appears in front of the mirror on the weekend and turns around, the mirror can activate the clothing recommendation function, providing outfit suggestions for the current day. These functions can also be used for other proactive interactions between the smart fitness mirror and users based on user and external information.

[0149] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multimodal interactive fitness mirror.

[0150] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0151] The memory can be used to store the computer program and / or modules. The processor implements various functions of the multimodal interactive fitness mirror device by running or executing the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart memory card, secure digital card, flash memory card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0152] Video streaming methods

[0153] The smart fitness mirror is configured to display video content from a video production facility on display 120. The video content can be streamed as live content or pre-recorded content. Alternatively, live content can be recorded and stored on a cloud server, allowing users to request and play the video content later, thus becoming pre-recorded content. During video streaming, the smart fitness mirror first sends a request for a specified course to the API server. The API server returns two parts: an overview of the specified course (including course images and a course description); and the address (URL) of the course video. The smart fitness mirror then requests the corresponding course video from the OSS server based on the received URL. When the course is pre-recorded content, the smart fitness mirror uses the HLS protocol for video playback; that is, the address (URL) of the course video is the corresponding M3U8 file. The smart fitness mirror's player then downloads the corresponding course video segment from the OSS server based on the information in the M3U8 file. When the course content is live, the smart fitness mirror uses the RTMP protocol to play the video. That is, the address (URL) of the course video is the corresponding RTMP live address of the course. Then, the smart fitness mirror's player downloads the corresponding course video live data from the OSS server according to the RTMP live address.

[0154] In some applications, smart fitness mirrors can connect to online streaming services that provide users with third-party video content streamed from servers (e.g., directly via a network router or indirectly via the user's smart device). Third-party content can be offered to users on a subscription basis. Third parties can also supply content to a centralized distribution platform that communicates with the smart fitness mirror over a network. One advantage of a centralized distribution platform is the simplified distribution of content to the smart fitness mirror. Alternatively, third parties can develop a separate distribution platform that allows users to access content via a separate software application on the smart device.

[0155] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the multimodal interactive fitness mirror.

[0156] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0157] An action recognition method,

[0158] like Figure 10 As shown, based on the above embodiments, the interaction process of this device specifically includes the following steps:

[0159] The S121 fitness mirror captures images of the user within a designated area to obtain images of the user's movements.

[0160] S122 generates the user's skeletal point information based on the motion image and establishes a coordinate system based on the skeletal point information;

[0161] S122.1 Skeletal point information includes top head skeletal points, pelvic skeletal points, left hand skeletal points, and right hand skeletal points;

[0162] S122.2 Obtain the straight-line distance between the top of the head bone point and the pelvic bone point, and obtain the unit length of the coordinate system based on the straight-line distance;

[0163] S122.21 Obtain the straight-line distance between the top of the head bone and the pelvic bone, and use one-sixth of the straight-line distance as the unit length of the coordinate system, that is, the size of the unit length 1 of the coordinate system is one-sixth of the straight-line distance;

[0164] S122.3 The straight line between the top of the head bone and the pelvic bone is used as the vertical coordinate, the pelvic bone is used as the origin, and a rectangular coordinate system is constructed using the obtained unit length.

[0165] S123 establishes several interactive regions based on the coordinate system;

[0166] In this embodiment, there are five interactive areas, namely the first interactive area, the second interactive area, the third interactive area, the fourth interactive area, and the fifth interactive area, as follows: Figure 4 As shown, the first interactive area is Figure 3 In region A, the second interactive region is Figure 4 Region B, the third interactive region is Figure 4 The C region, the fourth interactive region is Figure 4 The fifth interactive region is region D. Figure 4 Region E.

[0167] The first interaction area is centered on the top of the head bone, extending 5 units vertically and 6 units horizontally (12 units long and 10 units high). The top edge is 5 units vertically upwards and the bottom edge is 5 units vertically downwards; the left edge is 6 units horizontally to the left and the right edge is 6 units horizontally to the right. The preset control command for the first interaction area is "Confirm A," meaning that when the interaction posture is within the first interaction area, the corresponding control command, "Confirm A," is output.

[0168] The control command for the second interactive area is to move left, and the control command for the third interactive area is to move right. The second interactive area is a region with a length of 13 units and a height of 12 units, starting from the pelvic bone point, extending vertically upwards by 2 units to the top and downwards by 10 units to the bottom, starting from the top of the head bone point, extending horizontally to the left by 3 units to the right and 16 units to the left.

[0169] The third interactive area is a region with a length of 13 units and a height of 12 units, starting from the pelvic bone point, extending vertically upwards by 2 units to the top and downwards by 10 units to the bottom, starting from the top of the head bone point, extending horizontally to the right by 3 units to the left and 16 units to the right to the right.

[0170] The control command for the fourth interaction area is "Confirm" (D), and the control command for the fifth interaction area is "Confirm" (E). The fourth interaction area is an area with a length of 11 units and a height of 9 units, starting from the top of the head bone point, extending vertically upwards by 4 units to the top and downwards by 5 units to the bottom, extending horizontally to the left by 7 units to the right and 16 units to the left.

[0171] The fifth interactive area is an area with a length of 11 units and a height of 9 units, starting from the top of the skull, extending vertically upwards by 4 units to the top and downwards by 5 units to the bottom, extending horizontally to the right by 7 units to the left and 16 units to the right.

[0172] S124 identifies interactive gestures based on skeletal point information;

[0173] S124.1 Obtain the distance between the left-hand and right-hand bone points;

[0174] S124.2 The distance between the left and right hand bone points is compared with a threshold T. If the distance between the left and right hand bone points is less than or equal to the threshold T, an interactive gesture is identified. In this embodiment, the threshold T is 15cm. If the distance between the left and right hand bone points is less than or equal to the threshold T, it is determined that the user has made an interactive gesture, and the interactive gesture is identified. For example, in this embodiment, the interactive gesture is a high-five, that is, the user makes a high-five action. If the distance between the left and right hand bone points is greater than the threshold T, the user has not made a high-five action. In this embodiment, the specific interactive gestures are only illustrative and are not limited.

[0175] S124.3 If the user makes an interactive gesture, obtain the coordinates of the midpoint between the left-hand bone point and the right-hand bone point;

[0176] S125 determines whether the coordinates of the intermediate point are located within the interactive area;

[0177] S125.1 Determine whether the coordinates of the intermediate point are located within the interactive area;

[0178] S125.11 If the coordinates of the intermediate point are located in one of the interactive areas, then the interactive area is active and the control command corresponding to the interactive area is output; other interactive areas are inactive.

[0179] S125.12 outputs the control command corresponding to the interactive area while the interactive area is in an active state, and the other interactive areas are in an inactive state; the area of ​​the active interactive area increases, and the area of ​​the inactive interactive area decreases.

[0180] In this embodiment, the length of the active interactive area increases by 1.5 units, and the height increases by 0.5 units. Specifically, the length of the active interactive area increases outward by 0.75 units to the left and right, and the height increases outward by 0.25 units to the top and bottom. The length of the inactive interactive area decreases by 2 units, and the height decreases by 1 unit. Specifically, the length of the inactive interactive area decreases inward by 1 unit to the left and right, and the height decreases inward by 0.5 units to the top and bottom.

[0181] When no control command is output for the interactive area, the area of ​​the interactive area remains unchanged. After the control command is output, the area of ​​the interactive area is adjusted. After the control command is executed, the area of ​​the interactive area returns to its original state.

[0182] When the user makes an interactive gesture in the first interactive area, the fitness mirror recognizes the gesture and outputs the corresponding control command for the first interactive area. The area of ​​the first interactive area is expanded, that is, the top vertically increases by 0.25 units, the bottom vertically increases by 0.25 units, the left horizontally increases by 0.75 units, and the right horizontally increases by 0.75 units.

[0183] All parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications using the teachings of this invention. It should be understood that the foregoing embodiments are presented primarily by way of example, and that embodiments of the invention may be practiced in ways other than those specifically described and claimed within the scope of the appended claims and their equivalents. The inventive embodiments of this disclosure relate to each individual feature, system, article, material, kit, and / or method described herein.

[0184] Furthermore, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of this disclosure if they do not contradict each other. Other substitutions, modifications, alterations, and omissions may be made in the design, operating conditions, and arrangement of the corresponding elements in the exemplary embodiments without departing from the spirit of this disclosure. The use of numerical ranges does not exclude equivalents that perform the same function in the same manner to produce the same results, falling outside the range.

[0185] The above implementation scheme can be implemented in several ways. For example, it can be implemented using hardware, software, or a combination thereof. When implemented in software, the software code can be executed on a suitable processor or set of processors, whether it is provided in a single computer or distributed among multiple computers.

[0186] Furthermore, a computer can take any of a variety of forms, such as a rack-mount computer, desktop computer, laptop computer, or tablet computer. Additionally, a computer can be embedded in a device that is not typically considered a computer but has suitable processing capabilities, including a personal digital assistant (PDA), smartphone, or any other suitable portable or stationary electronic device.

[0187] In addition, a computer may have one or more input and output devices. Among other things, these devices may also be used to present a user interface. Examples of output devices that can be used to provide a user interface include a printer or display screen for visual presentation and a speaker or other sound-emitting device for audible presentation. Examples of input devices that can be used for a user interface include keyboards and pointing devices such as mice, touchpads, and digital tablets. As another example, a computer may receive input information via voice recognition or other audible formats.

[0188] Such computers can be interconnected in a suitable manner through one or more networks, including local area networks (LANs) or wide area networks (WANs), such as enterprise networks, intelligent networks (INs), or the Internet. These networks can be based on suitable technologies, operate according to suitable protocols, and can include wireless networks, wired networks, or fiber optic networks.

[0189] The various methods or processes outlined herein can be encoded as software that can be executed on one or more processors employing any of a variety of operating systems or platforms. Furthermore, such software can be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and can also be compiled into executable machine language code or intermediate code that executes on a framework or virtual machine. Some implementations may specifically employ one or more of a particular operating system or platform and a particular programming language and / or scripting tool to facilitate execution.

[0190] Furthermore, various inventive concepts can be embodied in one or more methods, of which at least one example has been provided. Actions performed as part of a method may, in some cases, be ordered differently. Thus, in some inventive embodiments, corresponding actions of a given method may be performed in an order different from that specifically shown, which may include performing several actions simultaneously (even if such actions are shown as sequential actions in exemplary embodiments).

[0191] All publications, patent applications, patents, and other references mentioned in this article are incorporated herein by reference in their entirety.

[0192] All definitions used herein should be understood as control dictionary definitions, definitions in referenced and incorporated literature, and / or the general meaning of the defined terms.

[0193] Unless explicitly contradictory, the indefinite articles “a” and “an” used in this specification and claims shall be understood to mean “at least one”.

[0194] The phrase “and / or” as used in this specification and claims should be understood to mean “one or two” elements so combined, that is, elements that exist together in some cases and separately in others. Multiple elements listed with “and / or” should be interpreted in the same way, that is, “one or more” elements so combined. Other elements may optionally be present in addition to those specifically indicated by the “and / or” clause, whether related to or unrelated to those specifically indicated. Thus, as a non-limiting example, when used in conjunction with open-ended language such as “comprising,” a reference to “A and / or B” may refer only to A (optionally including elements other than B) in one embodiment; only to B (optionally including elements other than A) in another embodiment; both A and B (optionally including other elements) in yet another embodiment; and so on.

[0195] As used in this specification and claims, “or” should be understood to have the same meaning as “and / or” as defined above. For example, when separating items in a list, “or” or “and / or” should be interpreted as inclusive, that is, including at least one, but also including more than one element in a list of many elements and (optionally) additional unlisted items. Unless explicitly contradictory, phrases such as “only one” or “exactly one” or, when used in claims, “consisting of” will refer to exactly one element in a list of many elements. Generally, when the term “or” as used herein is followed by an exclusive term such as “one of both,” “one of,” “only one of,” or “exact one of,” it should only be interpreted as indicating an exclusive alternative (i.e., “one or the other but not both”). “Substantially consisting of” when used in claims has the same general meaning as it is used in the field of patent law.

[0196] As used in this specification and claims, the phrase "at least one" in relation to a list of one or more elements should be understood to mean at least one element selected from any one or more elements in the list of elements, but does not necessarily include at least one element from each element specifically listed in the list of elements, and does not exclude any combination of elements in the list of elements. This definition also allows for the optional presence of elements other than those specifically represented in the list of elements referred to by the phrase "at least one," whether related to or unrelated to those specifically represented elements. Thus, as a non-limiting example, "at least one of A and B" (or equivalently, "at least one of A or B," or equivalently "at least one of A and / or B") in one embodiment may refer to at least one A, optionally including more than one A, without B (and optionally including elements other than B); in another embodiment, it may refer to at least one B, optionally including more than one B, without A (and optionally including elements other than A); in yet another embodiment, it may refer to at least one A, optionally including more than one A and at least one B, optionally including more than one B (and optionally including other elements); and so on.

[0197] In the claims and the description above, all transitional phrases such as “comprising,” “including,” “carrying,” “having,” “containing,” “involving,” “holding,” “consisting of,” etc., should be understood as open-ended, that is, including but not limited to.

[0198] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0199] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An interactive fitness mirror device, characterized in that, include A communication interface used to receive video images; A display for showing a user interface or exercise interface, the user interface having at least one interactive object providing control actions for the user to perform; the exercise interface including video images received via a communication interface. A mirror, wherein the mirror is used to display the user's reflection in the mirror, and the display shows the user interface to the user through the mirror; Sensors are used to collect user motion data; A processor, coupled to the display and the sensor, is configured to determine an operation based on the user's motion data, based on at least one of the actions displayed in the video image of the exercise interface and the control actions provided by the interactive object for the user to perform. The control actions and control commands preset in the mirror device are matched, and the control commands are used to control the user interface to complete a specified operation. When the processor determines, based on the motion data, that the action indicated by the motion data matches the control action, the processor executes the control command corresponding to the control action. When the processor determines, based on the motion data, that the action indicated by the motion data matches the action displayed in the video image of the exercise interface, the processor does not execute the operation of the control command. The processor determines the operation matched by the action indicated by the motion data based on the control action and the action displayed in the video image. The processor compares user motion data collected by the sensor within time period t with the motion displayed in the video footage of the exercise interface within time period t. Standard motions include standard static motions and standard dynamic motions. Standard static motions involve the user performing a predetermined posture with a predetermined body part and maintaining it for a specified time. Standard dynamic motions involve the user performing a predetermined action with a predetermined body part. The user interface includes a wake-up page, a main page, or other pages. Control instructions for the user interface include user interaction instructions, which include at least one of the following: previous page, next page, confirm, return, select course, add to favorites, add to plan, enter page switching interface, wake up voice assistant, and multimedia volume control. Control instructions for the exercise interface include user control instructions for playing exercise courses, including at least one of the following: pause course, play course, previous segment, next segment, course background music volume control, course instructor volume control, and course evaluation. The user interface and exercise interface also include page animation effects, which include page transition animation effects and page interaction animation effects. The page transition animation effects are used to generate dynamic effects when the page is operated or changed, and the page interaction animation effects are used to provide guidance to the user when interacting.

2. The interactive fitness mirror device according to claim 1, characterized in that, The user interface also includes pre-installed feature pages and third-party integrated application pages.

Images