Display device and fitness follow-up exercise error correction method

Abstract

The embodiment of the invention provides a display device and a fitness following exercise error correction method. And after the user indicates fitness and follow-up exercise, the display equipment displays a fitness and follow-up exercise video, and obtains a user image through the image collector. The display device obtains action group data, including first limb key point data and a target limb region of multiple frames of action images. And the display device compares the current frame of user image with the multiple frames of action images to obtain a target action image matched with the current frame of user image. And the display device detects second limb key point data of the current frame of user image, obtains error correction information of the target limb region according to the first limb key point data and the second limb key point data of the target action image, and displays the error correction information. Therefore, the display device can perform error correction on non-standard actions of the user and display the follow-up practice condition of the actions.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a fitness training error correction method. Background Art

[0002] A display device refers to a terminal device that can output specific display images. A display device can realize two-way human-computer interaction functions, integrating multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] Users can use display devices to assist in exercise, such as doing fitness exercises, to achieve the goal of strengthening their bodies. The display device can be installed with a fitness application that provides different training programs, such as yoga, aerobics, etc. After the user selects any training program, the display device can play a fitness follow-up video for that program. Users can imitate the movements in the fitness follow-up video to follow the fitness follow-up training. While the user is following the training, the display device can also use the image collector to capture the user's fitness images, and compare and score them with the fitness follow-up video played by the display device, thereby evaluating the accuracy of the user's fitness follow-up training.

[0004] However, when grading, the display device will give an overall score for multiple exercises in a fitness training video. Users can use this overall score to confirm their overall training of the entire set of exercises. However, they cannot know how well they have practiced each exercise individually, making it impossible to determine whether they have practiced each exercise correctly or to re-practice exercises that have not been performed well. Summary of the Invention

[0005] The present application provides a display device and a fitness follow-up error correction method to solve the problem in the related art that the display device will give an overall score of multiple actions in a fitness follow-up video, and the user cannot know his or her own individual follow-up status of each action.

[0006] In a first aspect, some embodiments of the present application provide a display device, comprising a display, a device interface, and a controller. The device interface is configured to connect to an image collector; the controller is configured to:

[0007] In response to a fitness training instruction, controlling the display to display a fitness training video and acquiring a user image through an image collector; the fitness training video includes a plurality of action groups, and the action groups include a plurality of frames of action images;

[0008] Acquire action group data of the current action group; the action group data includes first limb key point data and a target limb area of ​​multiple frames of action images, wherein the target limb area includes multiple limb key points;

[0009] Comparing the current frame user image with multiple frames of action images to obtain a target action image that matches the current frame user image;

[0010] Detecting the second limb key point data of the current frame user image; generating error correction information of the target limb area based on the first limb key point data and the second limb key point data of the target action image;

[0011] The display is controlled to display the error correction information.

[0012] In a second aspect, some embodiments of the present application provide a fitness training error correction method, applied to a display device, comprising:

[0013] In response to a fitness training instruction, controlling the display to display a fitness training video and acquiring a user image through an image collector; the fitness training video includes a plurality of action groups, and the action groups include a plurality of frames of action images;

[0014] Acquire action group data of the current action group; the action group data includes first limb key point data and a target limb area of ​​multiple frames of action images, wherein the target limb area includes multiple limb key points;

[0015] Comparing the current frame user image with multiple frames of action images to obtain a target action image that matches the current frame user image;

[0016] Detecting the second limb key point data of the current frame user image; generating error correction information of the target limb area based on the first limb key point data and the second limb key point data of the target action image;

[0017] The display is controlled to display the error correction information.

[0018] It can be seen from the above technical solutions that some embodiments of the present application provide a display device and a fitness follow-up error correction method. After the user instructs to follow the fitness training, the display device displays the fitness follow-up video and obtains the user image through the image collector. The display device obtains the action group data of the current action group, including the first limb key point data and the target limb area of ​​the multi-frame action image. The display device compares the current frame user image and the multi-frame action image to obtain the target action image that matches the current frame user image. The display device detects the second limb key point data of the current frame user image, obtains the error correction information of the target limb area according to the first limb key point data and the second limb key point data of the target action image, and displays it. Therefore, the display device can correct the user's non-standard actions and display the follow-up situation of the action. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 shows a usage scenario of a display device according to some embodiments;

[0021] Figure 2 shows a hardware configuration block diagram of the control device 100 according to some embodiments;

[0022] Figure 3 shows a hardware configuration block diagram of a display device 200 according to some embodiments;

[0023] Figure 4 shows a software configuration diagram in the display device 200 according to some embodiments;

[0024] Figure 5 shows a schematic diagram of an application panel in some embodiments;

[0025] Figure 6 A schematic diagram of a fitness application homepage in some embodiments is shown;

[0026] Figure 7 A schematic diagram of a training item list page in some embodiments is shown;

[0027] Figure 8 A schematic diagram of a training mode selection page in some embodiments is shown;

[0028] Figure 9 A schematic diagram showing the image collector collecting user images in some embodiments is shown;

[0029] Figure 10 A schematic diagram showing confirmation information of a fitness follow-up error correction mode in some embodiments;

[0030] Figure 11 A flowchart illustrating the interaction between components of a display device in some embodiments is shown;

[0031] Figure 12 A schematic diagram of a portrait recognition interface in some embodiments is shown;

[0032] Figure 13 Schematic diagrams of fitness training interfaces in some embodiments are shown;

[0033] Figure 14 A schematic diagram showing key points of a user's limbs in some embodiments;

[0034] Figure 15shows a schematic diagram of displaying error correction information in some embodiments;

[0035] Figure 16 Schematic diagrams of the practice and error correction interface in some embodiments are shown;

[0036] Figure 17 Schematic diagrams showing error correction markers in some embodiments;

[0037] Figure 18 A schematic diagram of a fitness training details interface in some embodiments is shown. DETAILED DESCRIPTION

[0038] In order to make the purpose and implementation of this application clearer, the exemplary implementation of this application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only part of the embodiments of this application, not all of the embodiments.

[0039] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0040] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0041] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0042] The display device provided in the embodiments of the present application may have various implementation forms, for example, it may be a television, a smart TV, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figure 1 and Figure 2 This is a specific implementation of the display device of the present application.

[0043] Figure 1 Schematic diagram of an operation scenario between a display device and a control device according to an embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control apparatus 100 .

[0044] In some embodiments, the control device 100 may be a remote controller. Communication between the remote controller and the display device may include infrared protocol communication, Bluetooth protocol communication, or other short-range communication methods, and the display device 200 may be controlled wirelessly or wired. The user may control the display device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, and the like.

[0045] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) can also be used to control the display device 200. For example, the display device 200 can be controlled using an application running on the smart device.

[0046] In some embodiments, the display device may not use the aforementioned smart device or control device to receive instructions, but may receive user control through touch or gestures.

[0047] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0048] In some embodiments, the display device 200 also communicates data with the server 400. Configurations may be provided to allow the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be a single cluster or multiple clusters, and may include one or more types of servers.

[0049] Figure 2 Schematically shows a block diagram of the configuration of the control device 100 according to an exemplary embodiment. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, acting as an interaction medium between the user and the display device 200.

[0050] like Figure 3 The display device 200 includes at least one of a tuner and demodulator 210, a communicator 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0051] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and first to nth interfaces for input / output.

[0052] The display 260 includes a display screen component for presenting images, and a driving component for driving image display, a component for receiving image signals output from a controller, and a component for displaying video content, image content, and a menu control interface and a user control UI interface.

[0053] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0054] Communicator 220 is a component used to communicate with external devices or servers using various communication protocols. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, or other network communication protocol chip or a near-field communication protocol chip, as well as an infrared receiver. Display device 200 can use communicator 220 to send and receive control signals and data signals with external control device 100 or server 400.

[0055] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote controller, etc.).

[0056] Detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or detector 230 includes an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or detector 230 includes a sound collector, such as a microphone, for receiving external sounds.

[0057] The device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by multiple of the above interfaces.

[0058] The tuner-demodulator 210 receives broadcast television signals via a wired or wireless reception method, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0059] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0060] Controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. Controller 250 controls the overall operation of display device 200. For example, in response to receiving a user command to select a UI object for display on display 260, controller 250 may perform operations related to the object selected by the user command.

[0061] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (Random Access Memory, RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0062] The user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific voice or gesture, and the user input interface may recognize the voice or gesture through a sensor to receive the user input command.

[0063] A user interface is the medium through which an application or operating system interacts with a user and exchanges information. It converts information between its internal form and a user-friendly format. A common user interface is the graphical user interface (GUI), which refers to a graphical user interface related to computer operations. It can be an icon, window, control, or other interface element displayed on an electronic device's display. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0064] like Figure 4 As shown, the system of the display device is divided into three layers, namely the application layer, the middleware layer and the hardware layer from top to bottom.

[0065] The application layer mainly includes commonly used applications on the TV and the application framework. Common applications are mainly browser-based applications, such as HTML5 apps, and native apps.

[0066] The Application Framework is a complete programming model that provides all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces for these functions (toolbars, status bars, menus, and dialog boxes).

[0067] Native apps can support online or offline, message push or local resource access.

[0068] The middleware layer includes various TV protocols, multimedia protocols, and system components. Middleware uses the basic services (functions) provided by system software to connect various parts of the application system or different applications on the network, enabling resource and function sharing.

[0069] The hardware layer mainly includes the HAL interface, hardware, and drivers. The HAL interface is a unified interface for all TV chips, and the specific logic is implemented by each chip. Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WiFi driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0070] The display device 200 can have multiple functions, such as browsing the web, playing media, playing games, and screen projection, thereby providing users with a variety of services. The user can control the display device 200 to start a related application to enable the corresponding function.

[0071] In some embodiments, when the user controls the display device to power on, the controller 250 can control the display 260 to display a user interface. The user interface can include specific target images, such as various media assets obtained from a network signal source, including videos, images, and other content. The user interface can also be some UI interface of the display device, such as the system home screen, various settings screens, and various control screens of application programs.

[0072] The user interface may include an application panel, such as a "My Applications" control. Through the application panel, users can view the applications installed on the display device 200, that is, the functions supported by the display device 200. It should be noted that the applications installed on the display device 200 can be system applications or third-party applications. By launching an application, the user controls the display device 200 to implement the corresponding functions of the application.

[0073] In some embodiments, the display device 200 is further provided with a fitness training function. The display device 200 can play fitness training videos containing standard movements provided by a supplier. Users can watch the fitness training videos and imitate the standard movements to perform fitness training.

[0074] A fitness training application may be installed in the display device 200. The user may control the display device 200 to call out an application panel to view the applications installed on the display device, and the application panel may include a fitness training control. Figure 5 Schematic diagrams of application panels in some embodiments are shown. Figure 5 As shown, the application panel includes three controls: "Browser," "Online Media," and "Fitness Training." A user can click the "Browser" control to control the display device 200 to open a browser application and browse various web pages. A user can click the "Online Media" control to view various online media on the display device 200. A user can click the "Fitness Training" control to enable the fitness training function on the display device 200, allowing the user to perform fitness training on the display device.

[0075] Figure 6 Schematic diagram of a fitness application homepage in some embodiments is shown. Figure 6 As shown, the homepage of the fitness application includes option controls for a variety of training items, including but not limited to aerobics, yoga, square dancing, parent-child sports, Tai Chi, slimming exercises, martial arts, etc.

[0076] In some embodiments, a user may register and log in to an account in a fitness application, so that the fitness application can record the user's historical training data under the account, such as Figure 6 As shown, the first area (e.g., the left area) on the homepage of the fitness application displays user information, energy consumption, training duration, last training item, and other information, wherein the user information includes but is not limited to user name and user avatar, etc., so that users can easily understand their recent training dynamics.

[0077] Figure 7 A schematic diagram of a training item list page in some embodiments is shown. A user can select any training item A of interest in the fitness application homepage and click on the option control corresponding to the training item A. The display device can display a training item list page. The training item list page includes subordinate training items of training item A. For example, if training item A is aerobics, the subordinate training items of aerobics include but are not limited to HIIT (High-Intensity Interval Training), Pamela, etc. In this way, users can more finely select the training items of interest based on their own preferences and abilities, such as selecting aerobics-HIIT.

[0078] In some embodiments, the user can select a training item on the training item list page, for example, select training item E. The display device can display relevant information about training item E in a preset area of ​​the training item list page (e.g., the upper part of the page), including but not limited to: training item name, theoretical calorie consumption, training part, difficulty level (e.g., entry-level, easy, difficult, high difficulty, etc.), training intensity level (e.g., beginner, intermediate, advanced, etc.), training duration, training item introduction, etc., so that the user can decide whether to select training item E for follow-up training.

[0079] In some embodiments, operation buttons may also be set in the preset area of ​​the training item list page, including but not limited to a start training button, a like button, a favorite button, a share button, a fitness VIP activation button, etc.

[0080] Figure 8 Schematic diagram of the training mode selection page in some embodiments is shown. When the user clicks the start training button associated with the training item E, the display device displays the training mode selection page. Figure 8 As shown, the training mode selection page includes training modes provided by fitness applications, including but not limited to follow-up mode, normal mode and movie viewing mode.

[0081] Normal mode only plays the training project video for users to watch. It doesn't support follow-up training features like keyframe marking, training scoring, or recording training data, and the image collector doesn't capture follow-up images. Viewer mode displays two separate windows on the app interface: Window 1 displays the training project video (which doesn't support follow-up training and the image collector doesn't capture follow-up images), and Window 2 displays other media resources (such as video resources and applications). This allows users to freely train while watching other media resources and referring to the training project video.

[0082] The follow-up training mode means that the operator marks the action keyframes for the training project video in advance. When the training project video is played to the key frame, the user refers to the standard action shown in the key frame, adjusts his or her body posture to perform the follow-up training action, and the image collector simultaneously collects the user's follow-up training image. The fitness application scores based on the degree of match between the standard action and the follow-up training action, and records training data such as training duration and calorie consumption.

[0083] In order to score the user's following training behavior, the display device 200 can obtain the user's movements during the fitness training process, and can use an image collector, such as a camera, to capture the user's image, and then compare the user image with a standard fitness training video to evaluate the user's following training behavior.

[0084] In some embodiments, the display device 200 may be equipped with an image collector to directly capture the user image.

[0085] The display device 200 is also provided with a device interface 240, which is used to connect an external device, thereby enabling the display device 200 to connect to the external device. The external device can be an image collector, such as a camera. When a user is within the camera's capture range, an image containing the user can be obtained, which is referred to as the user image in this embodiment of the application.

[0086] It should be noted that the camera can be used as an external device connected to the display device 200 via the external device interface 240 , or can be built into the display device 200 as a detector.

[0087] The image acquisition device built into the display device 200 may support a lifting function. Specifically, a camera may be mounted on a lifting mechanism. When image acquisition is required, specific lifting instructions are used to control the lifting mechanism to move, thereby raising the camera for image acquisition. When image acquisition is not required, specific lifting instructions are also used to control the lifting mechanism to move, thereby lowering the camera for concealment.

[0088] An image collector externally connected to the display device 200 can be connected to the external device interface 240 of the display device 200 to access the display device. The external image collector can be a standalone peripheral device connected to the display device 200 via a specific data interface. The display device 200 and the image collector have a data interface with the same interface specifications and functions, which can be a High Definition Multimedia Interface (HDMI). Through the HDMI interface, the display device 200 can receive user images captured by the image collector.

[0089] For the image acquisition device externally connected to the display device 200, other connection methods can also be supported, such as DVI (Digital Visual Interface), VGA (Video Graphics Array), USB (Universal Serial Bus), etc.; it can also be a wireless connection method, such as wireless LAN, Bluetooth connection, infrared connection, etc.

[0090] In order to facilitate the collection of user images, in some embodiments, the image collector externally connected to the display device 200 can be set at a position close to the display device 200, such as the image collector is clamped on the top of the display device 200 through a clamping device, or the image collector is placed on a desktop near the display device 200.

[0091] Figure 9 FIG. 1 shows a schematic diagram of an image collector collecting user images in some embodiments. Figure 9 As shown, after the user controls the display device 200 to turn on the fitness training function, the controller 250 can obtain the user's image through the image collector. The image collector can send the user's image to the display device 200, and the display device 200 can evaluate the user's fitness training behavior based on the user image.

[0092] When evaluating a user's follow-up training, the app can use image recognition algorithms to match standard movements in fitness training videos with the user's follow-up movements. For example, the app can match key points of the user's limbs in the image to obtain similarity information, indicating the standardization of the user's follow-up training behavior and providing a score for the user's follow-up movements.

[0093] The system can also generate comprehensive statistics for the entire training process and display them to the user after the training. The training data includes but is not limited to the final score, total energy consumption, accuracy, and total training time.

[0094] It's important to note that the Fitness Follow-Up Score is a comprehensive evaluation of the user's overall follow-up process, including the overall composite score for multiple exercises in a fitness follow-up video. This data allows users to understand their overall follow-up performance across the entire set of exercises in a training program.

[0095] However, a fitness training video may include multiple action groups, each containing different training movements, which can provide targeted training for certain body parts of the user. For example, the action groups included in aerobics may include: scissors, jumping jacks, cross lunges, squats, etc. At the same time, an action group may also include multiple movements.

[0096] Practice data only reflects the overall progress of all user exercises. Users cannot understand the progress of individual exercises, resulting in uncertainty about whether each exercise is being practiced correctly. Due to the large number of exercises, users cannot identify specific non-standard exercises and thus cannot focus on strengthening those non-standard exercises. Furthermore, users do not know why these exercises were not practiced correctly, making it impossible to re-practice them in a targeted manner, thus hindering the user's practice experience.

[0097] To address the above issues, the display device 200 provided in the embodiment of the present application may have a fitness training error correction function. During a user's fitness training, the display device may capture a user image and compare the user image with the currently displayed action set to match the action image corresponding to the user image. The display device may compare the user image with the action image to correct errors in the user image, for example, detecting whether the user's current action is standard and correct, thereby providing error correction prompts for the user's training process and improving the user's experience.

[0098] In some embodiments, a user can send a fitness training and error correction mode instruction to the display device 200 by operating a designated button on the remote control. In actual application, the correspondence between the fitness training and error correction mode instruction and the remote control button is pre-bound. For example, a fitness training and error correction mode button is set on the remote control. When the user touches the button, the remote control sends the fitness training and error correction mode instruction to the controller 250. At this time, the controller 250 controls the display device 200 to enter the fitness training and error correction mode to enable the fitness training and error correction function.

[0099] A fitness follow-up error correction mode option may also be set in the system UI interface of the display device 200. When the user clicks on the option, the display device 200 may be controlled to enter or exit the fitness follow-up error correction mode.

[0100] In some embodiments, to prevent the user from accidentally triggering the fitness and training error correction mode, when the controller 250 receives the fitness and training error correction mode instruction, it can control the display 260 to display the fitness and training error correction mode confirmation information, so that the user can perform a second confirmation whether to control the display device 200 to enter the fitness and training error correction mode. Figure 10 A schematic diagram illustrating confirmation information of a fitness follow-up error correction mode in some embodiments is shown.

[0101] Figure 11 A flowchart of the interaction between components of a display device in some embodiments is shown, including the following steps:

[0102] S101: In response to a fitness training instruction, controlling a display to display a fitness training video and acquiring a user image through an image collector. The fitness training video includes a plurality of action groups, each of which includes a plurality of frames of action images;

[0103] S102, obtaining action group data of the current action group; the action group data includes first limb key point data and a target limb region of multiple frames of action images, the target limb region including multiple limb key points;

[0104] S103, comparing the current frame user image with multiple frames of action images to obtain a target action image that matches the current frame user image;

[0105] S104, detecting the second limb key point data of the current frame user image; generating error correction information of the target limb area based on the first limb key point data and the second limb key point data of the target action image;

[0106] S105: Control the display to display the error correction information.

[0107] In some embodiments, a user can send a fitness training instruction to a display device to perform fitness training using the display device. The user can control the display device to launch a fitness training application. The fitness training application can provide a variety of fitness training videos, such as aerobics. The user can select one of the fitness training videos and send the fitness training instruction.

[0108] In response to the user's instruction, the controller may control the display to display a fitness training video and acquire a user image through the image collector.

[0109] In order to improve the accuracy of error correction for user fitness training, the display device can detect the collected user image to ensure that the user image includes the user's body part. After starting the image collector, the controller can control the display to display the portrait recognition interface. Figure 12 Schematic diagrams of portrait recognition interfaces in some embodiments are shown. Figure 12 As shown, the controller can detect whether the image contains a user. If no real human portrait is detected, the portrait recognition interface can display a virtual human image with an aperture area under the feet of the virtual human. This aperture area is the effective recognition area of ​​the image collector. The image collector can adjust the shooting position and angle. When the camera moves, the aperture moves with it, and eventually the aperture area is adjusted to the center of the screen. A guidance message is displayed, such as "Please stay alone in the circle." In this way, the user only needs to move their body into the aperture area, so that the image collector can successfully capture the human portrait.

[0110] As the user moves, the image collector and aperture also move accordingly, that is, the image collector will track the human body. When it detects that the user's position deviates far from the center of the screen (for example, at the edge of the screen), the display device can prompt the user to stand in the center of the screen.

[0111] In some embodiments, to facilitate comparison of the user's fitness movements with standard fitness movements and to facilitate user confirmation of their own movements, the controller can control the display to display a fitness training interface. The fitness training interface includes a first window and a second window, which can respectively display the user's image and the fitness training video.

[0112] The controller can display a fitness training video in a first window and a user image in a second window. This allows the user to simultaneously view the fitness training video and their own training movements. The user can compare and adjust their own training movements. Figure 13 Schematic diagrams of fitness training interfaces in some embodiments are shown. Figure 13 As shown, the left window in the fitness training interface is the first window, which can display the fitness training image. The right window is the second window, which can display the user image. The fitness training interface also includes a "Start Fitness" control. When the user clicks this control, the controller can start playing the fitness training video.

[0113] In some embodiments, the controller can download a fitness training video through a fitness training application for playback. The fitness training video includes multiple movement groups. The fitness training videos are provided by relevant operators. The movement groups included in each fitness training video are formulated by the operator, and the specific content of the movement groups is also formulated by the operator.

[0114] The controller can obtain the playback data corresponding to the fitness training video, including video data and audio data, and then play it.

[0115] The controller can compare the fitness training video and the user image to obtain the accuracy of the user's training. It should be noted that the action group contained in the fitness training video can instruct the user to perform targeted training on certain limb areas, such as training the arms or legs. The limb areas specifically indicated by different action groups can be different and are determined by the operator. The limb areas specifically indicated by the action group can be used as the focus of the user's training action correction. Therefore, when correcting the user's training, it is possible to specifically detect whether the limb area indicated by the current action group is accurate.

[0116] In some embodiments, the user's body area can be represented by multiple limb key points, and the accuracy of the user's training movements can be represented by comparing the matching degree of the limb key points in the fitness training video and the user image.

[0117] The user's body area can be set to 17 limb key points. Figure 14 Schematic diagrams showing key points of user limbs in some embodiments are shown. Figure 14 As shown, the limb key points may include facial key points: nose 0, left eye 1, right eye 2, left ear 3 and right ear 4, and torso key points: left shoulder 5, right shoulder 6, left elbow 7, right elbow 8, left wrist 9, right wrist 10, left hip 11, right hip 12, left knee 13, right knee 14, left ankle 15 and right ankle 16.

[0118] It should be noted that the purpose of the action group instruction training is not to focus on a specific limb key point, but rather the positional relationship formed by multiple limb key points, such as requiring the arm and waist to form a certain angle, or requiring the legs to be raised to form a certain angle. Therefore, the action group instruction training is for a specific limb area of ​​the user. In the embodiment of the present application, the limb area refers to the area formed by multiple limb key points.

[0119] To this end, the embodiment of the present application defines the following limb regions, each of which is composed of multiple limb key points. Figure 14 Taking the limb key points shown in the figure as an example, the limb areas include: the area formed by the left shoulder to the left hip {5,11,-1}, the area formed by the right shoulder to the right hip {6,12,-1}, the left arm area {5,7,9}, the area formed by the left arm and the body trunk {11,5,7}, the area formed by the right arm and the body trunk {8,6,12}, the right arm area {6,8,10}, the left leg area {11,13,15}, the area formed by the left shoulder to the left knee {5,11,13}, the left calf area {13,15,-1}, the area formed by the right shoulder to the right knee {6,12,14}, the right leg area {12,14,16}, and the right calf area {14,16,-1}.

[0120] Among them, "-1" means that the area is perpendicular to the ground. For example, the area {5,11,-1} means that the area from the left shoulder to the left hip (i.e., the left upper torso) must be perpendicular to the ground.

[0121] Limb areas can form a certain angle. Each action group can associate several limb areas and require these limb areas to form a certain degree of angle. For example, the action group can associate the left arm area {5,7,9} and the right arm area {6,8,10}, and require the left arm to present angle A (that is, the angle of angle A formed with the left elbow as the vertex), and require the right arm to present angle B. Considering that the angle is difficult to reflect in the image, the position coordinates of each limb key point in the image can be obtained, and the angle can be calculated by the position coordinates. Therefore, the action group can indicate the angle that the corresponding limb area needs to form through the position coordinates of the limb key points in the image.

[0122] It should be noted that an action group is essentially a video content, which is composed of multiple frames of images. An action group may include a large number of images. In the embodiment of the present application, it is assumed that the limb areas associated with the multiple frames of images included in each action group are the same.

[0123] In some embodiments, the operator who produces the fitness training video can set the limb area associated with each action group, and can also mark the limb key points in the multi-frame images of the action group, for example, mark the position coordinates of each limb key point.

[0124] This data can be packaged to generate motion parameters associated with the fitness training video. Simultaneously, the playback data corresponding to the fitness training video and these motion parameters can be further packaged and processed as download data for the fitness training video. After the user instructs to play the fitness training video, the controller can retrieve the download data for the fitness training video, thereby obtaining the playback data and motion parameters for the fitness training video.

[0125] In some embodiments, when a user practices a certain action group, the display device can compare the user image with the standard image of the current action group to analyze the similarity between the user's actions and the character actions in the standard image, thereby determining the accuracy of the user's practice.

[0126] It should be noted that an action group itself is a piece of video content, which consists of multiple frames of images at a certain frame rate. For example, an action group A with a duration of 15 seconds and a frame rate of 60 contains 900 frames of images. However, since the action group is a targeted training for users and its duration is relatively short, the number of standard actions set in an action group may be small. For example, a 15-second action group A may only have 10 standard actions, that is, 900 frames of images correspond to 10 standard actions. This will cause multiple adjacent frames of images to be too similar, that is, adjacent images may correspond to one standard action.

[0127] If every frame of an image is compared with the user's image, the comparison results may be relatively similar. For example, when a user is practicing standard action A, there may be 30 adjacent frames of images that all correspond to standard action A. These images are highly similar, and the comparison results between the user's image and each frame are also relatively similar, perhaps between 80% and 82%. In this case, the accuracy of the user's practice of standard action A is a range, rather than an exact value, resulting in inaccurate evaluation of the user's practice. Furthermore, the workload of comparing every frame of an image with the user's image is large, and the system's resource consumption rate is high.

[0128] To this end, the operator can extract keyframe images from the multiple frames of the action group and set the keyframe images as the images corresponding to the standard actions. When the display device compares the user image with the action group images, it only needs to compare the user image with the keyframe images. In the embodiments of the present application, the keyframe images are referred to as action images.

[0129] Each standard action in an action group can be assigned a fixed image frame. Each action frame can correspond to one standard action. For example, if 10 standard actions are set in a 15-second action group A, 10 keyframe images can be extracted. The keyframe image can be the one that is most similar to the standard action, as determined by the operator, and thus corresponds to each of the 10 standard actions.

[0130] It should be noted that the standard actions included in the action group are set by the operator. The operator can first specify the standard action and then obtain a keyframe image (i.e., action image) based on the standard action. In this case, the keyframe image is the image that is most similar to the pre-set standard action.

[0131] The operator can also directly extract the key frame image and set the action corresponding to the extracted key frame image as the standard action. The operator can extract images according to a certain period, for example, extracting one frame of image per second, as a key frame image (i.e., an action image). It is also possible to extract images at a certain frequency, for example, extracting one frame of image from 30 frames of image as a key frame image. The specific period or frequency is set by the operator. The operator can set each frame of image as an action image, or extract one frame from more frames of image as an action image. This is not limited in the embodiments of the present application.

[0132] After setting the action image of each action group, the operator can count the action images of an action group, including the limb key point data (position coordinates of the limb key points) in each frame of the action image, and package the limb key point data and the limb area information associated with the action group to form the action group data of each action group.

[0133] When the display device obtains a fitness training video, it can simultaneously obtain the playback data and the action group data of each action group. Based on the action group data, the display device can compare the user image with the current action group image to determine the accuracy of the user's training.

[0134] In some embodiments, the controller can detect the playback status of the fitness training video to determine the currently displayed action group, which is referred to as the current action group in this embodiment of the application.

[0135] The controller can obtain action group data for the current action group. The action group data includes limb key point data corresponding to multiple frames of action images of the current action group, referred to as first limb key point data in this embodiment of the application, and also includes a limb region associated with the current action group, referred to as a target limb region in this embodiment of the application. The target limb region includes multiple limb key points. The number of target limb regions can be one or more, determined by the operator.

[0136] In some embodiments, the controller may compare the user image captured by the image collector with images in the fitness training video.

[0137] Considering that users often engage in fitness training for extended periods of time, the image collector can send captured user images to a display device in real time. The image collector can capture user images at a specific frame rate, such as 30 frames per second. While the image collector is capturing user images, it can continuously send user images to the display device. The frame rate at which the image collector sends user images can be the same as the frame rate at which the image collector captures the images, allowing the display device to display the user images in real time.

[0138] The display device can analyze each frame of the captured user image to evaluate the user's fitness training and perform error correction.

[0139] After acquiring the current frame user image, the controller can compare the current frame user image with the current action group.

[0140] In some embodiments, the current frame of the user image can be compared with the most recent frame of the action image displayed on the display in chronological order. However, the frame rate of the current action group and the frame rate at which the image collector captures the user image may be different. It takes a certain amount of time for the user to perform an action after seeing the standard action group image. As a result, the captured user image and the most recent frame of the action image determined in chronological order may not match. In other words, the actions in the two frames are not the same, resulting in comparison failure.

[0141] To this end, the controller may compare the current frame user image with multiple frames of action images to obtain a target action image that matches the current frame user image.

[0142] In some embodiments, the controller may respectively calculate the image similarities of the multiple frames of action images and the current frame of the user image, and the controller may extract the maximum image similarity therebetween.

[0143] When calculating image similarity, the controller may first perform limb key point detection on the current frame user image to obtain second limb key point data, including the position coordinates of multiple limb key points. The controller may compare the second limb key point data with the limb key point data of multiple frames of action images to obtain the action matching degree between the multiple frames of action images and the current frame user image, and use it as image similarity. Image similarity is used to characterize the degree of similarity between two actions. Among them, the position coordinates of all limb key points can be compared to obtain the similarity between the coordinates. The similarity of all limb key points can be obtained by weighted summation and taking the average value as the image similarity.

[0144] It's important to note that there's no guarantee that all actions performed by the user will be in the action set. For example, they might be resting due to fatigue. The actions in the current frame of the user image might not be similar to those in any other action image. This means that the user isn't currently exercising, so fitness correction isn't required for the current frame.

[0145] To this end, a similarity threshold, such as 60%, may be preset to evaluate whether the user is performing fitness training.

[0146] After obtaining the maximum image similarity, the controller first compares the maximum image similarity with the similarity threshold.

[0147] If the maximum image similarity is greater than or equal to the similarity threshold, it indicates that an action image is similar enough to the current frame user image that the action image corresponding to the current frame user image can be matched. At this point, the action image corresponding to the maximum image similarity can be marked as the action image that matches the current frame user image, referred to as the target action image in this embodiment of the application.

[0148] If the maximum image similarity is less than the similarity threshold, it indicates that the current user image is not similar to any of the action images, and no action image can be matched to the current user image. In this case, the current user image can be ignored, meaning that no fitness correction or other evaluation processes are performed on the current user image. After the image collector captures the next user image, the controller can compare the next user image with multiple action images to perform error correction on the next user image.

[0149] In some embodiments, after obtaining the target action image that matches the current frame user image, the controller may further compare the current frame user image with the target action image to perform fitness correction on the current frame user image.

[0150] The controller may first detect second limb key point data for the current frame of the user image. The second limb key point data includes the position coordinates of multiple limb key points in the current frame of the user image. The controller may perform image recognition processing on the current frame of the user image based on a preset image recognition algorithm to obtain the position coordinates of the limb key points.

[0151] The controller can obtain error correction information of the target limb area in the current frame user image based on the first limb key point data of the target action image and the second limb key point data of the current frame user image.

[0152] It's important to note that action sets are associated with limb regions, with each frame of the action image having targeted training requirements for certain limb regions. Therefore, when a user follows the current action set, it can be specifically checked to see if the user is following the target limb region correctly.

[0153] Because the limb region is composed of multiple key points, which are required to form a certain angle, we can check whether the angle formed by the target limb region in the current frame of the user image is the same as the corresponding angle in the action image, thereby determining the accuracy of the follow-up training.

[0154] In some embodiments, the controller can obtain first position information of multiple limb key points in a target limb region based on the first limb key point data, and calculate a first limb angle based on the first position information. The limb region includes three limb key points that can form a certain angle, and the angle of the limb region can be calculated based on the position coordinates of the three limb key points.

[0155] The controller can obtain second position information of multiple limb key points in the target limb area based on the second limb key point data, and calculate the second limb angle based on the second position information.

[0156] Considering that the user may not be able to ensure that the angle of the target limb area in their own movement is exactly the same as the angle required in the action image during the training process, an angle threshold can be set to measure whether the user's movement is standard.

[0157] The controller can calculate the angle difference between the first limb angle and the second limb angle. If the angle difference is greater than an angle threshold, a first error correction message is generated, and the first error correction message table is used to indicate that the current action is incorrect. If the angle difference is less than or equal to the angle threshold, a second error correction message is generated, and the second error correction message table is used to indicate that the current action is correct.

[0158] It should be noted that the target limb area associated with an action group may be one or more. When there are multiple target limb areas, the controller generates error correction information for each target limb area. The error correction processes for multiple target limb areas are independent of each other and do not affect each other.

[0159] In some embodiments, after obtaining error correction information for the target limb area, the controller can control the display to display the error correction information to prompt the user. The controller can control the display to display the error correction information in the second window, and can control the display to display only erroneous error correction information, while omitting correct information, to highlight the error correction information to the user.

[0160] Figure 15 Schematic diagrams showing error correction information in some embodiments are shown. Figure 15As shown, a fitness training interface is displayed on the monitor, a fitness training image is displayed in the left window, and a user image is displayed in the right window. Multiple action groups in the fitness training video can be displayed below the fitness training interface, such as the action group currently being trained is jumping jacks. The left window includes the image content of the current action group, the playback progress, and the prompt information of the current action. The user image and error correction information are displayed in the right window. Among them, the error correction information 151 is "left elbow error, right elbow error", indicating that there are errors in the left and right arms of the limb area.

[0161] In some embodiments, after the user completes the current set of exercises, feedback on the user's progress can be provided to the user. The exercises that require correction in the set of standard exercises can be counted to prompt the user to re-practice these exercises.

[0162] The controller can detect the playback status of the current action group.

[0163] When the current action group is finished playing, the controller calculates the user's accuracy in following the multiple frames of action images of the current action group.

[0164] When obtaining the accuracy of follow-up training, the controller can count the user images that match the action image, where the user image can be at least one frame, and obtain the number of these user images, which is referred to as the first user image number in the embodiment of the present application.

[0165] The controller can obtain the number of user images whose error correction information is the second error correction information (and the action is correct) among these user images, which is referred to as the second user image number in the embodiment of the present application.

[0166] The controller calculates a first ratio of the number of the second user images to the number of the first user images to generate a user's accuracy in following the action image.

[0167] A threshold value, such as 60%, may be preset. The controller may detect the relationship between the accuracy of the training of the action image and the preset threshold value. If the accuracy of the training is less than the preset threshold value, the action image is determined to be an error correction action image.

[0168] In some embodiments, after obtaining the practice accuracy of multiple frames of action images, the controller may obtain a preset number of action images with the lowest practice accuracy, referred to as error-correcting action images in this embodiment of the application. The preset number can be set by the user, for example, setting it to 3, which means that the three action images with the lowest practice accuracy are obtained as error-correcting action images. The display device can display the error-correcting action images to the user, so that the user can repeat the practice.

[0169] The controller can control the display to display a follow-up correction interface, which is used to display correction action images. Figure 16 Schematic diagrams of the practice and error correction interface in some embodiments are shown. Figure 16 As shown, the practice and correction interface includes image controls corresponding to the correction action images, including three controls in the first window on the left: "Image 1," "Image 2," and "Image 3." The image controls can display thumbnails of the correction action images. The right window still displays the user image.

[0170] The user can select an image control to re-practice the error correction action image corresponding to the control.

[0171] In response to a user selecting a target image control, the controller may control the display to display a target correction action image and a correction marker corresponding to the target image control. The correction marker is used to mark the target limb area in the target correction action image. The target correction action image may be displayed in the first window. The correction marker may be a highlighted marker that can mark key limb points in the target limb area.

[0172] Figure 17 Schematic diagrams of error correction markers in some embodiments are shown. Figure 17 As shown, the first window displays the target correction action image. The correction mark 171 can be a circle mark that marks the limb area associated with the target correction action image. The limb area is formed by three limb key points forming an angle. The correction mark can be marked at the limb key point corresponding to the vertex of the angle to remind the user of the key point of the current action.

[0173] In some embodiments, if the user is prompted to re-practice incorrect movements after practicing a set of movements, this will interrupt the user's practice rhythm and affect the user's user experience. The user may want to complete the entire fitness training video before considering re-practice. To this end, the display device can prompt the user to re-practice incorrect movements after the user has completed the entire fitness training video.

[0174] The controller can detect the playback status of fitness and exercise videos.

[0175] When the fitness training video is finished playing, the controller obtains fitness training scores of the multiple action groups respectively.

[0176] When obtaining the fitness training score, the controller first obtains the user's training accuracy for the multiple frames of action images of the action group. The method for obtaining the training accuracy can refer to the above steps.

[0177] The controller can count the number of action images in the action group (referred to as the first number of action images in this embodiment) and can also count the number of action images in these images whose follow-up accuracy is less than an accuracy threshold (referred to as the second number of action images in this embodiment). The accuracy threshold can be pre-set, for example, 60%.

[0178] The controller can calculate a second ratio of the number of second action images to the number of first action images, and generate a fitness training score for the action group based on the second ratio. A maximum score, X, for the fitness training score can be pre-set, and the product of the maximum score, X, and the second ratio can be calculated as the fitness training score. For example, if the number of second action images is 7 and the number of first action images is 10, the second ratio is 0.7. Assuming the maximum score is 100, the fitness training score is 70.

[0179] The controller can control the display to display a fitness training details interface. The fitness training details interface includes fitness training scores for multiple action groups and retraining controls for multiple action groups.

[0180] Figure 18 Schematic diagrams of fitness training details interfaces in some embodiments are shown. Figure 18 As shown, the fitness training details interface includes detailed information about the user's training for each action group, such as energy consumption, fitness training scores, and training duration. Each action group also corresponds to a retraining control and a playback control. After the user selects the retraining control, the display device can redisplay the action group and continue to collect user images, allowing the user to retrain for the action group. After the user selects the playback control, the display device can replay the user image of the user training the action group.

[0181] In response to a selection operation on a retraining control of a target action group, the controller may control the display to display the target action group and acquire a user image through the image collector.

[0182] The present application also provides a fitness training error correction method, which is applied to a display device. The method includes:

[0183] Step 101: In response to a fitness training instruction, controlling a display to display a fitness training video and acquiring a user image through an image collector. The fitness training video includes multiple action groups, each action group includes multiple frames of action images.

[0184] Step 102: Acquire action group data of the current action group; the action group data includes first limb key point data and a target limb region of multiple frames of action images, wherein the target limb region includes multiple limb key points;

[0185] Step 103: Compare the current frame user image with multiple frames of action images to obtain a target action image that matches the current frame user image;

[0186] Step 104: Detect the second limb key point data of the current frame user image; generate error correction information of the target limb area based on the first limb key point data and the second limb key point data of the target action image;

[0187] Step 105: Control the display to display error correction information;

[0188] In some embodiments, comparing the current frame user image with multiple frames of action images further includes:

[0189] The image similarities of multiple frames of action images and the current frame of user image are calculated respectively, and the maximum image similarity is extracted.

[0190] If the maximum image similarity is greater than or equal to the similarity threshold, the action image corresponding to the maximum image similarity is marked as the target action image for matching the current frame user image. If the maximum image similarity is less than the similarity threshold, the next frame user image is compared with multiple frames of action images.

[0191] In some embodiments, generating error correction information of a target limb region based on the first limb key point data and the second limb key point data of the target action image further includes:

[0192] Based on the first limb key point data, first position information of multiple limb key points in the target limb region is obtained, and a first limb angle is calculated based on the first position information. Based on the second limb key point data, second position information of multiple limb key points in the target limb region is obtained, and a second limb angle is calculated based on the second position information. The angle difference between the first limb angle and the second limb angle is calculated.

[0193] If the angle difference is greater than the angle threshold, a first error correction information is generated, and the first error correction information table is used to indicate that the current action is wrong. If the angle difference is less than or equal to the angle threshold, a second error correction information is generated, and the second error correction information table is used to indicate that the current action is correct.

[0194] In some embodiments, after acquiring the user image through the image collector, the method further includes:

[0195] The display is controlled to display a fitness training interface; the fitness training interface includes a first window and a second window.

[0196] The display is controlled to display the fitness training video in the first window and the user image in the second window. After the error correction information is obtained, the display is controlled to display the error correction information in the second window.

[0197] In some embodiments, it further includes:

[0198] Detect the playback status of the current action group. When the current action group completes playback, calculate the follow-up accuracy of multiple frames of the action image for the current action group. Obtain a preset number of correction action images whose follow-up accuracy is less than a preset threshold. Control the display to display the follow-up correction interface, which is used to display the correction action images.

[0199] In some embodiments, obtaining the user's accuracy in following the multiple frames of action images of the current action group further includes:

[0200] Count at least one frame of user images that matches the action image, and obtain the number of first user images in the at least one frame of user images. Obtain the number of second user images in the at least one frame of user images whose error correction information is the second error correction information. Calculate a first ratio of the number of second user images to the number of first user images to generate an accuracy level of the user's training with the action image.

[0201] In some embodiments, the follow-up correction interface includes an image control corresponding to the correction action image, and the image control is used to display a thumbnail of the correction action image.

[0202] In response to the selection operation of the target image control, the display is controlled to display the target error correction action image and error correction mark corresponding to the target image control; the error correction mark is used to mark the target limb area in the target error correction action image.

[0203] In some embodiments, it further includes:

[0204] Detect the playback status of a fitness training video. When the fitness training video completes playback, obtain fitness training scores for each of the multiple exercise groups. Control the display to display a fitness training details interface; the fitness training details interface includes fitness training scores for the multiple exercise groups and retraining controls for the multiple exercise groups.

[0205] In response to a selection operation on a retraining control of a target action group, the display is controlled to display the target action group, and a user image is acquired through an image collector.

[0206] In some embodiments, obtaining fitness training scores for multiple action groups separately further includes:

[0207] Obtain the user's follow-up accuracy for the multiple frames of action images for the action set. Count the number of first action images in the multiple frames of action images for the action set, and the number of second action images for which the follow-up accuracy is less than a threshold. Calculate a second ratio of the number of second action images to the number of first action images, and generate a fitness follow-up score for the action set based on the second ratio.

[0208] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be repeated here.

[0209] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus a necessary general-purpose hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various embodiments of the present invention or certain portions of the embodiments.

[0210] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

[0211] For ease of explanation, the above description has been presented in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations are possible. The above embodiments have been selected and described to better explain the principles and practical applications, thereby enabling those skilled in the art to better utilize the embodiments and various variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: monitor; a device interface configured to connect to an image collector; The controller is configured as: In response to a fitness training instruction, controlling the display to display a fitness training video and acquiring a user image through an image collector; the fitness training video includes a plurality of action groups, and the action groups include a plurality of frames of action images; Acquire action group data of the current action group; the action group data includes first limb key point data and a target limb area of ​​multiple frames of action images, wherein the target limb area includes multiple limb key points; Comparing the current frame user image with multiple frames of action images to obtain a target action image that matches the current frame user image; Detecting the second limb key point data of the current frame user image; generating error correction information of the target limb area based on the first limb key point data and the second limb key point data of the target action image; The display is controlled to display the error correction information.

2. The display device according to claim 1, wherein The controller compares the current frame user image with multiple frames of action images, and is further configured to: Calculating image similarities between the multiple frames of action images and the current frame of the user image, and extracting the maximum image similarity; If the maximum image similarity is greater than or equal to the similarity threshold, marking the action image corresponding to the maximum image similarity as the target action image for matching the current frame user image; If the maximum image similarity is less than the similarity threshold, the next frame of the user image is compared with the multiple frames of action images.

3. The display device according to claim 1, wherein The controller generates error correction information of the target limb area based on the first limb key point data and the second limb key point data of the target action image, and is further configured to: Acquire first position information of a plurality of limb key points in the target limb region according to the first limb key point data, and calculate a first limb angle according to the first position information; Acquire second position information of a plurality of limb key points in the target limb region according to the second limb key point data, and calculate a second limb angle according to the second position information; calculating an angle difference between the first limb angle and the second limb angle; If the angle difference is greater than the angle threshold, generating first error correction information, wherein the first error correction information table is used to characterize the current action error; If the angle difference is less than or equal to the angle threshold, second error correction information is generated, and the second error correction information table is used to indicate that the current action is correct.

4. The display device according to claim 1, wherein After the controller acquires the user image through the image collector, it is further configured to: Controlling the display to display a fitness training interface; the fitness training interface includes a first window and a second window; Controlling the display to display the fitness training video in the first window and to display the user image in the second window; After acquiring the error correction information, controlling the display to display the error correction information in the second window.

5. The display device according to claim 3, wherein The controller is further configured to: Detecting the playback status of the current action group; When the current action group is finished playing, respectively calculating the accuracy of the follow-up of multiple frames of action images of the current action group; Obtaining a preset number of error correction action images whose training accuracy is less than a preset threshold; The display is controlled to display a follow-up practice and error correction interface, wherein the follow-up practice and error correction interface is used to display the error correction action image.

6. The display device according to claim 5, wherein: The controller is configured to respectively obtain the user's accuracy in following the multiple frames of action images of the current action group, and is further configured to: Counting at least one frame of user images that matches the action image, and obtaining a first number of user images of the at least one frame of user images; Acquire the number of second user images of user images whose error correction information is the second error correction information in the at least one frame of user image; A first ratio of the number of the second user images to the number of the first user images is calculated to generate a user's accuracy in following the action image.

7. The display device according to claim 5, wherein: The following practice and error correction interface includes an image control corresponding to the error correction action image, and the image control is used to display a thumbnail of the error correction action image; the controller is further configured to: In response to the selection operation of the target image control, the display is controlled to display the target error correction action image and error correction mark corresponding to the target image control; the error correction mark is used to mark the target limb area in the target error correction action image.

8. The display device according to claim 1, wherein The controller is further configured to: Detecting the playback status of the fitness training video; After the fitness training video is played, fitness training scores of the plurality of movement groups are obtained respectively; Control the display to display the fitness training details interface; The fitness training details interface includes fitness training scores for multiple action groups and retraining controls for multiple action groups; In response to a selection operation on a retraining control of a target action group, the display is controlled to display the target action group, and a user image is acquired through an image collector.

9. The display device according to claim 8, wherein The controller is configured to respectively obtain fitness training scores for multiple action groups and is further configured to: Obtaining the user's accuracy in following the multiple frames of action images of the action group; Counting the number of first action images in the multiple-frame action images of the action group and the number of second action images of the action images whose follow-up accuracy is less than an accuracy threshold; A second ratio of the number of the second action images to the number of the first action images is calculated, and a fitness follow-up score for the action group is generated according to the second ratio.

10. A fitness training error correction method, applied to a display device, characterized in that: The method comprises: In response to a fitness training instruction, controlling the display to display a fitness training video and acquiring a user image through an image collector; the fitness training video includes a plurality of action groups, and the action groups include a plurality of frames of action images; Acquire action group data of the current action group; the action group data includes first limb key point data and a target limb area of ​​multiple frames of action images, wherein the target limb area includes multiple limb key points; Comparing the current frame user image with multiple frames of action images to obtain a target action image that matches the current frame user image; Detecting the second limb key point data of the current frame user image; generating error correction information of the target limb area based on the first limb key point data and the second limb key point data of the target action image; The display is controlled to display the error correction information.

Images