Display device and follow-up training mode control method

Abstract

The invention provides a display device and a follow-up training mode control method, and the method comprises the steps: responding to an operation instruction for starting an AI follow-up training mode, and reading a function state mark value of an image collector, the function state mark value comprising a first numerical value and a second numerical value; if the function state mark value is a first numerical value, determining a follow-up training score according to an action group file corresponding to the training item video and a follow-up training action file of the user, and displaying the follow-up training score in a preset window; and if the functional state mark value is the second numerical value, starting a process of a video follow-up training mode in the follow-up training modes, and starting a process of an AI follow-up training mode in the follow-up training modes when it is detected that the functional state mark value of the image collector is the first numerical value. According to the method, the continuity of follow-up training of the user can be ensured, training cannot be interrupted even if the image collector goes wrong, and the problem that the follow-up training interface cannot be displayed and the user experience is affected when the AI follow-up training mode is interrupted in the follow-up training process is solved.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device and a control method for a follow-up mode. Background Technology

[0002] Fitness apps can be installed on display devices. When a user opens a fitness app, the display device will show the app's interface. Fitness apps can provide users with a variety of training programs. The app's homepage includes options for various training programs, allowing users to select the appropriate programs based on their personal training preferences for diversified training.

[0003] After selecting the desired training video, users can click "Start Training," which will redirect them to the training mode selection page. This page includes training modes offered by fitness apps, including but not limited to AI-powered follow-along mode, video follow-along mode, and movie viewing mode. When the display device lacks an image capture device, the user will select video follow-along mode. When the display device has an image capture device, the user can choose either video follow-along mode or AI follow-along mode. AI follow-along mode relies on the image capture device to detect limb data and match movements. Therefore, if the image capture device is present and functioning properly, the TV will respond to the user's selection of AI follow-along mode.

[0004] However, in AI follow-up mode, if the image acquisition device malfunctions, such as being removed or experiencing a problem, the TV will directly exit AI follow-up mode, causing the mode to be interrupted. After the interruption, the TV may display a follow-up error message, thus exiting the mode. Consequently, the display device will be unable to show the user's follow-up interface, impacting the user experience. Summary of the Invention

[0005] Some embodiments of this application provide a display device and a control method for a follow-up training mode to solve the problem that when the AI ​​follow-up training mode is interrupted during the follow-up training process, the display device cannot display the follow-up training interface, which affects the user experience.

[0006] In a first aspect, some embodiments of this application provide a display device, including: The monitor is configured to display the practice interface; The device interface is configured to connect to an image acquisition device, which is configured to acquire images of the user's follow-up training. A communication device configured to establish a communication connection with a server; The controller is configured as follows: In response to an operation command for activating the AI ​​follow-up mode, the function status flag value of the image acquisition device is read. The function status flag value includes a first value and a second value. The first value is used to indicate that the image acquisition device is in an available state, and the second value is used to indicate that the image acquisition device is in an unavailable state. If the function status flag value is the first value, start the AI ​​follow-up mode process in the follow-up mode; obtain the action group file corresponding to the training project media asset and the action file corresponding to the follow-up image, determine the follow-up score according to the action group file and the follow-up action file, and control the display to display the follow-up score on the follow-up interface. If the function status flag value is the second value, the process of the video follow-up mode in the follow-up mode will be started, and the function status flag value of the image acquisition device will be continuously read. When the function status flag value of the image acquisition device is detected to be the first value, the process of the AI ​​follow-up mode in the follow-up mode will be started; the video follow-up mode is a mode that does not support the acquisition of follow-up images.

[0007] Secondly, some embodiments of this application provide a control method for a follow-up mode, which can be applied to the display device of the first aspect. The display device includes a display, a device interface, a communication device, and a controller. The method includes: In response to an operation command for activating the AI ​​follow-up mode, the function status flag value of the image acquisition device is read. The function status flag value includes a first value and a second value. The first value is used to indicate that the image acquisition device is in an available state, and the second value is used to indicate that the image acquisition device is in an unavailable state. If the function status flag value is the first value, start the AI ​​follow-up mode process in the follow-up mode; obtain the action group file corresponding to the training project media asset and the action file corresponding to the follow-up image, determine the follow-up score according to the action group file and the follow-up action file, and control the display to display the follow-up score on the follow-up interface. If the function status flag value is the second value, the process of the video follow-up mode in the follow-up mode will be started, and the function status flag value of the image acquisition device will be continuously read. When the function status flag value of the image acquisition device is detected to be the first value, the process of the AI ​​follow-up mode in the follow-up mode will be started; the video follow-up mode is a mode that does not support the acquisition of follow-up images.

[0008] As can be seen from the above technical solutions, some embodiments of this application provide a display device and a control method for a follow-up training mode. The method includes: responding to an operation command for starting an AI follow-up training mode, reading the functional status flag value of an image acquisition device, the functional status flag value including a first value and a second value; the first value is used to indicate that the image acquisition device is in an available state, and the second value is used to indicate that the image acquisition device is in an unavailable state; if the functional status flag value is the first value, starting the process of the AI ​​follow-up training mode in the follow-up training mode; acquiring the action group file corresponding to the training project media assets and the action file corresponding to the follow-up training image, and determining the follow-up training score based on the action group file and the follow-up training action file, and controlling the display to display the follow-up training score on the follow-up training interface; if the functional status flag value is the second value, starting the process of the video follow-up training mode in the follow-up training mode, and continuously reading the functional status flag value of the image acquisition device, and starting the process of the AI ​​follow-up training mode in the follow-up training mode when the functional status flag value of the image acquisition device is detected to be the first value. The display device 200 can integrate AI-based follow-up mode with video-based follow-up mode. In AI-based follow-up mode, if the image capture device malfunctions, it switches to video-based follow-up mode to ensure uninterrupted user training. Simultaneously, it automatically detects the image capture device's status and switches back to AI-based follow-up mode once the image capture device reconnects, thus resuming AI fitness training. This ensures seamless switching between AI-based and video-based follow-up modes in scenarios reliant on image capture devices, guaranteeing the continuity of the user's fitness training and improving the user experience. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in some embodiments of this application or in the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application; Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application; Figure 3 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application; Figure 4 A schematic diagram illustrating a fitness application interface provided in some embodiments of this application; Figure 5 A schematic diagram of a training item list page provided for some embodiments of this application; Figure 6A schematic diagram of a training mode selection page provided in some embodiments of this application; Figure 7 This application provides schematic diagrams of AI follow-up training mode scenarios for some embodiments. Figure 8 A schematic diagram of key limb points of a trainee provided for some embodiments of this application; Figure 9 This application provides schematic diagrams of video follow-up training modes in some embodiments. Figure 10 This application provides a schematic flowchart of a control method for a display device to execute a follow-up mode, as shown in some embodiments. Figure 11 Timing diagrams for a control method for a display device to execute a follow-up mode, provided in some embodiments of this application; Figure 12 This is a schematic diagram of the image acquisition service architecture provided in some embodiments of this application; Figure 13 This is a schematic diagram illustrating the effect of the mode selection interface provided in some embodiments of this application; Figure 14 This application provides a schematic diagram of the control method architecture for a display device to execute a follow-up mode, as shown in some embodiments. Figure 15 This is a schematic diagram illustrating the effect of the mode switching control provided in some embodiments of this application; Figure 16 This is a schematic diagram illustrating the window effect when switching from video follow-up mode to AI follow-up mode, as provided in some embodiments of this application. Detailed Implementation

[0011] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0012] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0013] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0014] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0015] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0016] In this embodiment, the display device 200 generally refers to a device with screen display and data processing capabilities. For example, the display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0017] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, a user can operate the display device 200 via touch operation, a mobile terminal 300, and a control device 100. The control device 100 receives user input commands and converts them into control commands that the display device 200 can recognize and respond to. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0018] The mobile terminal 300 can function as a control device for human-computer interaction between the user and the display device 200. It can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can have software applications installed on it and communicate with the display device 200 via network communication protocols to achieve one-to-one control and data communication. Furthermore, it can transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronized display.

[0019] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application that controls the display device 200.

[0020] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0021] Display device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function that provides computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0022] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0023] In some embodiments, the display device 200 may include at least one of a tuner / demodulator 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface. In some embodiments, the detector 230 is used to acquire signals from the external environment or to interact with the external environment. For example, the detector 230 may include a light receiver, a sensor for acquiring ambient light intensity; or, the detector 230 may include an image acquisition device, such as an image acquisition device, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, the detector 230 may include a sound acquisition device, such as a microphone, for receiving external sounds.

[0024] In some embodiments, the device interface 240 may include, but is not limited to, one or more interfaces such as: High Definition Multimedia Interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0025] In some embodiments, the device interface 240 can be configured to connect to an image acquisition device, and can be an interface for a built-in image acquisition device or an external image acquisition device, which can be set according to the user's actual needs.

[0026] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0027] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 with Bluetooth functionality.

[0028] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0029] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200.

[0030] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0031] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface receives user input commands through the graphical user interface (GUI).

[0032] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0033] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0034] To enable user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program used to manage and control the hardware and software resources of the display device 200. The operating system can control the display device to provide a user interface; for example, the operating system can directly control the display device to provide a user interface, or it can provide a user interface by running an application. The operating system also allows users to interact with the display device 200.

[0035] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0036] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0037] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0038] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0039] like Figure 3As shown, the application framework layer in this embodiment includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. The managers include at least one of the following modules: an activity manager for interacting with all running activities in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0040] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes to the display window, such as shrinking the display window, shaking the display, or distorting the display.

[0041] In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime library layer, such as the C / C++ instruction library, to implement the functions to be performed by the framework layer.

[0042] In some embodiments, the kernel layer is a functional layer situated between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 3 As shown, hardware drivers can be configured in the kernel layer. The drivers included in the kernel layer can be at least one of the following: audio driver, display driver, Bluetooth driver, image acquisition driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0043] It should be noted that the above examples are merely a simple division of operating system functions and do not limit the specific form of the operating system of the display device 200 in this application embodiment. Depending on the function of the display device, the type of operating system, and other factors, the number of levels and the specific level type of the operating system may be expressed in other forms.

[0044] In some application scenarios, fitness applications can be installed on the display device 200. Figure 4 These are schematic diagrams of fitness application interfaces provided in some embodiments of this application, such as... Figure 4 As shown, when the application launch command for opening a fitness application is received, the controller 250 controls the display 260 to show the fitness application's homepage. The fitness application can provide users with a variety of training programs. The fitness application's homepage includes various training program option controls 41, including but not limited to aerobics, yoga, square dancing, parent-child exercises, Tai Chi, slimming exercises, martial arts, etc. Users can select the corresponding training program according to their personal training preferences to achieve diversified training.

[0045] In some embodiments, users can register and log in to an account within a fitness app, allowing the app to record the user's historical training data. (See also...) Figure 4 Fitness apps can display user information, energy consumption, training duration, and last training program on their homepage. User information includes, but is not limited to, username and avatar, allowing users to easily understand their recent training progress.

[0046] Figure 5 This is a schematic diagram of a training item list page provided in some embodiments of this application, such as... Figure 5 As shown, users can select any training program of interest, such as training program A, on the homepage of the fitness application and click on the option control 51 corresponding to training program A. The controller 250 responds to this operation by controlling the display 260 to show the training program list page. In some embodiments, the training program list page may include sub-training programs of training program A. For example, if training program A is aerobics, sub-training programs of aerobics may include, but are not limited to, HIIT (High-Intensity Interval Training), Pamela Reif, etc. This allows users to more precisely select training programs of interest based on their preferences and abilities, such as selecting aerobics-HIIT.

[0047] The display device 200 can be equipped with fitness applications to enable smart fitness on a large screen. For example, it can mainly achieve the fitness process by playing fitness videos, which users can watch and follow along with independently. Figure 6 This is a schematic diagram of a training mode selection page provided in some embodiments of this application, such as... Figure 6 As shown, in some embodiments, after the user selects the training video they want to watch, they can click "Start Training," which will redirect them to... Figure 6 The training mode selection page shown includes training modes offered by fitness apps, including but not limited to AI-powered follow-along mode, video follow-along mode, and movie viewing mode.

[0048] Figure 7 This is a schematic diagram of an AI follow-up training mode scenario provided in some embodiments of this application, such as... Figure 7 As shown, in some embodiments, the AI ​​follow-up mode is based on the media asset type being AI, and obtains the corresponding media asset tracking file, such as an action group file, based on the media asset ID. The action group file may include the action configuration, the start time of the follow-up action, the end time of the follow-up action, a specific list of action group IDs (each ID corresponds to a specific action in the action template file), the classification of this action group for the entire follow-up video action (e.g., warm-up, action, relaxation), the internal state of this action group (e.g., warm-up (config), action (action), relaxation (relax), and the action's location in the action group's configuration phase, etc.

[0049] In the AI-assisted training mode, key body points of the trainee can be captured using an image capture device. This image capture device can be either a pluggable device or one with a software lock. A pluggable device allows insertion into or removal from the TV set; it is usable when inserted and unusable otherwise. An image capture device with a software lock is integrated into the display device 200 and cannot be removed. However, its usability can be controlled within the settings application. For example, `Camera_lock = true` indicates the image capture device is unusable, while `Camera_lock = false` indicates it is usable.

[0050] Figure 8 This is a schematic diagram of key limb points of the trainee provided in some embodiments of this application, such as... Figure 8As shown, in some embodiments, key limb points of the human skeleton can be represented by some two-dimensional coordinate information, such as including but not limited to the following representation: 0 represents the nose, with coordinates (x0, y0); 1 represents the left eye, with coordinates (x1, y1); 2 represents the right eye, with coordinates (x2, y2); 3 represents the left ear (x3, y3), with coordinates (x4, y4); 4 represents the right ear, with coordinates (x4, y4); 5 represents the left shoulder, with coordinates (x5, y5); 6 represents the right shoulder, with coordinates (x6, y6); and 7 represents the left elbow, with coordinates (x7, y7). 8 represents the right elbow (x8, y8), 9 represents the left wrist (x9, y9), 10 represents the right wrist (x10, y10), 11 represents the left hip (x11, y11), 12 represents the right hip (x12, y12), 13 represents the left knee (x13, y13), 14 represents the right knee (x14, y14), 15 represents the left ankle (x15, y5), and 16 represents the right ankle (x16, y16). Here, X represents the horizontal axis coordinates, and Y represents the vertical axis coordinates.

[0051] After capturing key limb points of the trainee using an image acquisition device, these points can be input into a motion matching algorithm. Then, by analyzing media resources such as the playback progress of a fitness video, the algorithm obtains the motion group file information for the currently playing video. This motion group file information is then input into the motion matching algorithm. The algorithm can then match the motions using the standard motion group file and the limb points identified during the training. For example, it can use frame-by-frame angle calculation and inter-frame similarity calculation to identify which parts of the body are not performing correctly and their degree of standardization, providing standardized motion prompts and scoring the user's training. Simultaneously, the training score can be synchronized to the user interface, converting it into calorie consumption information and a score.

[0052] Figure 9 This is a schematic diagram of a video follow-up mode scenario provided in some embodiments of this application, such as... Figure 9 As shown, in some embodiments, in video follow-up mode, follow-up videos, such as dance videos or fitness videos, can be downloaded from the cloud and played through a media player. However, in video follow-up mode, only video files are included; there are no follow-up movement sets, no image capture preview, and no movement matching functionality.

[0053] Understandably, when an image capture device is not present on the TV, the user will choose the video follow-up mode. When an image capture device is present on the TV, the user can choose either the video follow-up mode or the AI ​​follow-up mode. The AI ​​follow-up mode relies on the image capture device to detect body data and perform motion matching. Thus, when the user selects the AI ​​follow-up mode, if the image capture device is present and functioning properly, the TV will respond to the user's selection of the AI ​​follow-up mode.

[0054] However, in AI follow-up mode, if the image capture device malfunctions, such as being removed or experiencing a problem, the TV will directly exit AI follow-up mode, causing the mode to be interrupted. After the interruption, the TV may display a follow-up error message, thus exiting the mode and preventing the user from continuing, negatively impacting the user experience.

[0055] To address the issue of user disruption and loss of experience when AI-powered follow-up mode is interrupted during practice, this application provides a display device 200. The display device 200 includes a monitor 260, a device interface, a communication device 220, and a controller 250. The monitor 260 is configured to display a user interface, the device interface is configured to connect to an image acquisition device configured to capture images of the user's follow-up workout, the communication device 220 is configured to establish a communication connection with a server 400, and the controller 250 executes a follow-up mode control method on the display device 200 by running an application. The display device 200 integrates AI-powered follow-up mode and video-based follow-up mode. When the image acquisition device malfunctions in AI-powered follow-up mode, it switches to video-based follow-up mode to ensure uninterrupted user workout. Simultaneously, it automatically detects the image acquisition device's status and switches back to AI-powered follow-up mode after the image acquisition device reconnects, thus resuming AI fitness follow-up. This ensures seamless switching between AI-powered and video-based training modes in scenarios reliant on image acquisition devices, guaranteeing the continuity of the trainee's workout and enhancing the user experience.

[0056] To facilitate understanding of the technical solutions in some embodiments of this application, the steps are described in detail below with reference to some specific embodiments and accompanying drawings. Figure 10 This is a schematic flowchart illustrating a control method for executing a follow-up mode on a display device according to some embodiments of this application, such as... Figure 10 As shown, in some embodiments, when the display device 200 executes the control method of the follow-up mode, it may include the following steps S1-S3, the specific contents of which are as follows: Step S1: In response to the operation command for starting the AI ​​follow-up mode, the display device 200 reads the function status flag value of the image acquisition device.

[0057] Figure 11 Timing diagrams for a control method for executing a follow-up mode on a display device provided in some embodiments of this application, such as... Figure 11 As shown, in some embodiments, a user can select any training video, such as an AI-powered follow-along video, from the homepage of a fitness application. In response to an operation command to activate the AI ​​follow-along mode, the display device 200, such as a television system, can first monitor the functional status flag value of an image acquisition device. The functional status flag value includes a first value and a second value; the first value indicates that the image acquisition device is in an available state, and the second value indicates that the image acquisition device is in an unavailable state.

[0058] Before detecting the functional status of the image acquisition device, in some embodiments, after the user starts the training video and selects the AI ​​follow-up mode, the display device 200 can first detect the connection signal between the device interface and the image acquisition device. If a connection signal is detected, the image acquisition device can be started, and if the interface parameters of the image acquisition device are greater than a preset value, the follow-up mode can be switched to the AI ​​follow-up mode. If no connection signal is detected, the display can be controlled to show a mode selection interface, which includes a first control and a second control. The first control is used to enable the video follow-up mode, and the second control is used to exit the follow-up mode.

[0059] For example, Figure 12 This is a schematic diagram of the image acquisition device service architecture provided in some embodiments of this application, such as... Figure 12 As shown, taking an image capture device as an example, the display device 200 can determine whether the image capture device exists. The determination strategy can be through the built-in image capture device platform interface. For example, the interface parameter of the image capture device service can be represented as `number`, which can be obtained using `number = Camera.getNumberOfCameras()`. During the determination, if `number > 0`, it indicates that the image capture device exists. After starting the image capture device, it can be determined whether the image capture device is in an available state. Based on the obtained image capture device status, if `number > 0`, the TV system can directly jump to the AI ​​follow-up mode. When the image capture device does not exist, such as when `Number == 0`, it indicates that no image capture device is currently inserted, and the user can be presented with... Figure 13 The mode selection interface shown includes two controls, 131 and 132. When the user selects and confirms control 131, the TV system will enter the video follow-up mode. When the user selects and confirms control 132, the TV system will exit the follow-up mode.

[0060] Since image acquisition devices are divided into two types, built-in and external, taking an image acquisition device as an example, an image acquisition device can be a pluggable image acquisition device or an image acquisition device with a software lock. Therefore, for different image acquisition devices, different listening methods can be adopted when reading the functional status of the image acquisition device when it is present.

[0061] For a pluggable image acquisition device, i.e. an external image acquisition device, when reading the functional status flag value of the image acquisition device, the display device 200 can first register a listening service with the image acquisition device. In response to the detection of a hot-plug signal from the device interface 240, it can generate acquisition device return status information, then parse the acquisition device return status information to obtain the acquisition device identification code; and then, based on the acquisition device identification code, read the functional status flag value of the image acquisition device according to the listening service.

[0062] For example, you can register an image capture insertion listener with the image capture service to determine whether an image capture device is available. For instance, you can listen for image capture device availability using the platform interface `CameraManager.registerAvaibilityCallback(CameraCallback)`. When the image capture server detects an image capture device insertion, it can obtain the return status from `CameraCallback`, thus obtaining the ID of the available image capture device from `CameraCallback`.

[0063] For an image acquisition device with a software lock, i.e., a built-in image acquisition device, when reading the function status flag value of the image acquisition device, the display device 200 can first obtain the software lock identifier of the image acquisition device. The software lock identifier includes a first identifier and a second identifier. Then, based on the software lock identifier, the display device listens to the function status flag value of the image acquisition device. When the software lock identifier is the first identifier, the function status flag value can be determined to be the first value; when the software lock identifier is the second identifier, the function status flag value can be determined to be the second value.

[0064] For example, for an image acquisition device with a software lock, its built-in software lock mechanism can be used to control the functional status flag value of the image acquisition device. When the display device 200 performs the function of acquiring the image acquisition device, it can first acquire the software lock identifier and determine the functional status flag value of the image acquisition device based on the identifier. For example, the first identifier can represent that the image acquisition device is in an unlocked state, i.e., an available state, in which the image acquisition device can work normally and acquire images. The second identifier can represent that the image acquisition device is in a locked state, i.e., an unavailable state, in which the image acquisition device cannot acquire images or videos. The specific representation of the identifier can be set according to actual needs, and this application does not make specific limitations on it.

[0065] After the functional status monitoring is completed, the monitoring results can be fed back to the TV system through the monitoring service or the built-in software lock. This way, by reading the functional status flag value of the image acquisition device, the display device 200 can ensure that the user understands the device status before starting training, avoiding disruptions to the training experience due to device problems. Simultaneously, the image acquisition device status can be fed back in real time, allowing the user to understand whether AI training can begin or whether further action is needed, such as checking the image acquisition device connection. After step S1 is completed, step S2 can be included.

[0066] Step S2: If the function status flag value is the first value, the display device 200 will start the AI ​​follow-up mode process in the follow-up mode; obtain the action group file corresponding to the training project media and the action file corresponding to the follow-up image, and determine the follow-up score according to the action group file and the follow-up action file, and control the display 260 to display the follow-up score on the follow-up interface.

[0067] Continue to combine Figure 11 When the image acquisition device's function status flag is set to the first value, the display device 200 can record the user's follow-up movements and read the corresponding action group files from the training program video. These files contain detailed descriptions and execution requirements of the standard movements. Simultaneously, the display device 200 can analyze the user's follow-up movements in real time and compare them with the standard movements in the action group files. Based on the comparison results, the display device 200 can calculate the user's follow-up score, which can be displayed in a preset window. The window's position and size can be adjusted according to user preferences. Afterward, the follow-up score can be transmitted to a fitness application via a television system, where it can be displayed. Users can view their scores to understand the training effect and adjust their training strategies as needed.

[0068] Figure 14 This is a schematic diagram of the control method architecture for executing a follow-up mode on a display device provided in some embodiments of this application, such as... Figure 14 As shown, training videos can be obtained from a cloud server. The display device 200 can be equipped with an Application Programming Interface (API). Through the API, the display device 200 can integrate playback control for the training video follow-up mode, image acquisition device management, calculation of follow-up scores using motion matching algorithms, and acquisition of motion group files. The playback control for the follow-up mode can include both video follow-up mode and AI follow-up mode. Simultaneously, the display device 200 can monitor the presence or functional status of the image acquisition device through an image acquisition device service or software lock. Figure 14The architecture shown enables control of the follow-up training mode. It is understandable that in AI follow-up training mode, when the image acquisition device is absent, the display device 200 will display something like... Figure 13 The interface shows the mode selection screen, and the user can choose to enter or exit the video practice mode.

[0069] To meet users' personalized needs and enable flexible switching between practice modes, in some embodiments, the display device 200 can provide users with a mode switching control. Therefore, when the practice mode is AI practice mode, the display device 200 can detect user click events based on the mode switching control, which is used to switch the user's practice mode; in response to the user's click event on the mode switching control, the process of the AI ​​practice mode is closed, and the process of the video practice mode is started.

[0070] For example, Figure 15 This is a schematic diagram illustrating the effect of the mode switching control provided in some embodiments of this application, such as... Figure 15 As shown, in some embodiments, when the current training mode is determined to be AI training mode, the display device 200 can listen for user click events on the mode switching control. For example, this can be achieved through an event listener, capturing user operation events and converting them into corresponding program responses. When the system detects that the user clicks the mode switching control, it can perform a series of operations to switch from AI training mode to video training mode. For example, it can stop AI-related data processing, adjust the user interface to reflect the mode change, and reset training parameters. In some use cases, although the user is initially in AI training mode, the user may not want their own image or video to appear on the interface. In this case, switching modes allows the user to choose the most suitable training method according to their preferences, needs, or device capabilities. It should be noted that in ordinary training scenarios, if the image acquisition device is available, the user can also switch from video training mode to AI training mode through the mode switching control, and the user can complete the switch according to their needs.

[0071] In order to accurately determine the user's current training stage, in some embodiments, before the display device 200 determines the training score based on the action group file corresponding to the training project video and the user's training action file, the display device 200 can first acquire the training image through the image acquisition device after listening to the change of the image acquisition device's functional status from unavailable to available. That is, the image acquisition device is activated. After acquiring the training image, the current playback progress of the training project video can be obtained, and then the action group file is updated according to the current playback progress.

[0072] For example, to ensure the image acquisition device functions correctly when needed, the television system can automatically activate the image acquisition device after the display device 200 detects a change in its functional status from unavailable to available. After acquiring the preview image captured by the image acquisition device, the current playback progress of the training video can be obtained to synchronize the real-time image captured by the image acquisition device with the playback progress of the training video. By obtaining the current playback progress, it can be ensured that the images captured by the image acquisition device are consistent with the actions in the training video, thereby improving the accuracy of action recognition. Subsequently, the action group file is updated according to the current playback progress to dynamically adjust the action sequences in the action group file based on the playback progress of the training video. By updating the action group file in real time, it can be ensured that the system can accurately identify and analyze the user's action performance during training, determine the user's current training stage, and thus provide the user with more precise exercise guidance and feedback.

[0073] In actual practice sessions, participants typically take a break after practicing one or several movements. Therefore, a practice session includes both a practice phase and a relaxation phase. Understandably, the practice phase involves scoring, while the relaxation phase does not. Therefore, it is necessary to distinguish between the practice and relaxation phases within a practice session.

[0074] To distinguish between the practice and relaxation phases, in some embodiments, after updating the motion group file according to the current playback progress, the display device 200 can execute the following process: First, the display device 200 can obtain multiple motion group files contained in the motion group file. Then, based on the current playback progress, it determines the target motion group file within the motion group file to which the user's practice action belongs. Next, based on the target motion group file, it determines the motion phase to which the user's practice action belongs. Motion phases include practice and relaxation phases. When the motion phase is a practice phase, it executes the step of determining the practice score based on the motion group file and the practice motion file.

[0075] For example, assuming the current video playback progress is `media_progress`, and the action group file corresponding to the training video is `actionGroupList`, and the entire video is divided into multiple action groups, the action group information for follow-along training can be denoted as `actionGroupList[Current_index]`, where `Current_index` represents one of the action groups. When querying the current video playback position using `media_progress` to determine which action group it belongs to in the action group list, a loop-based judgment method can be used. For example, it can query which action group the time corresponding to `media_progress` falls within, and then determine the segment it belongs to based on the time. By obtaining the action group file, determining the playback position, locating the target action group file, and distinguishing action segments, the display device 200 can accurately identify the user's current training actions, thereby distinguishing between follow-along training segments and relaxation segments.

[0076] In order to calculate the follow-up score obtained by the user, in some embodiments, the display device 200 can first obtain the target action group file in the action group file, then identify multiple follow-up group files in the follow-up action file, then determine the target follow-up group file corresponding to the current playback progress based on the target action group file and the follow-up action file, then compare the target action group file and the target follow-up group file to obtain the comparison result, and determine the follow-up score based on the comparison result.

[0077] For example, the display device 200 first acquires a target motion group file from the motion group file. The target motion group file may contain a preset sequence of fitness movements. When the user is following along, the display device 200 can record their actual movements and generate a follow-up motion file. The follow-up motion file may contain detailed data for each movement performed by the user. The display device 200 can match the corresponding target follow-up motion file from the target motion group file and compare it with the movements actually performed by the user. During the comparison, the motion standards in the target motion group file can be compared with the follow-up movements actually performed by the user. Based on the comparison results, the user's follow-up score can be calculated. For example, the follow-up score can be calculated using a motion matching algorithm or other methods; this application does not specifically limit this. After step S2 is completed, step S3 may be included.

[0078] Step S3: If the function status flag value is the second value, the display device 200 will start the process of the video follow-up mode in the follow-up mode, and continuously read the function status flag value of the image acquisition device. When the function status flag value of the image acquisition device is detected to be the first value, the process of the AI ​​follow-up mode in the follow-up mode will be started.

[0079] Continue to combine Figure 11 When the function status flag of the image acquisition device is the second value, the display device 200 can start the process of the video follow-up mode in the follow-up mode, and continuously read the function status flag value of the image acquisition device. When the function status flag of the image acquisition device is detected to be the first value, the process of the AI ​​follow-up mode in the follow-up mode will be started.

[0080] If the image acquisition device is initially unavailable, the display device can automatically start the video follow-up mode process in the follow-up mode, which only plays the training video and does not provide AI-based motion analysis and scoring feedback. Simultaneously, the TV system can continuously read the image acquisition device's functional status marker values. For example, it can periodically send status check requests to monitor whether it has returned to normal, so that it can switch back to AI follow-up mode in a timely manner when the image acquisition device returns to normal. In other words, after detecting that the image acquisition device's functional status has become available, the TV system will automatically and seamlessly switch the follow-up mode back to AI follow-up mode without manual user intervention. This ensures the continuity of user follow-up, preventing training interruptions even when the image acquisition device malfunctions. Automated mode switching reduces the complexity of user operations, improves the smoothness of the user experience, and solves the problem of users being unable to continue following up when the AI ​​follow-up mode is interrupted, thus affecting the user experience.

[0081] In order to synchronously display training program videos and user follow-up images, in some embodiments, after the display device 200 starts the AI ​​follow-up mode process in the follow-up mode, it can create a first window and a second window in the follow-up interface, and control the display 260 to display the training program video in the first window of the follow-up interface, and control the display 260 to display the follow-up image in the second window of the follow-up interface.

[0082] For example, the display device 200 can first acquire a training video and then play the video in the first window of the follow-up interface. Simultaneously, the display device 200 can capture the user's follow-up movements using an image acquisition device or other sensors, generate a follow-up image, and display the image in the second window of the follow-up interface. In this way, the user can see their own follow-up movements while watching the training video, thus achieving synchronized display.

[0083] Figure 16 This is a schematic diagram illustrating the window effect of switching from video follow-up mode to AI follow-up mode in some embodiments of this application, such as... Figure 16As shown, in video follow-up mode, the display device 200 can play the training video in the entire window, allowing users to follow along. When switching to AI follow-up mode, the display device 200 can present the training video in a floating window, with the image capture camera providing a full-screen preview. For example, the preview ratio is 1920*1080. Depending on the scenario, the window display ratio of the training video can be set to 640*360. It is understood that the above dimensions are for illustrative purposes only and can be set according to actual needs; this application does not impose specific limitations. The display device 200 can also present two windows side-by-side. Similarly, the size and position of the windows can be set according to actual needs. In this way, users can see both the training video and their own follow-up video, satisfying their follow-up needs.

[0084] As can be seen from the above technical solutions, the above embodiments provide a display device 200. In response to an operation command for starting an AI follow-up training mode, the display device reads the functional status flag value of an image acquisition device. The functional status flag value includes a first value and a second value. The first value is used to indicate that the image acquisition device is in an available state, and the second value is used to indicate that the image acquisition device is in an unavailable state. If the functional status flag value is the first value, the process of the AI ​​follow-up training mode in the follow-up training mode will be started. The action group file corresponding to the training project media asset and the action file corresponding to the follow-up image will be obtained, and the follow-up score will be determined according to the action group file and the follow-up action file. The follow-up score will be displayed on the follow-up interface. If the functional status flag value is the second value, the process of the video follow-up training mode in the follow-up training mode will be started. The functional status flag value of the image acquisition device will be continuously read. When the functional status flag value of the image acquisition device is detected to be the first value, the process of the AI ​​follow-up training mode in the follow-up training mode will be started. The display device 200 can integrate AI-based follow-up mode with video-based follow-up mode. In AI-based follow-up mode, if the image capture device malfunctions, it switches to video-based follow-up mode to ensure uninterrupted user training. Simultaneously, it automatically detects the image capture device's status and switches back to AI-based follow-up mode once the image capture device reconnects, thus resuming AI fitness training. This ensures seamless switching between AI-based and video-based follow-up modes in scenarios reliant on image capture devices, guaranteeing the continuity of the user's fitness training and improving the user experience.

[0085] Based on the display device 200 described above, some embodiments of this application also provide a control method for a follow-up mode, which can be applied to the display device 200 in the above embodiments. In some embodiments, the method may include the following: In response to an operation command for activating the AI ​​follow-up mode, the function status flag value of the image acquisition device is read. The function status flag value includes a first value and a second value. The first value is used to indicate that the image acquisition device is in an available state, and the second value is used to indicate that the image acquisition device is in an unavailable state. If the function status flag value is the first value, the AI ​​follow-up mode process in the follow-up mode will be started; the action group file corresponding to the training project media asset and the action file corresponding to the follow-up image will be obtained, and the follow-up score will be determined according to the action group file and the follow-up action file; and the follow-up score will be displayed on the follow-up interface. If the function status flag value is the second value, the process of the video follow-up mode in the follow-up mode will be started, and the function status flag value of the image acquisition device will be continuously read. When the function status flag value of the image acquisition device is detected to be the first value, the process of the AI ​​follow-up mode in the follow-up mode will be started. The video follow-up mode is a mode that does not support the acquisition of follow-up images.

[0086] As can be seen from the above technical solutions, the above embodiments provide a control method for a follow-up training mode. This method can integrate AI follow-up training mode and video follow-up training mode. In AI follow-up training mode, when the image capture device malfunctions, the AI ​​follow-up training mode is switched to video follow-up training mode to ensure uninterrupted user training. Simultaneously, the image capture device status is automatically detected, and after the image capture device reconnects, the system switches back to AI follow-up training mode to resume AI fitness training. This ensures seamless switching between AI follow-up training mode and video follow-up training mode in scenarios relying on image capture devices, guaranteeing the continuity of the trainee's fitness training and improving the user experience.

[0087] The same or similar parts among the various embodiments in this specification can be referred to mutually, and will not be repeated here.

[0088] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or certain parts of the embodiments of the present invention.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0090] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that, include: The monitor is configured to display the practice interface; The device interface is configured to connect to an image acquisition device, which is configured to acquire images of the user's practice sessions. A communication device configured to establish a communication connection with a server; The controller is configured as follows: In response to an operation command for activating the AI ​​follow-up mode, the function status flag value of the image acquisition device is read. The function status flag value includes a first value and a second value. The first value is used to indicate that the image acquisition device is in an available state, and the second value is used to indicate that the image acquisition device is in an unavailable state. If the function status flag value is the first value, start the process of AI follow-up mode in the follow-up mode; Obtain the action group file corresponding to the training project media asset and the action file corresponding to the follow-up image, determine the follow-up score based on the action group file and the follow-up action file, and control the display to show the follow-up score on the follow-up interface. If the function status flag value is the second value, start the process of the video follow-up mode in the follow-up mode, and continuously read the function status flag value of the image acquisition device. When the function status flag value of the image acquisition device is detected to be the first value, start the process of the AI ​​follow-up mode in the follow-up mode. The video follow-up mode is a mode that does not support the capture of follow-up images.

2. The display device according to claim 1, characterized in that, After responding to the operation command for initiating the AI ​​follow-up mode, the controller is further configured to: Detect the connection signal between the device interface and the image acquisition unit; When the connection signal is detected, the image acquisition device is started, and when the interface parameter of the image acquisition device is greater than a preset value, the follow-up mode is switched to AI follow-up mode. When the absence of the connection signal is detected, the display is controlled to show a mode selection interface. The mode selection interface includes a first control and a second control. The first control is used to enable the video follow-up mode, and the second control is used to exit the follow-up mode.

3. The display device according to claim 1, characterized in that, The controller is configured to read the function status flag value of the image acquisition device, specifically as follows: Register a listening service with the image acquisition device; In response to the detection of a hot-plug signal at the device interface, a collector return status information is generated; The collector's returned status information is parsed to obtain the collector's identification code; Based on the collector identification code, the functional status flag value of the image collector is read by the monitoring service.

4. The display device according to claim 1, characterized in that, The controller is configured to read the function status flag value of the image acquisition device, specifically as follows: Obtain the software lock identifier of the image acquisition device; the software lock identifier includes a first identifier and a second identifier; The functional status flag value of the image acquisition device is read based on the software lock identifier; When the software lock identifier is the first identifier, the function status flag value is determined to be the first value; When the software lock identifier is the second identifier, the function status flag value is determined to be the second value.

5. The display device according to claim 1, characterized in that, The controller is also configured to: When the training mode is the AI ​​training mode, the user's click event based on the mode switching control is detected; the mode switching control is used to switch the user's training mode. In response to the user's click event on the mode switching control, the process of the AI ​​follow-up mode is closed, and the process of the video follow-up mode is started.

6. The display device according to claim 1, characterized in that, Before the controller executes the step of determining the follow-up score based on the action group file corresponding to the training project video and the user's follow-up action file, it is also configured to: After the image acquisition device's functional status changes from the unavailable state to the available state, the training image is acquired through the image acquisition device. After obtaining the training images, obtain the current playback progress of the training project video; Update the action group file according to the current playback progress.

7. The display device according to claim 6, characterized in that, After executing the step of updating the action group file according to the current playback progress, the controller is further configured to: Obtain multiple action group files contained in the action group file; Based on the current playback progress, determine the target action group file to which the current user's practice action belongs within the action group file; The action segment to which the user's follow-up action belongs is determined based on the target action grouping file. The action segment includes a follow-up action segment and a relaxation segment. When the action segment is the follow-up exercise segment, the step of determining the follow-up exercise score based on the action group file and the follow-up exercise action file is performed.

8. The display device according to claim 7, characterized in that, The controller determines the follow-up score based on the action group file corresponding to the training project video and the user's follow-up action file, specifically configured as follows: Obtain the target action group file from the action group file; Identify multiple follow-up group files within the follow-up action file; Determine the target follow-up group file corresponding to the current playback progress based on the target action group file and the follow-up action file; The target action group file and the target follow-up training group file are compared to obtain the comparison results; The practice score is determined based on the comparison results.

9. The display device according to claim 1, characterized in that, After executing the step of initiating the AI ​​follow-up mode in the follow-up mode, the controller is further configured to: Create a first window and a second window in the training interface; The control display shows the training program video in the first window of the training interface, and the control display shows the training image in the second window of the training interface.

10. A control method for a follow-up mode, applied to the display device according to any one of claims 1-9, the display device comprising a display, a communication device, and a controller, characterized in that, The method includes: In response to an operation command for activating the AI ​​follow-up mode, the function status flag value of the image acquisition device is read. The function status flag value includes a first value and a second value. The first value is used to indicate that the image acquisition device is in an available state, and the second value is used to indicate that the image acquisition device is in an unavailable state. If the function status flag value is the first value, start the AI ​​follow-up mode process in the follow-up mode; obtain the action group file corresponding to the training project media asset and the action file corresponding to the follow-up image, determine the follow-up score according to the action group file and the follow-up action file, and control the display to display the follow-up score on the follow-up interface. If the function status flag value is the second value, the process of the video follow-up mode in the follow-up mode will be started, and the function status flag value of the image acquisition device will be continuously read. When the function status flag value of the image acquisition device is detected to be the first value, the process of the AI ​​follow-up mode in the follow-up mode will be started; the video follow-up mode is a mode that does not support the acquisition of follow-up images.

Images