Display device and fitness video playback method

Abstract

Some embodiments of the present application show a display device and a fitness video playing method, the method comprising: in response to an instruction of playing a fitness video, downloading media data and a log file of the fitness video, the log file comprising body key point information and a corresponding time of the body key point information; controlling the display to display the media data; determining a target range according to a playing progress of the media data, obtaining target body key point information in the target range; generating a motion prompt character image according to the target body key point information; and controlling the display to display the motion prompt character image. By analyzing the log file corresponding to the fitness video, the body key point information in the fitness video is extracted, and the motion prompt character is generated through the body key point information, so that the motion can be predicted, the user can prepare in advance, the motion accuracy is improved, and the user experience is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of display devices, in particular to a display device and a fitness video playing method. BACKGROUND

[0002] Current large-screen fitness applications mainly include two categories: one is ordinary fitness, only playing fitness videos, and people follow the videos to perform actions; the other is AI (Artificial Intelligence) fitness, a fitness video generates a log dot file in advance, and in the process of playing the fitness video, a device camera is started to shoot personal actions, and the personal action limb key points are matched with the action key point information in the log dot file in the time period to identify the action accuracy.

[0003] The log dot file in the fitness video is a file recording the action information in the fitness video, including the action name, action start and end time, action sampling limb key point position information, etc. contained in the video. There are multiple full-body limb point position information in a time period according to the sampling frequency. The log dot file contains one or more action information, which is generated in advance according to the video and corresponds to the fitness video, and is mainly used to detect the compliance of the follow-up action and the video action, and is used for accuracy scoring and calorie consumption, etc.

[0004] Current AI follow-up is also following the video action for follow-up. Since the video is continuously played, the fitness video action information cannot be obtained in advance, and only passive following is available, with poor action delay and accuracy. SUMMARY

[0005] Some embodiments of the present application provide a display device and a fitness video playing method. By analyzing the log dot file corresponding to the fitness video, the limb key point information in the fitness video is extracted, the action prompt character is generated through the limb key point information, the action can be predicted, the user can prepare in advance, the action accuracy is improved, and the user experience is improved.

[0006] In a first aspect, some embodiments of the present application provide a display device, comprising:

[0007] a display;

[0008] a controller configured to:

[0009] in response to an instruction to play a fitness video, download media data of the fitness video and a log dot file, the log dot file including limb key point information and a time corresponding to the limb key point information;

[0010] control the display to display the media data;

[0011] Determine a target range according to the playback progress of the media asset data, and obtain target limb key point information within the target range;

[0012] Generate an action prompt character image based on the target limb key point information;

[0013] The display is controlled to display the action prompt character image on the media data, and the action prompt character image is displayed in a time arrangement corresponding to the target limb key point information.

[0014] In some embodiments, the controller is further configured to determine a target range according to the playback progress of the media asset data and obtain target limb key point information within the target range:

[0015] If the log dot file includes a mark point, determining a target range, the mark point being the time corresponding to the mark action, the target range starting at a first time point and ending at a second time point, the difference between the playback progress and the first time point being a first preset value, the first preset value being greater than or equal to 0, and the difference between the second time point and the playback progress being a second preset value, the second preset value being greater than 0;

[0016] Obtain limb key point information of landmark points within the target range.

[0017] In some embodiments, the controller is further configured to determine a target range according to the playback progress of the media asset data and obtain target limb key point information within the target range:

[0018] If the log dot file does not include a marker point, determining a fixed point, wherein the fixed point is a time point at which a preset time interval is set based on a preset progress of the media asset data;

[0019] Determine a target range, where the target range starts at a first time point and ends at a second time point, a difference between the playback progress and the first time point is a first preset value, the first preset value is greater than or equal to 0, and a difference between the second time point and the playback progress is a second preset value, the second preset value is greater than 0;

[0020] Obtain limb key point information of fixed points within the target range.

[0021] In some embodiments, the controller is further configured to determine a target range according to the playback progress of the media asset data and obtain target limb key point information within the target range:

[0022] If the log dot file includes a mark point, determining the mark point corresponding to the playback progress as the target point, the mark point being the time corresponding to the mark action;

[0023] determining a target range, the target range including a target point, a first preset number of landmark points before the target point, and a second preset number of landmark points after the target point;

[0024] obtaining body key point information of the landmark points in the target range.

[0025] In some embodiments, the controller is further configured to determine the target range according to a play progress of the media data, and obtain the target body key point information in the target range, and the controller is further configured to:

[0026] if the landmark point is not included in the log file, determining a fixed point, the fixed point being a time point every interval of a preset time length based on a preset progress of the media data;

[0027] determining the fixed point corresponding to the play progress as the target point;

[0028] determining a target range, the target range including a target point, a first preset number of fixed points before the target point, and a second preset number of fixed points after the target point;

[0029] obtaining body key point information of the fixed points in the target range.

[0030] In some embodiments, the controller is configured to:

[0031] determining a display position of the action prompt character image according to a time corresponding to the target body key point information and the play progress;

[0032] controlling the display to display the action prompt character image according to the display position.

[0033] In some embodiments, the controller is configured to:

[0034] determining a display color or transparency of the action prompt character image according to a time corresponding to the target body key point information and the play progress;

[0035] controlling the display to display the action prompt character image according to the color or transparency.

[0036] In some embodiments, the controller is configured to:

[0037] determining a zooming ratio of the action prompt character image according to a time corresponding to the target body key point information and the play progress;

[0038] controlling the display to display the action prompt character image according to the zooming ratio.

[0039] In some embodiments, the controller generates the action prompt character image according to the target body key point information, and is further configured to:

[0040] Obtain information of a basic body key point in the target body key point information;

[0041] Draw points and lines based on the information of the basic body key point to generate the action prompt character image.

[0042] In a second aspect, some embodiments of the present application provide a fitness video playing method, including:

[0043] In response to an instruction to play a fitness video, downloading media data and a log point file of the fitness video, the log point file including body key point information and time corresponding to the body key point information;

[0044] Controlling the display to display the media data;

[0045] Determining a target range according to a playing progress of the media data, and obtaining target body key point information in the target range;

[0046] Generating an action prompt character image according to the target body key point information;

[0047] Controlling the display to display the action prompt character image on the media data, the action prompt character image being displayed in a time corresponding to the target body key point information.

[0048] Some embodiments of the present application provide a display device and a fitness video playing method, the method downloading a log point file synchronously with media data of a fitness video, obtaining body key point information after a current playing progress in the log point file, and generating an action prompt character image based on the target body key point information, displaying the action prompt character image while playing the media data, which can predict subsequent actions, facilitate users to prepare in advance, improve action accuracy, and improve user experience. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 An operation scenario between a display device and a control device according to some embodiments is shown;

[0050] Figure 2 A hardware configuration block diagram of a control device according to some embodiments is shown;

[0051] Figure 3 A hardware configuration block diagram of a display device according to some embodiments is shown;

[0052] Figure 4A diagram showing a software configuration in a display device according to some embodiments is shown;

[0053] Figure 5 A flowchart showing steps performed by a controller according to some embodiments is shown;

[0054] Figure 6 A diagram showing a human body limb key point according to some embodiments is shown;

[0055] Figure 7 A diagram showing a first method of determining a target range according to some embodiments is shown;

[0056] Figure 8 A diagram showing a second method of determining a target range according to some embodiments is shown;

[0057] Figure 9 A diagram showing a third method of determining a target range according to some embodiments is shown;

[0058] Figure 10 A diagram showing a fourth method of determining a target range according to some embodiments is shown;

[0059] Figure 11 A diagram showing a workout video playing user interface according to some embodiments is shown;

[0060] Figure 12 A diagram showing another workout video playing user interface according to some embodiments is shown;

[0061] Figure 13 A diagram showing yet another workout video playing user interface according to some embodiments is shown. DETAILED DESCRIPTION

[0062] For the purpose of making the objects and embodiments of the present application more clear, the following will describe the exemplary embodiments of the present application in detail with reference to the accompanying drawings. Obviously, the described exemplary embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application.

[0063] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the following described embodiments, and is not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0064] The terms "first", "second", and "third" and the like in the description and in the claims of the present application and above drawings are used for distinguishing between similar or identical objects and do not necessarily have a specific temporal or chronological order unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

[0065] The term "comprising", used in the description and in the claims of the present application, is intended to indicate a non-exclusive inclusion such that a product or device that comprises a list of components does not necessarily comprise only those components in the list and can include additional components not expressly listed or inherent to such product or device.

[0066] The display device provided by the embodiments of the present application can have various implementation forms, for example, can be a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figure 1 And Figure 2 As a specific embodiment of the display device of the present application.

[0067] Figure 1 As a specific embodiment of the display device of the present application. Figure 1 As shown in FIG. 1, a user can operate the display device 200 through the smart device 300 or the control device 100.

[0068] In some embodiments, the control device 100 can be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-distance communication modes, to control the display device 200 through wireless or wired mode. The user can input user instructions through the keys on the remote controller, voice input, control panel input, etc., to control the display device 200.

[0069] In some embodiments, the smart device 300 (such as a mobile terminal, a tablet computer, a computer, a notebook computer, etc.) can also be used to control the display device 200. For example, an application running on the smart device is used to control the display device 200.

[0070] In some embodiments, the display device can not receive instructions using the above-mentioned smart device or control device, but can receive user control through touch or gesture, etc.

[0071] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300, for example, the display device 200 can directly receive the user's voice instruction control through the voice instruction receiving module configured inside the display device 200, or the display device 200 can receive the user's voice instruction control through the voice control device set outside the display device 200.

[0072] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 can be communicatively connected through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 can provide various content and interactions to the display device 200. The server 400 can be a cluster or multiple clusters, and can include one or more types of servers.

[0073] Figure 2 An exemplary configuration block diagram of the control device 100 according to exemplary embodiments is shown. As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, a power supply. The control device 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the display device 200 can recognize and respond to, acting as an intermediary between the user and the display device 200. Figure 2

[0074] Figure 3 As shown, the display device 200 includes a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and at least one of a user interface.

[0075] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an n-th interface for input / output.

[0076] The display 260 includes a display screen component for presenting a picture, and a driving component for driving image display, a component for receiving an image signal originating from the controller output, and displaying video content, image content, and a menu control interface, and a user control UI interface.

[0077] The display 260 can be a liquid crystal display, an OLED display, and a projection display, and can also be a projection device and a projection screen.

[0078] The display 260 also includes a touch screen, which is used to receive user finger sliding or clicking actions on the touch screen to input control instructions.

[0079] ​​The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example, the communicator can include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0080] The user interface can be used to receive control signals of the control device 100 (such as an infrared remote controller, etc.).

[0081] The detector 230 is used to collect signals of the external environment or interaction with the outside. For example, the detector 230 includes a light receiver for collecting ambient light intensity, or the detector 230 includes an image collector such as a camera, which can be used to collect external environmental scenes, user attributes or user interaction gestures, or the detector 230 includes a sound collector such as a microphone, etc., for receiving external sounds.

[0082] The external device interface 240 can include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or data high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It can also be a composite input / output interface formed by the above multiple interfaces.

[0083] The tuner demodulator 210 receives broadcast television signals through wired or wireless reception, and demodulates audio and video signals and EPG data signals from multiple wireless or wired broadcast television signals.

[0084] In some embodiments, the controller 250 and the tuner demodulator 210 can be located in different split devices, that is, the tuner demodulator 210 can also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0085] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored on the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object displayed on the display 260, the controller 250 can perform an operation related to the object selected by the user command.

[0086] In some embodiments, the controller includes at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM (Random Access Memory), a ROM (Read-Only Memory), a first interface to an n-th interface for input / output, a communication bus, and the like.

[0087] The user can input a user command through a Graphical User Interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0088] A "user interface" is a medium interface for interaction and information exchange between an application or an operating system and a user, which realizes conversion between an internal form of information and a form acceptable by the user. A commonly used form of the user interface is a Graphic User Interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. The user interface can be an icon, a window, a control, and the like interface elements displayed in a display screen of an electronic device, wherein the control can include an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, and the like visible interface elements.

[0089] Referring to Figure 4 In some embodiments, the system is divided into four layers from top to bottom, namely, an Applications layer (referred to as "application layer" for short), an Application Framework layer (referred to as "framework layer" for short), an Android runtime and system library layer (referred to as "system runtime library layer" for short), and a kernel layer.

[0090] In some embodiments, at least one application program is run in the application layer, which can be a window (Window) program, a system setting program, or a clock program, and the like provided by the operating system; or can be an application program developed by a third-party developer. In a specific implementation, the application program package in the application layer is not limited to the above examples.

[0091] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some pre-defined functions. The application framework layer is equivalent to a processing center that decides which application in the application layer to act. The application can access resources in the system and obtain services of the system through the API interface in the execution.

[0092] As shown in Figure 4 the application embodiment, the application framework layer includes managers (Managers) and content providers (Content Provider), wherein the managers include at least one of the following modules: an activity manager (ActivityManager) for interacting with all activities running in the system; a location manager (Location Manager) for providing access to system location services for system services or applications; a package manager (Package Manager) for retrieving various information related to application packages currently installed on the device; a notification manager (NotificationManager) for controlling the display and clearing of notification messages; and a window manager (Window Manager) for managing icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0093] In some embodiments, the activity manager is used to manage the life cycle of each application and the general navigation back function, such as controlling the exit, opening, back, etc. of the application. The window manager is used to manage all window programs, such as obtaining the size of the display screen, determining whether there is a status bar, locking the screen, intercepting the screen, controlling the display window change (such as reducing the display window, shaking the display, twisting the display, etc.), etc.

[0094] In some embodiments, the system runtime library layer provides support for the upper layer, i.e. the framework layer. When the framework layer is used, the Android operating system runs the C / C++ library contained in the system runtime library layer to realize the functions of the framework layer.

[0095] In some embodiments, the kernel layer is a layer between hardware and software. As shown in Figure 4 the kernel layer contains at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a WIFI driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power supply driver, etc.

[0096] The log file in the fitness video records the action information in the fitness video, and is used to represent the action name, the start and end time of the action, the position information of the action sampling body key points, and the like. According to the sampling frequency, there are multiple point position information of the body key points in an action time period. The log file contains one or more action information, is generated in advance according to the video, and corresponds to the fitness video. The log file is mainly used to detect the fitness action and the video action, and is used for accuracy scoring and calorie consumption. AI fitness is to follow the video action for fitness. Since the video is continuously played, the action information of the fitness video cannot be obtained in advance, and can only be followed passively. The action delay and accuracy are poor.

[0097] To solve the above technical problems, an embodiment of the present application provides a display device 200. The structure and functions of each part of the display device 200 can be referred to the above embodiments. In addition, based on the display device 200 shown in the above embodiments, some functions of the display device 200 are further improved in this embodiment. As shown in Figure 5 The controller 250 performs the following steps:

[0098] Step S501: receiving a user input instruction to play a fitness video;

[0099] In some embodiments, the user can input the instruction to play the fitness video by opening an application and selecting a control of the fitness video in the interface of the application.

[0100] Step S502: in response to the instruction to play the fitness video, downloading media data and a log file of the fitness video;

[0101] The log file includes body key point information and time corresponding to the body key point information.

[0102] In response to the user's instruction to play the fitness video, the user selects the control corresponding to the media identifier, and sends the media identifier to the server. The display device can download the media data and the log file corresponding to the media identifier from the server.

[0103] The generation steps of the log file are as follows:

[0104] Capture video frames in the fitness video at a certain sampling frequency;

[0105] Detect the body action of the character in the video frame to generate body key point information;

[0106] The body key point information includes position coordinate information of 17 or 19 points of the human body. For example, taking 17 points of the human body as an example, as shown in Figure 6As shown, the 17 points of the human body include: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left ankle, and right ankle.

[0107] The continuously collected limb key point information generates an action log dotting file.

[0108] The fitness video type includes action videos and dance videos.

[0109] The action videos contain multiple fitness actions, and one fitness action is a relatively independent and complete process, such as bench press, push-up, pull-up, etc., and has high repeatability and certain regularity. All action-based video dotting files are divided into two types of files. One type is an action library index file containing all action information. The action library index file contains all action information, such as the IDs, names, and multiple action limb key point position information (including action limb range box coordinate information) of 150 actions. The other type is a video dotting file, which corresponds to a specific fitness video one-to-one. The video dotting file contains the IDs, start and end time information of multiple different actions in the corresponding fitness video, and the limb action position information of the corresponding action in the action library index file is matched and found through the action ID. Therefore, in the log dotting file production process, the limb key point information of a certain marker action can be extracted and marked for each type of action. The limb key point information of the marker action includes the name and start and end time of the marker action.

[0110] The dance fitness video is a continuous action process as a whole, and there is no obvious multiple actions. The dance dotting log file does not have action information, but only has limb key frame information composed according to a certain time interval, and the limb key frame information contains key frame time and key limb point coordinates. Therefore, in the log dotting file production process, the limb key point information does not need to be marked, or marked every interval of a preset time length.

[0111] The generation of the log dotting file can be completed by a computer or other terminal, and the generated log dotting file is uploaded to the server in correspondence with the media data of the fitness video. When a user needs to play, the log dotting file can be downloaded from the server.

[0112] In some embodiments, the limb key point information of the marker action can refer to the limb key point information of an action. For example, the start and end time of a certain set of actions is 00:02:00-00:03:00, and the marker action is the limb key point information corresponding to the character action at time 00:02:15, so the limb key point information corresponding to 00:02:15 is taken as the limb key point information of the marker action, and the start and end time is 00:02:00-00:03:00.

[0113] In some embodiments, the limb key point information of the mark action can refer to a set of limb key point information of multiple actions. For example, the start and end time of a set of actions is 00:02:00-00:03:00, the mark action is the limb key point information corresponding to the action of the character at the time of 00:02:10, 00:02:12 and 00:02:14, so the limb key point information corresponding to 00:02:10, 00:02:12 and 00:02:14 is taken as the limb key point information of the mark action, and the start and end time is 00:02:00-00:03:00.

[0114] Step S503: controlling the display 260 to display the media data;

[0115] In some embodiments, after downloading the media data of the fitness video, the media data needs to be decoded. The media data includes video data and audio data. The decoded video data is sent to the display 260 after rendering for display. The decoded audio data is sent to the audio output interface 270 for playing. The video data and the audio data are played synchronously.

[0116] Step S504: determining a target range according to the playing progress of the media data, and obtaining target limb key point information in the target range;

[0117] Before the step of determining the target range, it is necessary to detect whether the log dot file includes a mark point, and the mark point is the time corresponding to the mark action;

[0118] In some embodiments, the step of detecting whether the log dot file includes a mark point includes:

[0119] Detecting the value of the mark point flag bit in the log dot file;

[0120] If the value of the mark point flag bit in the log dot file is 1, it is determined that the log dot file includes a mark point;

[0121] If the value of the mark point flag bit in the log dot file is 0, it is determined that the log dot file does not include a mark point.

[0122] In some embodiments, the step of detecting whether the log dot file includes a mark point includes:

[0123] Detecting the value of the video type flag bit in the log dot file;

[0124] If the value of the video type flag bit in the log dot file is 1, it is determined that the type of the fitness video is an action video, i.e., the log dot file includes a mark point;

[0125] If the value of the video type flag in the log dot file is 2, it is determined that the type of the fitness video is a dance video, that is, no marker point is included in the log dot file.

[0126] In some embodiments, if it is detected that the marker point is included in the log dot file, a target range is determined, the target range has a first time point as a starting point and a second time point as a terminal point, a difference between the play progress and the first time point is a first preset value, the first preset value is greater than or equal to 0, a difference between the second time point and the play progress is a second preset value, and the second preset value is greater than 0.

[0127] The step of determining the target range based on the play progress of the media asset data includes:

[0128] Subtracting the play progress from the first preset value to obtain the first time point;

[0129] Adding the second preset value to the play progress to obtain the second time point;

[0130] Determining that the target range is a time period with the first time point as a starting point and the second time point as a terminal point, as shown in Figure 7

[0131] Obtaining target limb key point information in the target range, wherein the target limb key point information refers to the limb key point information of the marker point.

[0132] In some embodiments, the log dot file contains a large amount of limb key point information, and the limb key point information includes limb key point information of a marker action (marker point) and limb key point information of a non-marker action (non-marker point).

[0133] The step of obtaining the limb key point information of the marker point in the target range includes:

[0134] Traversing the limb key point information in the target range;

[0135] Detecting the value of the flag of the limb key point information;

[0136] If the value of the flag of the limb key point information is 1, it is determined that the limb key point information is the limb key point information of the marker point, and the limb key point information of the marker point is obtained.

[0137] If the value of the flag of the limb key point information is 0, it is determined that the limb key point information is the limb key point information of the non-marker point, and the value of the flag of the next limb key point information is detected.

[0138] ​In some embodiments, the body key point information of the landmark points in the fitness video is saved separately in a file, and the body key point information of the landmark points in the target range can be directly obtained from the file.

[0139] In some embodiments, the first preset value and the second preset value are fixed values.

[0140] For example, the first preset value is 60 seconds, the second preset value is 120 seconds, the current playing progress is 00:10:23, the first time point is 00:09:23, and the second time point is 00:12:23. The target range is 00:09:23-00:12:23. The body key point information of the landmark points in 00:09:23-00:12:23 is obtained.

[0141] In some embodiments, the duration of the fitness video is not fixed. If the total duration of the fitness video is short, but the first preset value and the second preset value are large, the action figure images corresponding to the landmark points will be displayed too densely, and even may be superimposed together, so that the user cannot identify. If the total duration of the fitness video is long, but the first preset value and the second preset value are small, the action figure images corresponding to the landmark points will be displayed too sparsely, and even no other landmark points in the target range can be obtained, so that the prediction function cannot be achieved.

[0142] The total duration of the fitness video is divided by the first preset multiple and rounded to obtain the first preset value;

[0143] The total duration of the fitness video is divided by the second preset multiple and rounded to obtain the second preset value.

[0144] For example, the current playing progress is 00:02:23, the total duration of the fitness video is 5 minutes and 30 seconds, the first preset multiple is 15, the second preset multiple is 10, the first preset value is 22s, and the second preset value is 33s. The first time point is 00:02:01, and the second time point is 00:02:56. The target range is 00:02:01-00:02:56. The body key point information of the landmark points in 00:02:01-00:02:56 is obtained.

[0145] In some embodiments, the time corresponding to the landmark point is often a time period. As long as there is a point in the target range falling within the time period corresponding to the landmark point, it can be determined that the landmark point is in the target range. For example, the target range is 00:02:01-00:02:56, and the start and end time of the landmark point is 00:01:40-00:02:10. The landmark point is in the target range.

[0146] In some embodiments, the time corresponding to a landmark point is usually a time period. If all of the time period corresponding to a landmark point falls within the target range, it is determined that the landmark point is not within the target range. For example, the target range is 00:02:01-00:02:56, the start and end time of landmark point 1 is 00:01:40-00:02:10, and the start and end time of landmark point 2 is 00:02:11-00:02:30. It is determined that landmark point 1 is not within the target range, and landmark point 2 is within the target range.

[0147] In some embodiments, if the log file does not include a landmark point, a fixed point is determined.

[0148] The fixed point is a time point every preset time interval based on a preset progress of the media data. The preset progress can be a start progress, an end progress, or any progress in the media data. For example, the preset progress of the media data is the start progress of the media data. As shown in FIG. 3, a fixed point is determined every T time interval starting from the start progress 00:00:00. Figure 8

[0149] The step of determining the target range based on the playing progress of the media data includes:

[0150] Subtracting the playing progress from a first preset value to obtain a first time point, wherein the first preset value is greater than or equal to 0;

[0151] Adding the playing progress to a second preset value to obtain a second time point, wherein the second preset value is greater than 0;

[0152] Determining that the target range is a time period with the first time point as a start point and the second time point as an end point, as shown in FIG. 4. Figure 8

[0153] Obtaining target limb key point information in the target range, wherein the target limb key point information refers to limb key point information of a fixed point.

[0154] In some embodiments, the log file includes a large amount of limb key point information, which includes limb key point information of a fixed point and limb key point information of a non-fixed point.

[0155] The step of obtaining the limb key point information of the fixed point in the target range includes:

[0156] Traversing the limb key point information in the target range;

[0157] Detecting the value of the fixed flag of the limb key point information;

[0158] ​​If the value of the fixed flag bit of the body key point information is 1, it is determined that the body key point information is fixed point body key point information, and the fixed point body key point information is obtained.

[0159] If the value of the fixed flag bit of the body key point information is 0, it is determined that the body key point information is non-fixed point body key point information, and the value of the fixed flag bit of the next body key point information is detected.

[0160] In some embodiments, the fixed point body key point information in the fitness video is saved separately in a file, and the fixed point body key point information in the target range can be directly obtained in the file.

[0161] In some embodiments, if there is no fixed flag bit in the body key point information, i.e., no fixed point is marked in the log point file, the controller can select a fixed point every preset time interval based on a preset progress as a reference point, and save the body key point information corresponding to the fixed point and the start and end time information in the file. The fixed point body key point information in the target range can be directly obtained in the file.

[0162] In some embodiments, the fixed point body key point information can refer to the body key point information of one action. For example, a fixed point is determined every 1 minute based on the starting progress of the media data as a reference point, and the fixed point times are 00:00:00, 00:01:00, 00:02:00, and the fixed point body key point information is the body key point information corresponding to 00:00:00, 00:01:00, 00:02:00 at 00:00:00, 00:01:00, 00:02:00.

[0163] In some embodiments, the fixed point body key point information can refer to a collection of body key point information of multiple actions. For example, a fixed point is determined every 1 minute based on the starting progress of the media data as a reference point, and the fixed point times are 00:00:00, 00:01:00, 00:02:00, and the fixed point body key point information of 00:01:00 is the body key point information corresponding to 00:01:00, 00:01:01, 00:01:02 at 00:01:00, 00:01:01, 00:01:02.

[0164] In some embodiments, if it is detected that the log point file includes a marker point, it is determined that the marker point corresponding to the playing progress is the target point.

[0165] The marker point corresponding to the playing progress can be determined by determining which marker point the playing progress falls into. For example, the playing progress is 00:01:00, and the start and end times of the marker point A are 00:00:50-00:01:20, so the marker point corresponding to the playing progress is the marker point A.

[0166] determining a target range, the target range including a target point, a first preset number of landmark points before the target point, and a second preset number of landmark points after the target point, wherein the first preset number is greater than or equal to 0, and the second preset number is greater than 0, as shown in Figure 9

[0167] The first preset number of landmark points before the target point refers to the first preset number of landmark points closest in time to the target point before the target point. The second preset number of landmark points after the target point refers to the second preset number of landmark points closest in time to the target point after the target point. For example, if there are landmark points 1-8 in ascending and descending time order, and the target point is landmark point 5, and the first preset number and the second preset number are both 2, then the target range is landmark points 3-7.

[0168] obtaining target limb key point information in the target range, wherein the target limb key point information refers to limb key point information of the landmark points.

[0169] In some embodiments, if the landmark points are not included in the log dot file, determining fixed points, the fixed points being time points every preset time interval based on preset progress of the media data;

[0170] determining the fixed point corresponding to the playing progress as the target point;

[0171] If the playing progress is exactly the same as the time of the fixed point, the fixed point is determined as the target point. If the playing progress is different from the time of the fixed point, it is determined that the playing progress is between which two fixed points, and the fixed point earlier in time is taken as the target point. For example, if a fixed point is determined every 1 minute based on the starting progress of the media data as the reference point, the time of the fixed point is 00:00:00, 00:01:00, 00:02:00, and so on. If the playing progress is 00:01:00, the fixed point 00:01:00 corresponding to the playing progress is determined as the target point. If the playing progress is 00:01:30, the playing progress falls between the fixed point 00:01:00 and the fixed point 00:02:00, and the fixed point 00:01:00 corresponding to the playing progress is determined as the target point.

[0172] determining a target range, the target range including a target point, a first preset number of landmark points before the target point, and a second preset number of landmark points after the target point, wherein the first preset number is greater than or equal to 0, and the second preset number is greater than 0, as shown in Figure 10

[0173] ​​The first preset number of fixed points before the target point refers to the first preset number of fixed points before the target point and closest in time to the target point. The second preset number of fixed points after the target point refers to the second preset number of fixed points after the target point and closest in time to the target point. For example, if there are fixed points 1-8 in time sequence, the target point is fixed point 5, and the first preset number and the second preset number are both 1, the target range is fixed points 4-6.

[0174] Obtaining target limb key point information in the target range, wherein the target limb key point information refers to limb key point information of the fixed points.

[0175] Step S505: generating an action prompt character image according to the target limb key point information;

[0176] In some embodiments, the step of generating an action prompt character image according to the target limb key point information comprises:

[0177] Obtaining information of basic limb key points in the target limb key point information;

[0178] The target limb key point information includes coordinate information of 17 or 19 limb points, and the information of the basic limb key points is part of the 17 or 19 limb points. For example, the basic limb key points include nose 0, left shoulder 5, right shoulder 6, left elbow 7, right elbow 8, left wrist 9, right wrist 10, left crotch 11, right crotch 12, left knee 13, right knee 14, left ankle 15, and right ankle 16.

[0179] Drawing points and lines based on the information of the basic limb key points to generate an action prompt character image.

[0180] In some embodiments, the limb points are classified and associated.

[0181] a) The center points of the nose 0 and the two shoulders (5 and 6) form the head;

[0182] b) The two shoulders (5, 6) and the two crotches (11, 12) form the torso;

[0183] c) The right shoulder (6), elbow (8), and wrist (10) form the right arm;

[0184] d) The left shoulder (5), elbow (7), and wrist (9) form the left arm;

[0185] e) The right crotch (12), knee (14), and ankle (16) form the right leg;

[0186] f) The left crotch (11), knee (13), and ankle (15) form the left leg;

[0187] g) The virtual animated figure is composed of the six parts of the body mentioned above.

[0188] The head of the virtual animated figure is drawn by taking the nose coordinates as the center and a fixed value as the radius, and the above body parts are connected by simple lines to generate a motion prompt figure image like a matchstick man. The motion prompt figure image of a cartoon figure can also be generated by filling the figure image based on the information of the above body parts.

[0189] Step S506: Control the display 260 to display the motion prompt figure image on the media data, and the motion prompt figure image is displayed in the order of the time corresponding to the target body key point information.

[0190] In some embodiments, the media data is displayed in the video playing layer, a transparent browser window is created on the upper layer of the video playing layer, and the motion prompt figure image is displayed in proportion on the browser window.

[0191] In some embodiments, the display area required on the screen is inconsistent with the size of the identified picture, and all coordinate points need to be scaled in proportion to the display area. The width dw and the height dh of the picture taken by the camera; the display range width rw and the height rh; the picture scaling ratio r = Math.min(rw / dw, rh / dh), the scaled width w = dw*r, and the height h = dh*r. All coordinate points are processed in the same scaling ratio to obtain the position of the human body key points in the display area.

[0192] In some embodiments, the motion prompt figure image is displayed in the order of the time corresponding to the target body key point information in reverse order horizontally, as shown in Figure 11 .

[0193] In some embodiments, the motion prompt figure image is displayed in the order of the time corresponding to the target body key point information in reverse order vertically, as shown in Figure 12 .

[0194] In some embodiments, the display position of the motion prompt figure image is determined according to the time corresponding to the target body key point information and the playing progress.

[0195] In some embodiments, the current playing progress corresponding motion prompt figure image is determined, and the current playing progress corresponding motion prompt figure image is displayed in the third position, and the first time corresponding to the motion prompt figure image is obtained.

[0196] If the first difference between the time corresponding to the motion prompt figure image and the first time is greater than the first preset difference, the motion prompt figure image is displayed in the first position, and the first preset difference is greater than 0.

[0197] If a first difference between the time corresponding to the action prompt character image and the first time is less than a first preset difference and greater than 0, displaying the action prompt character image at a second position;

[0198] If a second difference between the first time and the time corresponding to the action prompt character image is greater than a second preset difference, displaying the action prompt character image at a fourth position, and the second preset difference is greater than 0;

[0199] If a second difference between the first time and the time corresponding to the action prompt character image is less than a second preset difference and greater than 0, displaying the action prompt character image at the fifth position;

[0200] If the action prompt character image corresponds to a marker point, the time corresponding to the action prompt character image is the start time of the marker point. If the action prompt character image corresponds to a fixed point, the time corresponding to the action prompt character image is the time of the fixed point. If multiple action prompt character images are displayed at the same location, the multiple action prompt character images can be displayed side by side.

[0201] For example, Figure 12 As shown, the current playback progress is 00:04:00, and a total of action figure images 1-6 are generated, with corresponding times of 00:02:00, 00:03:20, 00:03:40, 00:04:00, 00:04:30, and 00:06:00, respectively. The first preset difference and the second preset difference are 60s, the coordinates of the first position are (1800, 100), the coordinates of the second position are (1800, 300), the coordinates of the third position are (1800, 500), the coordinates of the fourth position are (1800, 700), and the coordinates of the third position are (1800, 900). Then, action figure image 1 is displayed at the fifth position, action figure images 2 and 3 are displayed at the fourth position, action figure image 4 is displayed at the third position, action figure image 5 is displayed at the second position, and action figure image 6 is displayed at the first position.

[0202] In some embodiments, a first position, a second position, and a third position for displaying the action prompt character image are determined. The first position is the starting position for displaying the action prompt character image. The second position is the display position of the action prompt character image generated by the target limb key point information corresponding to the current playback progress. The third position is the ending position for displaying the action prompt character image, and the second position and the third position may be the same. The first position, the second position, and the third position all refer to the position of the center point of the action prompt character image.

[0203] Obtaining a first distance between the first position and the second position, and a second distance between the second position and the third position;

[0204] determining a current playing progress corresponding action prompt character image and displaying the current playing progress corresponding action prompt character image at the second position, obtaining a first time corresponding to the action prompt character image;

[0205] if a first difference between the time corresponding to the action prompt character image and the first time is greater than a first preset difference, displaying the action prompt character image at the first position, the first preset difference being greater than 0;

[0206] if the first difference between the time corresponding to the action prompt character image and the first time is less than the first preset difference and greater than 0, determining a display position of the action prompt character image according to the first difference, the first preset difference and a first distance, the distance between the action prompt character image and the second position being (first difference / first preset difference)*first distance;

[0207] if a second difference between the first time and the time corresponding to the action prompt character image is greater than a second preset difference, displaying the action prompt character image at a third position, the second preset difference being greater than 0;

[0208] if the second difference between the first time and the time corresponding to the action prompt character image is less than the second preset difference and greater than 0, determining a display position of the action prompt character image according to the second difference, the second preset difference and a second distance, the distance between the action prompt character image and the second position being (second difference / second preset difference)*second distance;

[0209] If the action prompt character image corresponds to a landmark point, the time corresponding to the action prompt character image refers to the starting time of the landmark point. If the action prompt character image corresponds to a fixed point, the time corresponding to the action prompt character image refers to the time of the fixed point. If there are multiple action prompt character images displayed at the same position, the multiple action prompt character images can be displayed side by side or superimposed, with the uppermost layer being the action prompt character image closest to the playing progress time.

[0210] For example, Figure 11As shown, the current playing progress is 00:04:00, a total of 1-5 action figure images are generated, and the corresponding times are 00:02:00, 00:03:20, 00:04:00, 00:05:30 and 00:06:00 respectively, the first preset difference and the second preset difference are 60s, the coordinates of the first position are (100, 1000), the coordinates of the second position are (1200, 1000), the coordinates of the third position are (1800, 1000), the first distance is 1100, and the second distance is 600. Then, the action figure image 1 is displayed at the third position, the horizontal coordinate of the action figure image 2 is 1600, and the display position is (1600, 1000); the action figure image 3 is displayed at the second position, the horizontal coordinate of the action figure image 4 is 650, and the display position is (650, 1000); and the action figure image 5 is displayed at the first position.

[0211] In some embodiments, as shown in FIG. 6, the action figure image is displayed in the upper right corner, and the display position of the action figure image changes in both horizontal and vertical directions. The horizontal and vertical coordinates of the action figure image can be calculated in the above manner. Figure 13

[0212] In some embodiments, the limb key point information of the marked action and fixed point can refer to the limb key point information of one action, and the action figure image corresponding to the limb key point information can be displayed at the specified position.

[0213] In some embodiments, the limb key point information of the marked action and fixed point can refer to a set of limb key point information of multiple actions. The multiple actions can be displayed at the position corresponding to the marked point in a certain order and frequency. For example, the marked point corresponds to three limb key point information, i.e., there are three action figure images, and the display of the action figure images can be switched cyclically at intervals of 1s to achieve a simple animation effect.

[0214] In some embodiments, the display color of the action prompt figure image is determined according to the time corresponding to the target limb key point information and the playing progress.

[0215] The action prompt figure image corresponding to the current playing progress is determined, and the color of the action prompt figure image is set to a first color. The first time corresponding to the action prompt figure image is obtained.

[0216] If the first difference between the time corresponding to the action prompt figure image and the first time is greater than a first preset difference, the color of the action prompt figure image is set to a second color, and the first preset difference is greater than 0.

[0217] ​If the first difference between the time corresponding to the action prompt character image and the first time is less than the first preset difference and greater than 0, a color of the action prompt character image is set to a third color.

[0218] If the first difference between the time corresponding to the action prompt character image and the first time is less than 0, a color of the action prompt character image is set to a fourth color, and the second color can be the same as the fourth color.

[0219] In some embodiments, the transparency of the action prompt character image is determined according to the time corresponding to the target body key point information and the playing progress.

[0220] An action prompt character image corresponding to the current playing progress is determined, and the transparency of the action prompt character image is set to a first transparency. A first time corresponding to the action prompt character image is obtained.

[0221] In some embodiments, if the first difference between the time corresponding to the action prompt character image and the first time is greater than the first preset difference, the transparency of the action prompt character image is set to a second transparency, and the first preset difference is greater than 0.

[0222] If the first difference between the time corresponding to the action prompt character image and the first time is less than the first preset difference and greater than 0, the transparency of the action prompt character image is set to a third transparency.

[0223] If the first difference between the time corresponding to the action prompt character image and the first time is less than 0, the transparency of the action prompt character image is set to a fourth transparency, and the first transparency> the third transparency> the second transparency≥ the fourth transparency.

[0224] In some embodiments, if the first difference between the time corresponding to the action prompt character image and the first time is greater than the first preset difference, the transparency of the action prompt character image is set to a second transparency, and the first preset difference is greater than 0.

[0225] If the first difference between the time corresponding to the action prompt character image and the first time is less than the first preset difference and greater than 0, the transparency of the action prompt character image is determined according to the first difference, the first preset difference, the first transparency, and the second transparency. The transparency of the action prompt character image is (first difference / first preset difference)*(first transparency-second transparency).

[0226] If the first difference between the time corresponding to the action prompt character image and the first time is less than 0, the transparency of the action prompt character image is set to the second transparency, and the first transparency is greater than the second transparency.

[0227] In some embodiments, the scaling ratio of the action prompt character image is determined according to the time corresponding to the target body key point information and the playing progress.

[0228] The action prompt character image corresponding to the current playing progress is determined, and the scaling ratio of the action prompt character image is set as a first scaling ratio. A first time corresponding to the action prompt character image is obtained;

[0229] In some embodiments, if a first difference between the time corresponding to the action prompt character image and the first time is greater than a first preset difference, the scaling ratio of the action prompt character image is set as a second scaling ratio, and the first preset difference is greater than 0.

[0230] If the first difference between the time corresponding to the action prompt character image and the first time is less than the first preset difference and greater than 0, the scaling ratio of the action prompt character image is set as a third scaling ratio.

[0231] If the first difference between the time corresponding to the action prompt character image and the first time is less than 0, the scaling ratio of the action prompt character image is set as the second scaling ratio, and the first scaling ratio> the third scaling ratio> the second scaling ratio.

[0232] In some embodiments, if the first difference between the time corresponding to the action prompt character image and the first time is greater than the first preset difference, the scaling ratio of the action prompt character image is set as the second scaling ratio, and the first preset difference is greater than 0.

[0233] If the first difference between the time corresponding to the action prompt character image and the first time is less than the first preset difference and greater than 0, the scaling ratio of the action prompt character image is determined according to the first difference, the first preset difference, the first scaling ratio and the second scaling ratio. The transparency of the action prompt character image is (first difference / first preset difference)*(first scaling ratio-second scaling ratio).

[0234] If the first difference between the time corresponding to the action prompt character image and the first time is less than 0, the scaling ratio of the action prompt character image is set as the second scaling ratio.

[0235] In some embodiments, when the action prompt character image corresponding to the playing progress is displayed, some special effects or action prompt character images can be appropriately added around the extra markers or image processing, so that the user can quickly distinguish which action prompt character image is the current action.

[0236] In some embodiments, the name of the sign action can be displayed below the action prompt character image, which can help the user better understand the next action in advance and improve the accuracy of the action.

[0237] Some embodiments of the present application provide a fitness video playing method, which is suitable for a display device including a display and a controller, and the controller is configured to: in response to an instruction of playing a fitness video, download media data and a log file of the fitness video, the log file including body key point information and time corresponding to the body key point information; control the display to display the media data; determine a target range according to a playing progress of the media data, and acquire target body key point information in the target range; generate an action prompt character image according to the target body key point information; and control the display to display the action prompt character image, which is displayed according to time corresponding to the target body key point information. By analyzing the log file corresponding to the fitness video, the body key point information in the fitness video is extracted, and the action prompt character is generated according to the body key point information, so that the action can be predicted, the user can prepare in advance, the action accuracy is improved, and the user experience is improved.

[0238] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0239] For the convenience of explanation, the above description has been made in combination with specific embodiments. However, the above exemplary discussion is not intended to exhaust or limit the embodiments to the specific forms disclosed above. Various modifications and variations can be derived according to the above teachings. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different modified embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: monitor; The controller is configured as: In response to an instruction to play a fitness video, downloading media data and a log dot file of the fitness video, wherein the log dot file includes limb key point information and the time corresponding to the limb key point information; Controlling the display to display the media data; Determine a target range according to the playback progress of the media asset data, and obtain target limb key point information within the target range; Generate an action prompt character image based on the target limb key point information; The display is controlled to display the action prompt character image on the media data, and the action prompt character image is displayed in a time arrangement corresponding to the target limb key point information.

2. The display device according to claim 1, wherein The controller determines the target range according to the playback progress of the media asset data, and obtains the target limb key point information within the target range, and is further configured to: If the log dot file includes a mark point, determining a target range, the mark point being the time corresponding to the mark action, the target range starting at a first time point and ending at a second time point, the difference between the playback progress and the first time point being a first preset value, the first preset value being greater than or equal to 0, and the difference between the second time point and the playback progress being a second preset value, the second preset value being greater than 0; Obtain limb key point information of landmark points within the target range.

3. The display device according to claim 2, wherein The controller determines the target range according to the playback progress of the media asset data, and obtains the target limb key point information within the target range, and is further configured to: If the log dot file does not include a marker point, determining a fixed point, wherein the fixed point is a time point at which a preset time interval is set based on a preset progress of the media asset data; Determine a target range, where the target range starts at a first time point and ends at a second time point, a difference between the playback progress and the first time point is a first preset value, the first preset value is greater than or equal to 0, and a difference between the second time point and the playback progress is a second preset value, the second preset value is greater than 0; Obtain limb key point information of fixed points within the target range.

4. The display device according to claim 1, wherein The controller determines the target range according to the playback progress of the media asset data, and obtains the target limb key point information within the target range, and is further configured to: If the log dot file includes a mark point, determining the mark point corresponding to the playback progress as the target point, the mark point being the time corresponding to the mark action; Determining a target range, the target range including a target point, a first preset number of marker points before the target point, and a second preset number of marker points after the target point; Obtain limb key point information of landmark points within the target range.

5. The display device according to claim 4, wherein: The controller determines a target range according to the playback progress of the media asset data and obtains target limb key point information within the target range, and is further configured to: If the log dot file does not include a marker point, determining a fixed point, wherein the fixed point is a time point at which a preset time interval is set based on a preset progress of the media asset data; Determine the fixed point corresponding to the playback progress as the target point; determining a target range, the target range including a target point, a first preset number of fixed points before the target point, and a second preset number of fixed points after the target point; Obtain limb key point information of fixed points within the target range.

6. The display device according to claim 1, wherein The controller is configured to: Determining a display position of the action prompt character image according to the time corresponding to the target limb key point information and the playback progress; The display is controlled to display the action prompt character image according to the display position.

7. The display device according to claim 1, wherein The controller is configured to: Determining the display color or transparency of the action prompt character image according to the time corresponding to the target limb key point information and the playback progress; The display is controlled to display the action prompt character image according to the display color or transparency.

8. The display device according to claim 1, wherein The controller is configured to: Determining a scaling ratio of the action prompt character image according to the time corresponding to the target limb key point information and the playback progress; The display is controlled to display the action prompt character image according to the zoom ratio.

9. The display device according to claim 1, wherein The controller generates an action prompt character image based on the target limb key point information, and is further configured to: Obtaining information of basic limb key points in the target limb key point information; Based on the information of the basic limb key points, points and lines are drawn to generate an action prompt character image.

10. A method for playing a fitness video, characterized in that: include: In response to an instruction to play a fitness video, downloading media data and a log dot file of the fitness video, wherein the log dot file includes limb key point information and the time corresponding to the limb key point information; Controlling the display to display the media asset data; Determining a target range based on the playback progress of the media asset data; Acquire key point information of the target limb within the target range; Generate an action prompt character image based on the target limb key point information; The display is controlled to display the action prompt character image on the media data, and the action prompt character image is displayed in a time arrangement corresponding to the target limb key point information.

Images