Video playback method, device, equipment and storage medium

Abstract

The present invention discloses a video playback method, device, equipment and storage medium, the method comprising: obtaining a basic picture layer of a second terminal, and pulling an enhanced picture layer of video content; performing time domain alignment on the basic picture layer and the enhanced picture layer; superimposing the basic picture layer and the enhanced picture layer after time domain alignment to obtain superimposed enhanced video content, and playing the enhanced video content. The present invention enables a first terminal and a second terminal to have different playback effects, and improves the playback effect of video playback in the first terminal, thereby increasing the richness of the user's video playback experience.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a video playback method, device, equipment and storage medium. Background Art

[0002] With the development of terminal technology, more and more terminal devices have multi-screen interactive functions to play content synchronously on different terminal devices. For example, a terminal device with a large screen can project the screen of the terminal device to other terminal devices with small screens for synchronous playback to facilitate user browsing or improve the playback effect of the content. However, when performing multi-screen interaction, the only difference between different screens is the viewing method, and there is no difference in the playback effect of watching video content. Summary of the invention

[0003] The main purpose of the present invention is to provide a video playback method, device, equipment and storage medium, aiming to solve the problem of how to improve the playback effect of video playback.

[0004] To achieve the above object, the present invention provides a video playback method, which comprises the following steps:

[0005] Obtain the basic picture layer of the second terminal and pull the enhanced picture layer of the video content;

[0006] Performing temporal alignment of the basic picture layer and the enhanced picture layer;

[0007] The basic picture layer and the enhanced picture layer after time domain alignment are superimposed to obtain superimposed enhanced video content, and the enhanced video content is played.

[0008] Optionally, the step of performing temporal alignment of the basic picture layer and the enhanced picture layer comprises:

[0009] Determining a time delay difference between the basic picture layer and the enhanced picture layer;

[0010] According to the delay difference, the enhanced picture layer and the basic picture layer are aligned in time domain.

[0011] Optionally, the step of determining the time delay difference between the basic picture layer and the enhanced picture layer includes:

[0012] Determine a basic frequency spectrum graph corresponding to the basic picture layer and an enhanced frequency spectrum graph of the enhanced picture layer;

[0013] Determine a first frequency domain variation of the basic spectrum graph as it varies with the time domain, and a second frequency domain variation of the enhanced spectrum graph as it varies with the time domain;

[0014] A time delay difference between the basic picture layer and the enhanced picture layer is determined according to the first frequency domain change value and the second frequency domain change amount.

[0015] Optionally, the step of determining a first frequency domain variation of the basic spectrum graph as it varies with time domain, and a second frequency domain variation of the enhanced spectrum graph as it varies with time domain comprises:

[0016] Determine a first sampling region of the basic spectrum graph and a second sampling region of the enhanced spectrum graph;

[0017] Determine the first frequency domain variation of the basic spectrum graph according to a frequency domain variation mean of the first sampling area as it varies in the time domain;

[0018] The second frequency domain variation of the enhanced frequency spectrum is determined according to a frequency domain variation mean of the second sampling area varying with time domain.

[0019] Optionally, the step of pulling the enhanced picture layer of the video content includes:

[0020] After the second terminal plays the video content, a binding relationship between the second terminal and the first terminal is established, and a connection channel between the second terminal and the first terminal is established;

[0021] Based on the binding relationship and the connection channel, the enhanced picture layer of the video content is pulled up.

[0022] Optionally, the step of pulling up the enhanced picture layer of the video content based on the binding relationship and the connection channel includes:

[0023] Determine, according to the user identity and the binding relationship, a second terminal corresponding to the first terminal;

[0024] Obtaining the video content being played by the second terminal and a timestamp corresponding to the video content;

[0025] An enhanced picture layer of the video content is determined based on local attribute characteristics, the video content being played by the second terminal and a timestamp corresponding to the video content, wherein the attribute characteristics include at least one of user attributes, device attributes, network attributes and regional attributes.

[0026] Optionally, the step of determining the enhanced picture layer of the video content according to the local attribute characteristics, the video content being played by the second terminal, and the timestamp corresponding to the video content includes:

[0027] Determine the user vector based on local attribute features;

[0028] Acquire multiple picture sub-layers of each layer according to the video content being played by the second terminal and the timestamp corresponding to the video content, and determine the layer vector of each picture sub-layer;

[0029] Determine the relevance between the picture sublayer and the local area according to the user vector and the layer vector;

[0030] An enhanced picture layer of the video content is determined according to the correlation.

[0031] To achieve the above object, the present invention further provides a video playback device, the video playback device comprising:

[0032] An acquisition module, used to acquire a basic picture layer of the second terminal and pull an enhanced picture layer of the video content;

[0033] A superposition module, used for performing temporal alignment between the basic picture layer and the enhanced picture layer;

[0034] The playing module is used to superimpose the basic picture layer and the enhanced picture layer after time domain alignment to obtain the superimposed enhanced video content, and play the enhanced video content.

[0035] To achieve the above objectives, the present invention also provides a video playback device, which includes a memory, a processor, and a video playback program stored in the memory and executable on the processor, and when the video playback program is executed by the processor, the various steps of the video playback method described above are implemented.

[0036] To achieve the above object, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a video playback program, and when the video playback program is executed by a processor, the various steps of the video playback method described above are implemented.

[0037] The present invention provides a video playback method, device, equipment and storage medium, which obtains the basic picture layer of the second terminal, and pulls the enhanced picture layer of the video content; the basic picture layer and the enhanced picture layer are aligned in time domain; the basic picture layer and the enhanced picture layer after time domain alignment are superimposed to obtain the superimposed enhanced video content, and the enhanced video content is played. Through the delay difference, the enhanced picture layer and the basic picture layer are aligned in time domain, and the enhanced picture layer after time domain alignment is superimposed on the basic picture layer, so that a complete video picture layer is formed on the first terminal, and the first terminal and the second terminal have different playback effects, and the playback effect of the video playback is improved in the first terminal, which increases the richness of the user's video playback experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1A schematic diagram of the hardware structure of a video playback device involved in an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of a flow chart of a first embodiment of a video playback method of the present invention;

[0040] Figure 3 A schematic diagram of superimposing a basic picture layer and an enhanced picture layer in the video playback method of the present invention;

[0041] Figure 4 Detailed flowchart of step S20 of the second embodiment of the video playback method of the present invention;

[0042] Figure 5 A schematic diagram of a sampling area in the video playback method of the present invention;

[0043] Figure 6 It is a schematic diagram of time domain and frequency domain curves in the video playback method of the present invention;

[0044] Figure 7 A schematic diagram of the time domain difference corresponding to the time domain and frequency domain curves in the video playback method of the present invention;

[0045] Figure 8 This is a detailed flowchart of step S10 of the third embodiment of the video playback method of the present invention;

[0046] Fig. 9 A schematic diagram of a binding relationship between a second terminal and a first terminal in a video playback method of the present invention;

[0047] Fig.10 The figure is a schematic diagram of the logical structure of a video playback device involved in an embodiment of the present invention.

[0048] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0050] The main solution of the embodiment of the present invention is: obtain the basic picture layer of the second terminal, and pull the enhanced picture layer of the video content; align the basic picture layer and the enhanced picture layer in time domain; superimpose the basic picture layer and the enhanced picture layer after time domain alignment to obtain the superimposed enhanced video content, and play the enhanced video content.

[0051] Through the delay difference, the enhanced picture layer and the basic picture layer are aligned in time domain, and the enhanced picture layer after time domain alignment is superimposed on the basic picture layer, forming a complete video picture layer on the first terminal. The first terminal and the second terminal have different playback effects, and the playback effect of the video playback in the first terminal is improved, increasing the richness of the user's video playback experience.

[0052] As an implementation scheme, the video playback device can be as follows Figure 1 shown.

[0053] The embodiment of the present invention relates to a video playback device, which includes: a processor 101, such as a CPU, a memory 102, and a communication bus 103. The communication bus 103 is used to realize connection and communication between these components.

[0054] The memory 102 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Figure 1 As shown, the memory 102 as a computer-readable storage medium may include a video playback program; and the processor 101 may be used to call the video playback program stored in the memory 102 and perform the following operations:

[0055] Obtain the basic picture layer of the second terminal and pull the enhanced picture layer of the video content;

[0056] Performing temporal alignment of the basic picture layer and the enhanced picture layer;

[0057] The basic picture layer and the enhanced picture layer after time domain alignment are superimposed to obtain superimposed enhanced video content, and the enhanced video content is played.

[0058] Optionally, the processor 101 may be used to call a video playback program stored in the memory 102 and perform the following operations:

[0059] Determining a time delay difference between the basic picture layer and the enhanced picture layer;

[0060] According to the delay difference, the enhanced picture layer and the basic picture layer are aligned in time domain.

[0061] Optionally, the processor 101 may be used to call a video playback program stored in the memory 102 and perform the following operations:

[0062] Determine a basic frequency spectrum graph corresponding to the basic picture layer and an enhanced frequency spectrum graph of the enhanced picture layer;

[0063] Determine a first frequency domain variation of the basic spectrum graph as it varies with the time domain, and a second frequency domain variation of the enhanced spectrum graph as it varies with the time domain;

[0064] A time delay difference between the basic picture layer and the enhanced picture layer is determined according to the first frequency domain change value and the second frequency domain change amount.

[0065] Optionally, the processor 101 may be used to call a video playback program stored in the memory 102 and perform the following operations:

[0066] Determine a first sampling region of the basic spectrum graph and a second sampling region of the enhanced spectrum graph;

[0067] Determine the first frequency domain variation of the basic spectrum graph according to a frequency domain variation mean of the first sampling region as it varies in the time domain;

[0068] The second frequency domain variation of the enhanced frequency spectrum is determined according to a frequency domain variation mean of the second sampling area varying with time domain.

[0069] Optionally, the processor 101 may be used to call a video playback program stored in the memory 102 and perform the following operations:

[0070] After the second terminal plays the video content, a binding relationship between the second terminal and the first terminal is established, and a connection channel between the second terminal and the first terminal is established;

[0071] Based on the binding relationship and the connection channel, the enhanced picture layer of the video content is pulled up.

[0072] Optionally, the processor 101 may be used to call a video playback program stored in the memory 102 and perform the following operations:

[0073] Determine, according to the user identity and the binding relationship, a second terminal corresponding to the first terminal;

[0074] Obtaining the video content being played by the second terminal and a timestamp corresponding to the video content;

[0075] An enhanced picture layer of the video content is determined based on local attribute characteristics, the video content being played by the second terminal and a timestamp corresponding to the video content, wherein the attribute characteristics include at least one of user attributes, device attributes, network attributes and regional attributes.

[0076] Optionally, the processor 101 may be used to call a video playback program stored in the memory 102 and perform the following operations:

[0077] Determine the user vector based on local attribute features;

[0078] Acquire multiple picture sub-layers of each layer according to the video content being played by the second terminal and the timestamp corresponding to the video content, and determine the layer vector of each picture sub-layer;

[0079] Determine the relevance between the picture sublayer and the local area according to the user vector and the layer vector;

[0080] An enhanced picture layer of the video content is determined according to the correlation.

[0081] Based on the hardware architecture of the above-mentioned video playback device, an embodiment of the video playback method of the present invention is proposed.

[0082] Reference Figure 2 , Figure 2 The first embodiment of the video playback method of the present invention comprises the following steps:

[0083] Step S10, obtaining a basic picture layer of the second terminal, and pulling an enhanced picture layer of the video content.

[0084] Optionally, a large screen playback layer, i.e., a basic picture layer, and a small screen playback layer, i.e., an enhanced picture layer, are generated or determined according to the video content. The picture layer is formed by compressing the video content and the surrounding information extended based on the video content into the video stream picture, and each layer includes multiple sub-layers.

[0085] The bottom layer of the video stream is associated with the key value and timestamp. For example, the key value of the first video frame is I_00001_HH:MM:SS:MM, and this is used as the key value of the layer synchronized with the timestamp. That is, the key value of the enhanced picture layer and the basic picture layer of the same video frame is the same, as shown in the following table:

[0086]

[0087] Optionally, the execution subject of the video playback method is a first terminal, wherein the first terminal can be a small-screen client, such as a mobile phone or other device.

[0088] Optionally, the basic picture layer is a video layer of the second terminal, wherein the second terminal may be a large-screen client, such as a television, a projector, or other device. Optionally, the basic picture layer of the second terminal may be obtained by shooting the video content being played by the second terminal through a camera of the first terminal, for example, by capturing each frame of the basic layer at a preset time interval, such as a preset time interval of 0.1 seconds, or by obtaining the basic picture layer of the video content being played by the second terminal stored in the cloud.

[0089] Optionally, the enhanced picture layer is a video layer of the first terminal, wherein the first terminal may be a mobile device such as a mobile phone, a tablet computer, etc. Optionally, the first terminal pulls the enhanced picture layer corresponding to the basic picture layer based on the acquired basic picture layer.

[0090] Optionally, the display effects of the basic picture layer and the enhanced picture layer are different, for example, the display parameters of the basic picture layer and the enhanced picture layer are different, or the display contents of the basic picture layer and the enhanced picture layer are different. The second terminal plays the basic picture layer, and the first terminal plays the enhanced picture layer. The first terminal and the second terminal are combined to watch the complete layers of all video pictures, wherein the basic picture layer and the enhanced picture layer are organically combined.

[0091] Step S20: aligning the basic picture layer and the enhanced picture layer in time domain.

[0092] Optionally, the second terminal uploads the key value of the basic picture layer to the server, the first terminal queries the key value of the basic picture layer on the server, determines the timestamp corresponding to the basic picture layer according to the key value of the basic picture layer, and determines the timestamp of the enhanced picture layer according to the key value of the enhanced picture layer, and aligns the basic picture layer and the enhanced picture layer in time domain according to the timestamps.

[0093] Optionally, a delay difference is determined according to picture contents of the basic picture layer and the enhanced picture layer, and the basic picture layer and the enhanced picture layer are aligned in time domain according to the delay difference.

[0094] Step S30, superimposing the basic picture layer and the enhanced picture layer after time domain alignment to obtain superimposed enhanced video content, and playing the enhanced video content.

[0095] Optionally, the time-domain aligned basic picture layer and enhanced picture layer are superimposed, that is, the pixel points of the enhanced picture layer of the first terminal are superimposed with the pixel points of the basic picture layer of the second terminal, and the complete video picture layer is restored to obtain the superimposed enhanced video content, such as Figure 3 As shown, the augmented video content is played on the first terminal, which can provide the user with an augmented reality video playback experience.

[0096] Optionally, for the first terminal with insufficient processing power, the complete video picture layer can be synthesized in the cloud, and after receiving the instruction of layer overlay synchronization between the first terminal and the second terminal, a compensation layer is sent to the first terminal. The first terminal compensates and repairs the picture layer to obtain a complete video picture layer, and enhances the video content including the video picture layer.

[0097] In the technical solution of this embodiment, by obtaining the basic picture layer being played by the second terminal and synchronizing and correcting it with the enhanced picture layer played by the first terminal, the picture elements of the basic picture layer and the enhanced picture layer are intelligently superimposed and rendered, and a complete video picture layer is formed on the first terminal. The first terminal and the second terminal have different playback effects, and the playback effect of the video playback in the first terminal is improved, thereby increasing the richness of the user's video playback experience.

[0098] Reference Figure 4 , Figure 4 This is a second embodiment of the video playback method of the present invention, based on the first embodiment, step S20 includes:

[0099] Step S21, determining a time delay difference between the basic picture layer and the enhanced picture layer;

[0100] Step S22: aligning the enhanced picture layer with the basic picture layer in the time domain according to the delay difference.

[0101] Optionally, the second terminal stores the key value of the basic picture layer to the server, and the first terminal can query the timestamp corresponding to the basic picture layer through the server. After the first terminal obtains the enhanced picture layer, it determines the timestamp corresponding to the enhanced picture layer, and determines the delay difference between the basic picture layer and the enhanced picture layer according to the difference in timestamps.

[0102] Optionally, determine a basic spectrum graph corresponding to the basic picture layer, and an enhanced spectrum graph of the enhanced picture layer; determine a first frequency domain change in the basic spectrum graph that changes with time domain, and a second frequency domain change in the enhanced spectrum graph that changes with time domain; determine the time delay difference between the basic picture layer and the enhanced picture layer based on the first frequency domain change value and the second frequency domain change.

[0103] Optionally, a first sampling area of ​​the basic spectrum graph and a second sampling area of ​​the enhanced spectrum graph are determined; Optionally, there are multiple sampling areas, such as Figure 5 For example, there are 5 sampling areas, namely block 1, block 2, block 3, block 4 and block 5, where the weight factor of block 1 is 1, the weight factor of block 2 is 0.8, the weight factor of block 3 is 0.7, the weight factor of block 4 is 0.6, and the weight factor of block 5 is 0.5. Optionally, each first sampling area is sampled, the frequency domain change mean of each block is calculated, and then the frequency domain change mean of each first sampling area is weighted to obtain the first frequency domain change of the basic spectrum. Optionally, each second sampling area is sampled, the frequency domain change mean of each second sampling area is calculated, and then the frequency domain change mean of each second sampling area is weighted to obtain the second frequency domain change of the enhanced spectrum.

[0104] Optionally, a frequency domain and time domain change curve is drawn according to the first frequency domain change of the basic spectrum graph of the basic picture layer, and a frequency domain and time domain change curve is drawn according to the second frequency domain change of the enhanced spectrum graph of the enhanced picture layer. Figure 6 According to the frequency domain and time domain change curves of the basic spectrum graph and the enhanced spectrum graph, the delay difference between the basic picture layer and the enhanced picture layer is determined.

[0105] Optionally, the frequency domain curve of the enhanced picture layer and the frequency domain curve of the basic picture layer are compared based on the time domain, such as Figure 7 As shown, the delay difference of the two frequency domain curves is calculated, and the delay difference of the basic picture layer is compensated in the enhanced picture layer, so that the frequency domain curves of the picture layer based on the time domain overlap, that is, the alignment of the picture layer in the time domain is completed. Optionally, the delay difference calculation method is shown in the following formula, assuming that u is the time domain coordinate value, v is the frequency domain coordinate value, the frequency domain and time domain curve functions of the basic picture layer are v1=f(u1); the frequency domain and time domain curve functions of the enhanced picture layer are v2=f(u2); when v1=v2, the delay difference R=u1-u2.

[0106] In the technical solution of this embodiment, the enhanced picture layer and the basic picture layer are aligned in time domain through the delay difference, and the enhanced picture layer after time domain alignment is superimposed on the basic picture layer to form a complete video picture layer on the first terminal. The first terminal and the second terminal have different playback effects, and the playback effect of the video playback is improved in the first terminal.

[0107] Reference Figure 8 , Figure 8 This is a third embodiment of the video playback method of the present invention, based on the first or second embodiment, step S10 includes:

[0108] Step S11, after the second terminal plays the video content, a binding relationship between the second terminal and the first terminal is established, and a connection channel between the second terminal and the first terminal is established;

[0109] Step S12: Pull up the enhanced picture layer of the video content based on the binding relationship and the connection channel.

[0110] Optionally, before step S10, it also includes: the second terminal initiates a play request through the user ID, device ID, and content ID, the content ID is converted into a video bit rate file ID after being mapped by the media resource system, and then the corresponding TS (Transport Stream) file is pulled through the bit rate file, the TS file encapsulates the I frame corresponding to the video content, and the I frame is decoded to obtain the corresponding video content played by the second terminal. The large-screen player of the second terminal initiates a play request through the user ID, device ID, and content ID, the content ID is converted into a corresponding audio bit rate file ID after being mapped by the media resource, and then the audio stream corresponding to the video content is pulled through the bit rate file. When decoding the audio stream, the large-screen player uses the timestamp of the video I frame as the reference value to synchronously play the corresponding audio clip in the audio stream, thereby realizing the synchronous broadcast of the video image and the audio in separate channels.

[0111] Optionally, the second terminal periodically reports the content ID and video frame ID of the currently playing video content to the server. Optionally, the synchronization time period from the second terminal to the server can be dynamically adjusted according to the pressure configuration of the server, for example, the synchronization time period is to report one message to the server when the video starts playing, report one message to the server every five minutes thereafter, and report one message to the server when the video ends.

[0112] Optionally, a binding relationship is established between the second terminal and the first terminal, wherein the binding relationship includes code scanning binding, local area network search device binding, voiceprint binding, etc. Fig. 9 As shown, an association relationship is established through the device IDs of the second terminal and the first terminal, and a subordinate relationship is established between the second terminal, the first terminal and the user, as shown in the following table:

[0113]

[0114] Optionally, the first terminal queries the server for the large-screen device ID corresponding to the user ID through the user ID, thereby displaying a bound device list and the current usage status of the second terminal, wherein the usage status includes the played content and progress.

[0115] Optionally, the first terminal queries the device ID of the second terminal corresponding to the user ID on the server through the user ID, and sends a real-time query instruction to query the video content ID currently being played by the device of the second terminal and the timestamp of the current corresponding I frame, and returns the acquired content ID and the timestamp of the corresponding I frame to the first terminal, so that the first terminal obtains the accurate playback time progress.

[0116] Optionally, the second terminal corresponding to the first terminal is determined based on the user identity and the binding relationship; the video content being played by the second terminal is obtained; and the enhanced picture layer of the video content is determined based on local attribute characteristics and the video content being played by the second terminal, wherein the attribute characteristics include at least one of user attributes, device attributes, network attributes, and regional attributes.

[0117] The binding relationship of the second terminal includes the device ID of the second terminal and the device ID of the first terminal, and a connection channel is established between the second terminal and the first terminal, wherein the connection channel may be a long connection message channel, and the second terminal transmits the acquired message back to the first terminal in real time. The first terminal obtains the enhanced picture layer of the video based on the returned video content ID and the attribute characteristics of the first terminal to obtain a richer playback experience.

[0118] Among them, the returned video content ID is a unified content ID. After the first terminal obtains the video content ID, the first terminal carries the personalized information of the first terminal. The personalized information includes user ID, device ID, content ID, total content length, timestamp of the current video frame, device model, network type, network speed, location information, power information, screen information, and client version information. The first terminal requests the content server according to the personalized information and obtains the enhanced picture layer from the content server.

[0119] Optionally, the enhanced picture layer of the video is generated in real time by the content server according to the attribute characteristics of the client, wherein the attribute characteristics include user attributes, device attributes, network attributes and / or regional attributes. Exemplarily, user attributes include user ID, device ID, etc., device attributes include device model, battery power, screen information, client version information, system version, etc., and network attributes include network type, network rate, etc. Regional attributes include location information, etc. For each type of attribute characteristics, attribute labels are unified, and encoding and quantization assignment is performed for each attribute label, wherein the encoding algorithm uses one-hot encoding. The weight factors of the attribute labels are composed into vectors, and the four types of vectors are composed into vector matrices.

[0120] Optionally, a user vector is determined based on local attribute characteristics; multiple picture sub-layers of each layer are obtained based on the video content being played by the second terminal, and a layer vector of each picture sub-layer is determined; the correlation between the picture sub-layer and the local area is determined based on the user vector and the layer vector; and an enhanced picture layer of the video content is determined based on the correlation.

[0121] Optionally, the user vector is determined based on local attribute characteristics, wherein the user attributes include user attribute tags obtained based on the user ID. For example, attribute tags include male, sports, literary youth, etc. Optionally, device attributes include device model, brand, system version, plane size, battery power, etc. For example, attribute tags include young, fashionable, sports, etc. Optionally, network attributes include 5G, 4G, network speed range, etc. For example, the network speed range includes 0-1M, 1M-5M, and above 5M. For example, the attribute tags are high-speed network, 5G, large traffic, etc. Optionally, regional attributes include positioning information, IP address. The user attribute features of the four dimensions of user attributes, device attributes, network attributes and regional attributes are matched with a coded quantized factor for each tag, and the quantized usage tags are combined into a tag vector, thereby forming a vector matrix of four-dimensional attribute tags, namely the user vector.

[0122] Optionally, multiple picture sub-layers of each layer are obtained according to the video content being played by the second terminal, and a coding quantization is assigned to the label of each layer according to the label corresponding to the picture sub-layer. Each picture sub-layer of each layer forms a label vector, i.e., a layer vector. The label vectors of each sub-layer of each layer are used to perform correlation operations with the user vector to obtain the correlation feature value, and the correlation feature values ​​are sorted by size. Then, the sub-layer pictures of the layer with higher personalized correlation are pushed to the user according to the sorting priority. The correlation operation algorithm is shown in the following formula:

[0123]

[0124] Among them, r is the cross-correlation result, y is the label vector of the layer, and x is the label vector of the user.

[0125] In the technical solution of this embodiment, after the video content is played on the second terminal, a binding relationship is established between the second terminal and the first terminal, and a connection channel is established between the second terminal and the first terminal; based on the binding relationship and the connection channel, the enhanced picture layer of the video content is pulled up, so that the first terminal obtains the enhanced picture layer, which facilitates the subsequent superposition of the enhanced picture layer and the basic picture layer, and forms a complete video picture layer on the first terminal. The first terminal and the second terminal have different playback effects, and the playback effect of the video playback is improved in the first terminal.

[0126] Reference Fig.10 The present invention further provides a video playback device, the video playback device comprising:

[0127] The acquisition module 100 is used to acquire the basic picture layer of the second terminal and pull the enhanced picture layer of the video content;

[0128] The superposition module 200 is used to perform temporal alignment between the basic picture layer and the enhanced picture layer;

[0129] The playing module 300 is used to superimpose the basic picture layer and the enhanced picture layer after time domain alignment to obtain superimposed enhanced video content, and play the enhanced video content.

[0130] Optionally, the step of performing temporal alignment of the basic picture layer and the enhanced picture layer comprises:

[0131] Determining a time delay difference between the basic picture layer and the enhanced picture layer;

[0132] According to the delay difference, the enhanced picture layer and the basic picture layer are aligned in time domain.

[0133] Optionally, the step of determining the time delay difference between the basic picture layer and the enhanced picture layer includes:

[0134] Determine a basic frequency spectrum graph corresponding to the basic picture layer and an enhanced frequency spectrum graph of the enhanced picture layer;

[0135] Determine a first frequency domain variation of the basic spectrum graph as it varies with the time domain, and a second frequency domain variation of the enhanced spectrum graph as it varies with the time domain;

[0136] A time delay difference between the basic picture layer and the enhanced picture layer is determined according to the first frequency domain change value and the second frequency domain change amount.

[0137] Optionally, the step of determining a first frequency domain variation of the basic spectrum graph as it varies with time domain, and a second frequency domain variation of the enhanced spectrum graph as it varies with time domain comprises:

[0138] Determine a first sampling region of the basic spectrum graph and a second sampling region of the enhanced spectrum graph;

[0139] Determine the first frequency domain variation of the basic spectrum graph according to a frequency domain variation mean of the first sampling region as it varies in the time domain;

[0140] The second frequency domain variation of the enhanced frequency spectrum is determined according to a frequency domain variation mean of the second sampling area varying with time domain.

[0141] Optionally, the step of pulling the enhanced picture layer of the video content includes:

[0142] After the second terminal plays the video content, a binding relationship between the second terminal and the first terminal is established, and a connection channel between the second terminal and the first terminal is established;

[0143] Based on the binding relationship and the connection channel, the enhanced picture layer of the video content is pulled up.

[0144] Optionally, the step of pulling up the enhanced picture layer of the video content based on the binding relationship and the connection channel includes:

[0145] Determine, according to the user identity and the binding relationship, a second terminal corresponding to the first terminal;

[0146] Obtaining the video content being played by the second terminal and a timestamp corresponding to the video content;

[0147] An enhanced picture layer of the video content is determined based on local attribute characteristics, the video content being played by the second terminal and a timestamp corresponding to the video content, wherein the attribute characteristics include at least one of user attributes, device attributes, network attributes and regional attributes.

[0148] Optionally, the step of determining the enhanced picture layer of the video content according to the local attribute characteristics, the video content being played by the second terminal, and the timestamp corresponding to the video content includes:

[0149] Determine the user vector based on local attribute features;

[0150] Acquire multiple picture sub-layers of each layer according to the video content being played by the second terminal and the timestamp corresponding to the video content, and determine the layer vector of each picture sub-layer;

[0151] Determine the relevance between the picture sublayer and the local area according to the user vector and the layer vector;

[0152] An enhanced picture layer of the video content is determined according to the correlation.

[0153] The present invention also provides a video playback device, which includes a memory, a processor, and a video playback program stored in the memory and executable on the processor. When the video playback program is executed by the processor, the various steps of the video playback method described in the above embodiment are implemented.

[0154] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a video playback program, and when the video playback program is executed by a processor, the steps of the video playback method described in the above embodiment are implemented.

[0155] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0156] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, system, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, system, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, system, article or device including the element.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment system can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a parking management device, an air conditioner, or a network device, etc.) to execute the system described in each embodiment of the present invention.

[0158] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A video playback method, characterized in that: Applied to the first terminal, the video playing method includes: Acquire a basic picture layer of the second terminal, and pull an enhanced picture layer of the video content, wherein the display effects of the basic picture layer and the enhanced picture layer are different; The basic picture layer and the enhanced picture layer are aligned in time domain, including: determining a delay difference according to picture contents of the basic picture layer and the enhanced picture layer, and aligning the basic picture layer and the enhanced picture layer in time domain according to the delay difference; The basic picture layer and the enhanced picture layer after time domain alignment are superimposed to obtain superimposed enhanced video content, and the enhanced video content is played so that the first terminal and the second terminal have different playback effects.

2. The video playback method according to claim 1, wherein: The step of determining the delay difference according to the picture contents of the basic picture layer and the enhanced picture layer comprises: Determine a basic frequency spectrum graph corresponding to the basic picture layer and an enhanced frequency spectrum graph of the enhanced picture layer; Determine a first frequency domain variation of the basic spectrum graph as it varies with the time domain, and a second frequency domain variation of the enhanced spectrum graph as it varies with the time domain; A time delay difference between the basic picture layer and the enhanced picture layer is determined according to the first frequency domain change value and the second frequency domain change amount.

3. The video playback method according to claim 2, characterized in that: The step of determining a first frequency domain variation of the basic spectrum graph that varies with time domain and a second frequency domain variation of the enhanced spectrum graph that varies with time domain comprises: Determine a first sampling region of the basic spectrum graph and a second sampling region of the enhanced spectrum graph; Determine the first frequency domain variation of the basic spectrum graph according to a frequency domain variation mean of the first sampling area as it varies in the time domain; The second frequency domain variation of the enhanced frequency spectrum is determined according to a frequency domain variation mean of the second sampling area varying with time domain.

4. The video playback method according to claim 1, wherein: The step of pulling the enhanced picture layer of the video content includes: After the second terminal plays the video content, a binding relationship between the second terminal and the first terminal is established, and a connection channel between the second terminal and the first terminal is established; Based on the binding relationship and the connection channel, the enhanced picture layer of the video content is pulled up.

5. The video playback method according to claim 4, characterized in that: The step of pulling up the enhanced picture layer of the video content based on the binding relationship and the connection channel includes: Determine, according to the user identity and the binding relationship, a second terminal corresponding to the first terminal; Acquire the video content being played by the second terminal; An enhanced picture layer of the video content is determined according to local attribute characteristics and the video content being played by the second terminal, wherein the attribute characteristics include at least one of user attributes, device attributes, network attributes and regional attributes.

6. The video playback method according to claim 5, characterized in that: The step of determining the enhanced picture layer of the video content according to the local attribute characteristics, the video content being played by the second terminal and the timestamp corresponding to the video content comprises: Determine the user vector based on local attribute features; Acquire multiple picture sub-layers of each layer according to the video content being played by the second terminal, and determine a layer vector of each picture sub-layer; Determine the relevance between the picture sublayer and the local area according to the user vector and the layer vector; An enhanced picture layer of the video content is determined according to the correlation.

7. A video playback device, characterized in that: Applied to a first terminal, the video playback device comprises: An acquisition module, used to acquire a basic picture layer of the second terminal and pull an enhanced picture layer of the video content, wherein the display effects of the basic picture layer and the enhanced picture layer are different; The superposition module is used to perform time domain alignment on the basic picture layer and the enhanced picture layer, including: determining a delay difference according to picture contents of the basic picture layer and the enhanced picture layer, and performing time domain alignment on the basic picture layer and the enhanced picture layer according to the delay difference; The playing module is used to superimpose the basic picture layer and the enhanced picture layer after time domain alignment to obtain the superimposed enhanced video content, and play the enhanced video content so that the first terminal and the second terminal have different playing effects.

8. A video playback device, characterized in that: The video playback device includes a memory, a processor, and a video playback program stored in the memory and executable on the processor. When the video playback program is executed by the processor, each step of the video playback method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a video playback program, and when the video playback program is executed by the processor, each step of the video playback method according to any one of claims 1 to 6 is implemented.

Images