Video playing method and device, computer equipment and computer readable storage medium

Abstract

The invention discloses a video playing method and apparatus, a device and a storage medium. The method comprises the steps of obtaining to-be-played video data and thumbnail data corresponding to the to-be-played video data; determining a target timestamp based on the thumbnail data and the user preference information; and playing the to-be-played video data based on the target timestamp. By adopting the method provided by the invention, the video playing can be accelerated while the continuity of the video playing time is kept.

Description

Technical Field

[0001] This application relates to the field of video playback technology, specifically to a video playback method, apparatus, computer device, and computer-readable storage medium. Background Technology

[0002] With the explosive growth of video data and the rapid development of mobile internet and IoT technologies, users' demands for efficient and personalized video content access are becoming increasingly prominent. While high-speed internet and smart devices have improved video accessibility, the sheer volume of content has led to information overload for users. To enable users to quickly browse video content, existing methods typically employ video summarization techniques to extract video summaries. However, video summaries extracted using these techniques often introduce temporal discontinuities, making it difficult for users to understand the context between segments. Summary of the Invention

[0003] This application provides a video playback method, apparatus, computer device, and computer-readable storage medium that can accelerate video playback while maintaining the continuity of video playback time.

[0004] The technical solution adopted by this invention to solve the problem is as follows: Firstly, this application provides a video playback method, including: Get the video data to be played and the thumbnail data corresponding to the video data to be played; Determine the target timestamp based on thumbnail data and user preference information; Play the video data to be played based on the target timestamp.

[0005] In some embodiments of this application, the thumbnail data includes multiple candidate thumbnails, and a target timestamp is determined based on the thumbnail data and user preference information, including: Input thumbnail data into an image classification model, and the image classification model outputs the image category information corresponding to each candidate thumbnail. Based on image category information and user preference information, a target thumbnail is determined from multiple candidate thumbnails; the target thumbnail is the candidate thumbnail whose image category information matches the user preference information among the multiple candidate thumbnails. The timestamp corresponding to the target thumbnail in the video data to be played is determined as the target timestamp.

[0006] In some embodiments of this application, the image classification model includes: a feature extraction module, an attention module, a pooling module, a fully connected module, a normalization module, and an activation module; The feature extraction module's input is configured to receive thumbnail data. The feature extraction module's output is connected to the attention module's input, the attention module's output is connected to the pooling module's input, the pooling module's output is connected to the fully connected module's input, the fully connected module's output is connected to the normalization module's input, and the normalization module's output is connected to the activation module's input. The feature extraction module is configured to extract features from thumbnail data; The attention module is configured to perform attention computation on the feature representation output by the feature extraction module; The pooling module is configured to perform pooling on the feature representations output by the attention module; The fully connected module is configured to perform a non-linear mapping on the feature representation output by the pooling module; The normalization module is configured to normalize the feature representations output by the fully connected module; The activation module is configured to perform a nonlinear transformation on the feature representation output by the normalization module to obtain the image category information corresponding to each candidate thumbnail.

[0007] In some embodiments of this application, the attention module includes: a first attention unit, a second attention unit, a third attention unit, and a feature fusion module; The outputs of the first attention unit, the second attention unit, and the third attention unit are connected to the output of the feature extraction module, and the outputs of the first attention unit, the second attention unit, and the third attention unit are connected to the input of the feature fusion module. The first attention unit, the second attention unit, and the third attention unit are respectively configured to perform attention calculations on the feature representations output by the feature extraction module; The feature fusion module is configured to perform an average operation on the feature representations output by the first attention unit, the second attention unit, and the third attention unit.

[0008] In some embodiments of this application, the first attention unit includes: a first dimension transpose unit, a first pooling unit, a first fully connected unit, a first activation unit, a first fusion unit, and a second dimension transpose unit; The input of the first dimension transpose unit is connected to the output of the feature extraction module, the output of the first dimension transpose unit is connected to the input of the first pooling unit, the output of the first pooling unit is connected to the input of the first fully connected unit, the output of the first fully connected unit is connected to the input of the first activation unit, the output of the first activation unit and the output of the first dimension transpose unit are respectively connected to the input of the first fusion unit, and the output of the first fusion unit is connected to the input of the second dimension transpose unit. The first dimension transpose unit is configured to transpose the feature representation output by the feature extraction module. The first pooling unit is configured to perform pooling processing on the feature representation output by the first dimension transpose unit; The first fully connected unit is configured to perform weighted calculations on the feature representation output by the first pooling unit; The first activation unit is configured to normalize the feature representation output by the first fully connected unit; The first fusion unit is configured to perform element-wise multiplication of the feature representation output by the first activation unit and the feature representation output by the first dimension transpose unit; The second dimension transpose unit is configured to transpose the feature representation output by the first fusion unit.

[0009] In some implementations of this application, the video data to be played is played based on a target timestamp, including: Based on the target timestamp, determine the playback speed corresponding to each video frame in the video data to be played; The video data to be played is played based on the playback speed corresponding to each video frame.

[0010] In some embodiments of this application, the playback speed corresponding to each video frame in the video data to be played is determined based on the target timestamp, including: For any video frame in the video data to be played, when the timestamp of the video frame in the video data to be played is within the time range corresponding to the target timestamp, the playback speed corresponding to the video frame is determined as the first speed; When the timestamp of the video frame in the video data to be played is not within the time range corresponding to the target timestamp, the playback speed corresponding to the video frame is determined to be the second speed; wherein, the first speed is less than the second speed.

[0011] Secondly, embodiments of the present invention also provide a video playback device, comprising: The data acquisition module is used to acquire the video data to be played and the thumbnail data corresponding to the video data to be played. The timestamp determination module is used to determine the target timestamp based on thumbnail data and user preference information; The video playback module is used to play the video data to be played based on the target timestamp.

[0012] Thirdly, this application also provides a computer device, which includes: One or more processors; Memory; and One or more applications, wherein the applications are stored in memory and configured to be executed by a processor to implement the video playback method of any of the first aspects.

[0013] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps of the video playback method of any of the first aspects.

[0014] The beneficial effects of this invention are as follows: By acquiring the video data to be played and the corresponding thumbnail data, determining the target timestamp based on the thumbnail data and user preference information, and playing the video data to be played based on the target timestamp, the video playback can be accelerated while maintaining the continuity of the video playback time. In addition, determining the target timestamp based on the thumbnail data and user preference information can reduce the amount of computation and increase the user's interest in the video data to be played by combining user preferences. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a video playback system provided in an embodiment of the present invention; Figure 2 This is a schematic flowchart of an embodiment of the video playback method provided by the present invention; Figure 3 This is a schematic diagram of a specific embodiment of the process for determining a target timestamp provided in this invention. Figure 4 This is a schematic diagram of the structure of the image classification model provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the attention module provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of a specific embodiment of the attention module provided in this invention; Figure 7 This is a schematic diagram of a specific embodiment of playing video data to be played, provided by an embodiment of the present invention; Figure 8 This is a schematic block diagram of a specific embodiment of the video playback device provided in this invention; Figure 9 This is a schematic diagram of an embodiment of the computer device provided in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more features.

[0019] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0020] It should be noted that since the method in this application embodiment is executed in a computer device, the processing objects of each computer device exist in the form of data or information, such as time, which is essentially time information. It is understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data that exist so that the computer device can process them. Specific details will not be elaborated here.

[0021] This application provides a video playback method, apparatus, computer device, and computer-readable storage medium, which will be described in detail below.

[0022] Please see Figure 1 , Figure 1 This is a schematic diagram of a video playback system provided in an embodiment of this application. The video playback system may include a computer device and a video service provider (VSP). The computer device and the video service provider (VSP) are connected in communication. The computer device integrates a video playback device.

[0023] In this embodiment, the Video Service Provider (VSP) is the entity that provides video-related services. The Video Service Provider (VSP) includes one or more of the following: integrated streaming media platforms, short video / live streaming platforms, vertical content platforms, video modules of cloud service providers, professional video technology service providers, edge computing video service providers, broadcasting platforms, IPTV / OTT services of telecom operators, enterprise training platforms, and internal live / on-demand platforms of institutions. (Refer to...) Figure 1 As shown, a Video Service Provider (VSP) hosts two key repositories: a database of thumbnail containers for storing the visual representation of videos, and a database of video clips containing the actual video content in various formats, including HLS, DASH, AAC, and MP4. Upon receiving a clip request carrying a video ID from a computer device, the VSP can transmit the video data to be played and its corresponding thumbnail data to the computer device at a very low bitrate.

[0024] In this embodiment, the computer device is mainly used to acquire the video data to be played and the corresponding thumbnail data; determine the target timestamp based on the thumbnail data and user preference information; and play the video data to be played based on the target timestamp, which can accelerate video playback while maintaining the continuity of video playback time. In addition, determining the target timestamp based on the thumbnail data and user preference information can reduce the amount of computation and increase the user's interest in the video data to be played by combining user preferences.

[0025] In this embodiment, the computer device can be a standalone server, a server network, or a server cluster. For example, the computer device described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0026] It is understood that the computer device used in the embodiments of this application can be a device that includes both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a device may include: cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the computer device may be a desktop terminal or a mobile terminal, and may also be one of a smart TV, mobile phone, tablet computer, laptop computer, etc.

[0027] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of computer devices shown is more or less, for example Figure 1 The video playback system shown includes a video service provider and computer equipment. It can be understood that the video playback system may also include only computer equipment, that is, video data and the corresponding thumbnail data may also be directly stored in the computer equipment. The specifics are not limited here.

[0028] In addition, the video playback system may also include a memory for storing data, such as image data, such as video data to be played, thumbnail data, etc., and timestamps, such as target timestamps, etc.

[0029] It should be noted that, Figure 1 The schematic diagram of the video playback system shown is merely an example. The video playback system and scenario described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of video playback systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0030] First, this application provides a video playback method. The execution subject of the video playback method is a video playback device, which is applied to a computer device. The video playback method includes: acquiring video data to be played and thumbnail data corresponding to the video data to be played; determining a target timestamp based on the thumbnail data and user preference information; and playing the video data to be played based on the target timestamp.

[0031] like Figure 2 The diagram shown is a flowchart of an embodiment of the video playback method in this application. This video playback method is applied to a computer device and may include the following steps S201 to S203, as detailed below: Step S201: Obtain the video data to be played and the thumbnail data corresponding to the video data to be played.

[0032] In this embodiment, the video data to be played is the video data that needs to be played. The video data to be played can be complete video data or video segments from complete video data. The thumbnail data corresponding to the video data to be played refers to static image data extracted from the original video frames of the video data to be played and generated through compression.

[0033] Optionally, the computer device can obtain the video data to be played and the corresponding thumbnail data from a local cache. Alternatively, the computer device can obtain the video data to be played and the corresponding thumbnail data from other computer devices via network, Bluetooth, infrared, or other means. For example, refer to... Figure 1 As shown, when a computer device receives a playback instruction for a video segment to be played, it sends a segment retrieval request containing the video ID to the video service provider (VSP). The video service provider (VSP) returns the video segment to be played and the corresponding thumbnail data to the computer device based on the segment retrieval request.

[0034] Step S202: Determine the target timestamp based on thumbnail data and user preference information.

[0035] In this embodiment, user preference information is used to characterize the video frame types preferred by the target user. The target user is the user requesting playback of the video data to be played. The video frame types include one or more of the following: soccer shot, soccer juggling, running, vehicle movement, pedestrian crossing, and traffic congestion. Optionally, user preference information can be directly input by the user through an input device (e.g., mouse, keyboard, touchscreen, etc.), or it can be implicitly inferred from the target user's historical video viewing records. This embodiment does not limit this approach.

[0036] Furthermore, the target timestamp is used to characterize the timestamp corresponding to the video frame type preferred by the target user in the video data to be played. For example, if the user preference information is "football shot," then the target timestamp is used to characterize the timestamp corresponding to the video frame containing the football shot in the video data to be played. Optionally, the target timestamp can be determined based on the image category information and user preference information corresponding to each thumbnail in the thumbnail data. The target timestamp can also be determined by calculating the similarity between each thumbnail in the thumbnail data and the preference image corresponding to the user preference information. This embodiment does not limit this.

[0037] In some embodiments, thumbnail data includes multiple candidate thumbnails, referencing Figure 3 As shown, the step S202 above, which determines the target timestamp based on thumbnail data and user preference information, may include steps S301 to S303, as detailed below: S301. Input the thumbnail data into the image classification model, and output the image category information corresponding to each candidate thumbnail through the image classification model.

[0038] In this embodiment of the application, the image classification model is a pre-trained neural network model used to output image category information corresponding to each candidate thumbnail in the thumbnail data based on the thumbnail data. The input end of the image classification model is configured to receive thumbnail data, and the output end of the image classification model is configured to output image category information corresponding to each candidate thumbnail in the thumbnail data.

[0039] In some embodiments, refer to Figure 4As shown, the image classification model includes: a feature extraction module, an attention module, a pooling module, a fully connected module, a normalization module, and an activation module. The input of the feature extraction module is configured to receive thumbnail data. The output of the feature extraction module is connected to the input of the attention module, the output of the attention module is connected to the input of the pooling module, the output of the pooling module is connected to the input of the fully connected module, the output of the fully connected module is connected to the input of the normalization module, the output of the normalization module is connected to the input of the activation module, and the output of the activation module is configured to output the image category information corresponding to each candidate thumbnail. The feature extraction module is configured to extract features from the thumbnail data; the attention module is configured to perform attention calculation on the feature representation output by the feature extraction module; the pooling module is configured to perform pooling processing on the feature representation output by the attention module; the fully connected module is configured to perform non-linear mapping on the feature representation output by the pooling module; the normalization module is configured to normalize the feature representation output by the fully connected module; and the activation module is configured to perform non-linear transformation on the feature representation output by the normalization module to obtain the image category information corresponding to each candidate thumbnail. This application uses a feature extraction module to extract basic visual features (e.g., edges, textures, shapes, etc.) from each thumbnail in the thumbnail data. An attention module suppresses redundant features, allowing subsequent modules to focus on valuable information. A pooling module reduces feature dimensionality while preserving global information of key features, reducing the computational load of subsequent fully connected modules, avoiding model overfitting, and improving operational efficiency. The nonlinear mapping of the fully connected module transforms the pooled low-dimensional features into feature representations directly related to image categories. A normalization module constrains feature values ​​within a reasonable range, avoiding gradient vanishing / exploding problems caused by excessively large / small feature values. An activation module converts the normalized feature output into probability values ​​for each candidate thumbnail belonging to different categories, ultimately obtaining clear and quantifiable image category information.

[0040] Optionally, the feature extraction module can be built based on the EfficientNetV2-S network model, the MobileNet series models, the SqueezeNet series models, the EfficientNet-Lite series models, the ResNet series models, etc. In some embodiments, the feature extraction module is built based on the EfficientNetV2-S network model. The feature extraction module based on the EfficientNetV2-S network model has a good accuracy-to-computation ratio when extracting high-level semantic representations from thumbnails, while maintaining low computational resource consumption.

[0041] Optionally, the pooling module can be built based on an average pooling layer or a max pooling layer. In some embodiments, the pooling module is built based on a global average pooling (GAP) layer. The global average pooling layer can significantly reduce parameters and suppress overfitting by averaging all elements of each feature map (rather than aggregating within a local window), and can also preserve global semantic information and enhance feature consistency.

[0042] Optionally, the normalization module can be built based on a layer normalization (LN) layer, a batch normalization (BN) layer, or a group normalization (GN) layer; this embodiment does not limit the scope. In some embodiments, the normalization module is built based on a batch normalization (BN) layer. Optionally, the activation module can be constructed based on the Softmax activation function, the ReLU activation function, or other activation functions such as GELU and PReLU. In some embodiments, the activation module can be constructed based on the Softmax activation function. The Softmax activation function can transform the feature representation output by the normalization module into a probability distribution of multiple candidate classification results. For example, the Softmax activation function can transform the feature representation output by the normalization module into a probability distribution of 10 candidate classification results. Each value in the probability distribution of these 10 candidate classification results represents the predicted probability of the corresponding candidate classification result. The candidate classification result with the highest predicted probability value among these 10 candidate classification results is the image category information corresponding to the candidate thumbnail.

[0043] In some embodiments, refer to Figure 5As shown, the attention module includes a first attention unit, a second attention unit, a third attention unit, and a feature fusion module. The outputs of the first, second, and third attention units are connected to the output of the feature extraction module, and their outputs are connected to the input of the feature fusion module. The first, second, and third attention units are configured to perform attention calculations on the feature representations output by the feature extraction module. The feature fusion module is configured to perform an averaging operation on the feature representations output by the first, second, and third attention units. This embodiment uses the first, second, and third attention units to capture cross-dimensional interactive features in the feature space, and the feature fusion module can fuse different cross-dimensional interactive features, improving the accuracy of the determined image classification information.

[0044] In some embodiments, continue to refer to Figure 5 As shown, the first attention unit includes: a first dimension transpose unit, a first pooling unit, a first fully connected unit, a first activation unit, a first fusion unit, and a second dimension transpose unit. The input of the first dimension transpose unit is connected to the output of the feature extraction module, the output of the first dimension transpose unit is connected to the input of the first pooling unit, the output of the first pooling unit is connected to the input of the first fully connected unit, the output of the first fully connected unit is connected to the input of the first activation unit, the outputs of the first activation unit and the first dimension transpose unit are respectively connected to the input of the first fusion unit, and the output of the first fusion unit is connected to the input of the second dimension transpose unit. The first dimension transpose unit is configured to transpose the feature representation output by the feature extraction module. The first pooling unit is configured to pool the feature representation output by the first dimension transpose unit. The first fully connected unit is configured to perform weighted calculation on the feature representation output by the first pooling unit. The first activation unit is configured to normalize the feature representation output by the first fully connected unit. The first fusion unit is configured to perform element-wise multiplication on the feature representation output by the first activation unit and the feature representation output by the first dimension transpose unit. The second dimension transpose unit is configured to transpose the feature representation output by the first fusion unit. For example, refer to... Figure 6As shown, the feature representation output by the feature extraction module is C×H×W, where C represents the number of channels, H represents the height, and W represents the width. The first dimension transpose unit can convert the feature representation output by the feature extraction module into a W×H×C feature representation. After the feature representation output by the first dimension transpose unit is processed by the first pooling unit, the first fully connected unit, the first activation unit, and the first fusion unit, it is then converted into a C×H×W feature representation by the second dimension transpose unit. This enables interactive feature extraction between the width and channel dimensions.

[0045] In some embodiments, continue to refer to Figure 5 As shown, the second attention unit includes: a third-dimensional transpose unit, a second pooling unit, a second fully connected unit, a second activation unit, a second fusion unit, and a fourth-dimensional transpose unit. The input of the third-dimensional transpose unit is connected to the output of the feature extraction module, the output of the third-dimensional transpose unit is connected to the input of the second pooling unit, the output of the second pooling unit is connected to the input of the second fully connected unit, the output of the second fully connected unit is connected to the input of the second activation unit, the outputs of the second activation unit and the fourth-dimensional transpose unit are respectively connected to the input of the second fusion unit, and the output of the second fusion unit is connected to the input of the fourth-dimensional transpose unit. The third-dimensional transpose unit is configured to transpose the feature representation output by the feature extraction module. The second pooling unit is configured to pool the feature representation output by the third-dimensional transpose unit. The second fully connected unit is configured to perform weighted calculation on the feature representation output by the second pooling unit. The second activation unit is configured to normalize the feature representation output by the second fully connected unit. The second fusion unit is configured to perform element-wise multiplication of the feature representation output by the second activation unit and the feature representation output by the third-dimensional transpose unit. The fourth-dimensional transpose unit is configured to transpose the feature representation output by the second fusion unit. For example, refer to... Figure 6 As shown, the feature representation output by the feature extraction module is C×H×W, where C represents the number of channels, H represents the height, and W represents the width. The third-dimensional transpose unit can convert the feature representation output by the feature extraction module into an H×C×W feature representation. After the feature representation output by the third-dimensional transpose unit is processed by the second pooling unit, the second fully connected unit, the second activation unit, and the second fusion unit, it is then converted into a C×H×W feature representation by the fourth-dimensional transpose unit. In this way, interactive feature extraction between the height and channel dimensions can be achieved.

[0046] In some embodiments, continue to refer to Figure 5As shown, the third attention unit includes a third pooling unit, a third fully connected unit, a third activation unit, and a third fusion unit. The input of the third pooling unit is connected to the output of the feature extraction module; the output of the third pooling unit is connected to the input of the third fully connected unit; the output of the third fully connected unit is connected to the input of the third activation unit; and the output of the third activation unit is connected to the input of the third fusion unit. The third pooling unit is configured to pool the feature representation output by the feature extraction module; the third fully connected unit is configured to perform weighted calculations on the feature representation output by the third pooling unit; the third activation unit is configured to normalize the feature representation output by the third fully connected unit; and the third fusion unit is configured to perform element-wise multiplication between the feature representation output by the third activation unit and the feature representation input to the third pooling unit. For example, refer to... Figure 6 As shown, the feature representation output by the feature extraction module is C×H×W, where C represents the number of channels, H represents the height, and W represents the width. The feature representation output by the feature extraction module is processed by the third pooling unit, the third fully connected unit, the third activation unit, and the third fusion unit, which can realize interactive feature extraction between the height and width dimensions.

[0047] Optionally, the first, second, and third pooling units can be constructed based on a max pooling layer, or they can be constructed based on an average pooling layer, or they can be constructed based on a Z-pooling layer. For example, refer to... Figure 6 As shown, the first pooling unit, the second pooling unit, and the third pooling unit are all constructed based on the Z-pool layer. The Z-pool layer aggregates features of the dimension by combining max pooling and average pooling, which can generate a compressed feature representation.

[0048] Optionally, the first, second, and third activation modules can be constructed based on the Sigmoid activation function, or they can be constructed based on the Softmax activation function. Furthermore, the first, second, and third activation modules can also be constructed based on the ReLU activation function, GELU activation function, PReLU activation function, etc. For example, refer to... Figure 6 As shown, the first activation module, the second activation module, and the third activation module are all constructed based on the Sigmoid activation function.

[0049] In some embodiments, the fully connected module includes a fourth fully connected unit and a fifth fully connected unit. The input of the fourth fully connected unit is connected to the output of the pooling module, and the output of the fourth fully connected unit is connected to the input of the fifth fully connected unit. The fourth fully connected unit is configured to perform a nonlinear mapping on the feature representation output by the pooling module, and the fifth fully connected unit is configured to perform a nonlinear mapping on the feature representation output by the fourth fully connected unit. In this embodiment, multi-layer nonlinear mapping can be achieved through the fourth and fifth fully connected units, thus building a more powerful feature representation capability and improving the accuracy of the obtained image category information. For example, the fully connected module includes two cascaded fully connected units, each of which includes 512 neurons. Each neuron includes a ReLU activation function and a Dropout layer. The nonlinearity introduced by the ReLU activation function can enhance the model's expressive power, and the Dropout layer can randomly discard some neurons (e.g., 20% of neurons), thus preventing the model from overfitting.

[0050] In some embodiments, the image classification model is trained based on the following steps: obtaining sample image data and image category labels corresponding to the sample image data; inputting the sample image data into a preset network model and outputting the predicted image category corresponding to the sample image data through the preset network model; determining a loss value based on the predicted image category and the image category label; when the loss value does not converge or the number of updates of the preset network model is less than a preset number threshold, updating the model parameters of the preset network model based on a preset learning rate, and continuing to execute the steps of inputting sample image data into the preset network model and outputting the predicted image category corresponding to the sample image data through the preset network model, until the loss value converges or the number of updates of the preset network model is greater than or equal to the preset number threshold.

[0051] S302. Based on the image category information and user preference information, determine a target thumbnail from the multiple candidate thumbnails; the target thumbnail is a candidate thumbnail whose image category corresponding to the image category information in the multiple candidate thumbnails matches the user preference information.

[0052] In this embodiment, the target thumbnail is a candidate thumbnail among the candidate thumbnails whose image category information matches the user preference information. Matching the image category information with the user preference information can mean that the image category information and the user preference information are the same, or that the similarity between the image category information and the user preference information is higher than a similarity threshold. For example, if the user preference information is "football shot," the target thumbnail is a candidate thumbnail among the candidate thumbnails whose image category information matches "football shot."

[0053] S303. The timestamp corresponding to the target thumbnail in the video data to be played is determined as the target timestamp.

[0054] In this embodiment, the target timestamp is the timestamp corresponding to the target thumbnail in the video data to be played. For example, if the timestamp corresponding to target thumbnail A in the video data to be played is 12 minutes 20 seconds to 12 minutes 40 seconds, then 12 minutes 20 seconds to 12 minutes 40 seconds is the target timestamp. This embodiment outputs image category information corresponding to each candidate thumbnail through the image classification model, which can significantly reduce computational complexity without sacrificing semantic relevance, thereby improving video playback speed. Furthermore, based on the image category information and user preference information, a target thumbnail is determined from multiple candidate thumbnails, and the target timestamp is determined based on the timestamp corresponding to the target thumbnail. This allows for video playback based on user preference information, increasing user interest in the video data to be played.

[0055] S203. Play the video data to be played based on the target timestamp.

[0056] In this embodiment, when playing video data based on a target timestamp, the playback speed of each video frame in the video data can be determined based on the target timestamp, and the video data can be played based on the playback speed of each video frame. Alternatively, the video segment to be played can be determined based on the target timestamp and played accordingly. This embodiment determines the target timestamp based on thumbnail data and user preference information, and plays the video data based on the target timestamp. This maintains temporal continuity while accelerating video data playback. Compared to existing summary generation methods, it reduces the temporal discontinuity of the played video data. Furthermore, determining the target timestamp based on thumbnail data and user preference information reduces computational load, and combining user preferences with video data playback increases user interest in the video data to be played.

[0057] In some embodiments, refer to Figure 7 As shown, step S203, which involves playing the video data to be played based on the target timestamp, may include steps S401 to S402, as follows: S401. Based on the target timestamp, determine the playback speed corresponding to each video frame in the video data to be played.

[0058] In this embodiment, when determining the playback speed of each video frame in the video data to be played based on the target timestamp, the playback speed of the video frame corresponding to the target timestamp in the video data to be played can be set as the first speed, and the playback speed of other video frames in the video data to be played can be set as the second speed. Alternatively, the playback speed of the video frame corresponding to the target timestamp in the video data to be played can be set as the first speed, and the playback speed of other video frames in the video data to be played can be set differently according to their distance from the video frame corresponding to the target timestamp. This embodiment does not impose any limitations.

[0059] In some embodiments, the step of determining the playback speed corresponding to each video frame in the video data to be played based on the target timestamp specifically includes: for any video frame in the video data to be played, if the timestamp of the video frame in the video data to be played is within the time range corresponding to the target timestamp, the playback speed corresponding to the video frame is determined to be a first speed; if the timestamp of the video frame in the video data to be played is not within the time range corresponding to the target timestamp, the playback speed corresponding to the video frame is determined to be a second speed; wherein, the first speed is less than the second speed. For example, if the timestamp of video frame A in the video data to be played is within the time range corresponding to the target timestamp, the playback speed corresponding to video frame A is determined to be 1.0 times the target timestamp; if the timestamp of video frame B in the video data to be played is not within the time range corresponding to the target timestamp, the playback speed corresponding to video frame B is determined to be 2.0 times the target timestamp.

[0060] S402. Play the video data to be played based on the playback speed corresponding to each video frame.

[0061] This embodiment determines the playback speed corresponding to each video frame in the video data to be played based on the target timestamp, and plays the video data to be played based on the playback speed corresponding to each video frame. It can play preferred content at normal playback speed and accelerate the playback of content that the user is not interested in, thus accelerating the video playback speed while maintaining the continuity of video playback time.

[0062] To better implement the video playback method in the embodiments of this application, based on the video playback method, the embodiments of this application also provide a video playback device, such as... Figure 8 As shown, the video playback device 600 includes: Data acquisition module 610 is used to acquire the video data to be played and the thumbnail data corresponding to the video data to be played; The timestamp determination module 620 is used to determine the target timestamp based on thumbnail data and user preference information; The video playback module 630 is used to play the video data to be played based on the target timestamp.

[0063] In this embodiment, by acquiring the video data to be played and the corresponding thumbnail data, and determining the target timestamp based on the thumbnail data and user preference information, the video data to be played is played based on the target timestamp. This can accelerate video playback while maintaining the continuity of video playback time. In addition, determining the target timestamp based on thumbnail data and user preference information can reduce the amount of computation and increase the user's interest in the video data to be played by combining user preferences.

[0064] In some embodiments of this application, the thumbnail data includes multiple candidate thumbnails, and the timestamp determination module 620 is specifically used for: Input thumbnail data into an image classification model, and the image classification model outputs the image category information corresponding to each candidate thumbnail. Based on image category information and user preference information, a target thumbnail is determined from multiple candidate thumbnails; the target thumbnail is the candidate thumbnail whose image category information matches the user preference information among the multiple candidate thumbnails. The timestamp corresponding to the target thumbnail in the video data to be played is determined as the target timestamp.

[0065] In some embodiments of this application, the image classification model includes: a feature extraction module, an attention module, a pooling module, a fully connected module, a normalization module, and an activation module; The feature extraction module's input is configured to receive thumbnail data. The feature extraction module's output is connected to the attention module's input, the attention module's output is connected to the pooling module's input, the pooling module's output is connected to the fully connected module's input, the fully connected module's output is connected to the normalization module's input, and the normalization module's output is connected to the activation module's input. The feature extraction module is configured to extract features from thumbnail data; The attention module is configured to perform attention computation on the feature representation output by the feature extraction module; The pooling module is configured to perform pooling on the feature representations output by the attention module; The fully connected module is configured to perform a non-linear mapping on the feature representation output by the pooling module; The normalization module is configured to normalize the feature representations output by the fully connected module; The activation module is configured to perform a nonlinear transformation on the feature representation output by the normalization module to obtain the image category information corresponding to each candidate thumbnail.

[0066] In some embodiments of this application, the attention module includes: a first attention unit, a second attention unit, a third attention unit, and a feature fusion module; The outputs of the first attention unit, the second attention unit, and the third attention unit are connected to the output of the feature extraction module, and the outputs of the first attention unit, the second attention unit, and the third attention unit are connected to the input of the feature fusion module. The first attention unit, the second attention unit, and the third attention unit are respectively configured to perform attention calculations on the feature representations output by the feature extraction module; The feature fusion module is configured to perform an average operation on the feature representations output by the first attention unit, the second attention unit, and the third attention unit.

[0067] In some embodiments of this application, the first attention unit includes: a first dimension transpose unit, a first pooling unit, a first fully connected unit, a first activation unit, a first fusion unit, and a second dimension transpose unit; The input of the first dimension transpose unit is connected to the output of the feature extraction module, the output of the first dimension transpose unit is connected to the input of the first pooling unit, the output of the first pooling unit is connected to the input of the first fully connected unit, the output of the first fully connected unit is connected to the input of the first activation unit, the output of the first activation unit and the output of the first dimension transpose unit are respectively connected to the input of the first fusion unit, and the output of the first fusion unit is connected to the input of the second dimension transpose unit. The first dimension transpose unit is configured to transpose the feature representation output by the feature extraction module. The first pooling unit is configured to perform pooling processing on the feature representation output by the first dimension transpose unit; The first fully connected unit is configured to perform weighted calculations on the feature representation output by the first pooling unit; The first activation unit is configured to normalize the feature representation output by the first fully connected unit; The first fusion unit is configured to perform element-wise multiplication of the feature representation output by the first activation unit and the feature representation output by the first dimension transpose unit; The second dimension transpose unit is configured to transpose the feature representation output by the first fusion unit.

[0068] In some embodiments of this application, the video playback module 630 is specifically used for: Based on the target timestamp, determine the playback speed corresponding to each video frame in the video data to be played; The video data to be played is played based on the playback speed corresponding to each video frame.

[0069] In some embodiments of this application, the video playback module 630 is further configured to: For any video frame in the video data to be played, when the timestamp of the video frame in the video data to be played is within the time range corresponding to the target timestamp, the playback speed corresponding to the video frame is determined as the first speed; When the timestamp of the video frame in the video data to be played is not within the time range corresponding to the target timestamp, the playback speed corresponding to the video frame is determined to be the second speed; wherein, the first speed is less than the second speed.

[0070] This application also provides a computer device that integrates any of the video playback devices provided in this application. The computer device includes: One or more processors; Memory; and One or more applications, wherein the applications are stored in memory and configured to be executed by a processor from the steps of the video playback method in any of the above embodiments of the video playback method.

[0071] This application also provides a computer device that integrates any of the video playback devices provided in this application. For example... Figure 9 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically: The computer device may include components such as a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a power supply 803, and an input unit 804. Those skilled in the art will understand that... Figure 9 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 801 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 802, and by calling data stored in the memory 802, it performs various functions of the computer device and processes data, thereby providing overall monitoring of the computer device. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 801.

[0072] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0073] The computer device also includes a power supply 803 that supplies power to the various components. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 803 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0074] The computer device may also include an input unit 804, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0075] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 801 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 runs the application programs stored in the memory 802 to realize various functions, as follows: Get the video data to be played and the thumbnail data corresponding to the video data to be played; Determine the target timestamp based on thumbnail data and user preference information; Play the video data to be played based on the target timestamp.

[0076] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0077] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the video playback methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps: Get the video data to be played and the thumbnail data corresponding to the video data to be played; Determine the target timestamp based on thumbnail data and user preference information; Play the video data to be played based on the target timestamp.

[0078] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0079] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0080] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0081] The foregoing has provided a detailed description of a video playback method, apparatus, device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A video playback method, characterized in that, include: Obtain the video data to be played and the thumbnail data corresponding to the video data to be played; Based on the thumbnail data and user preference information, determine the target timestamp; Based on the target timestamp, the video data to be played is played.

2. The video playback method according to claim 1, characterized in that, The thumbnail data includes multiple candidate thumbnails. Determining the target timestamp based on the thumbnail data and user preference information includes: The thumbnail data is input into an image classification model, and the image classification model outputs the image category information corresponding to each candidate thumbnail. Based on the image category information and user preference information, a target thumbnail is determined from a plurality of candidate thumbnails; the target thumbnail is a candidate thumbnail whose image category corresponding to the image category information in the plurality of candidate thumbnails matches the user preference information. The timestamp corresponding to the target thumbnail in the video data to be played is determined as the target timestamp.

3. The video playback method according to claim 2, characterized in that, The image classification model includes: a feature extraction module, an attention module, a pooling module, a fully connected module, a normalization module, and an activation module; The input of the feature extraction module is configured to receive the thumbnail data. The output of the feature extraction module is connected to the input of the attention module. The output of the attention module is connected to the input of the pooling module. The output of the pooling module is connected to the input of the fully connected module. The output of the fully connected module is connected to the input of the normalization module. The output of the normalization module is connected to the input of the activation module. The feature extraction module is configured to extract features from the thumbnail data; The attention module is configured to perform attention calculations on the feature representation output by the feature extraction module; The pooling module is configured to perform pooling processing on the feature representation output by the attention module; The fully connected module is configured to perform a non-linear mapping on the feature representation output by the pooling module; The normalization module is configured to normalize the feature representation output by the fully connected module; The activation module is configured to perform a nonlinear transformation on the feature representation output by the normalization module to obtain the image category information corresponding to each candidate thumbnail.

4. The video playback method according to claim 3, characterized in that, The attention module includes: a first attention unit, a second attention unit, a third attention unit, and a feature fusion module; The output terminals of the first attention unit, the second attention unit, and the third attention unit are respectively connected to the output terminal of the feature extraction module, and the output terminals of the first attention unit, the second attention unit, and the third attention unit are respectively connected to the input terminal of the feature fusion module. The first attention unit, the second attention unit, and the third attention unit are respectively configured to perform attention calculations on the feature representation output by the feature extraction module; The feature fusion module is configured to perform an average operation on the feature representations output by the first attention unit, the second attention unit, and the third attention unit.

5. The video playback method according to claim 4, characterized in that, The first attention unit includes: a first dimension transpose unit, a first pooling unit, a first fully connected unit, a first activation unit, a first fusion unit, and a second dimension transpose unit; Wherein, the input end of the first dimension transpose unit is connected to the output end of the feature extraction module, the output end of the first dimension transpose unit is connected to the input end of the first pooling unit, the output end of the first pooling unit is connected to the input end of the first fully connected unit, the output end of the first fully connected unit is connected to the input end of the first activation unit, the output ends of the first activation unit and the first dimension transpose unit are respectively connected to the input end of the first fusion unit, and the output end of the first fusion unit is connected to the input end of the second dimension transpose unit; The first dimension transpose unit is configured to transpose the feature representation output by the feature extraction module. The first pooling unit is configured to perform pooling processing on the feature representation output by the first dimension transpose unit; The first fully connected unit is configured to perform weighted calculations on the feature representation output by the first pooling unit; The first activation unit is configured to normalize the feature representation output by the first fully connected unit; The first fusion unit is configured to perform element-wise multiplication of the feature representation output by the first activation unit and the feature representation output by the first dimension transpose unit; The second dimension transpose unit is configured to transpose the feature representation output by the first fusion unit.

6. The video playback method according to claim 1, characterized in that, Playing the video data to be played based on the target timestamp includes: Based on the target timestamp, determine the playback speed corresponding to each video frame in the video data to be played; The video data to be played is played based on the playback speed corresponding to each video frame.

7. The video playback method according to claim 6, characterized in that, The step of determining the playback speed corresponding to each video frame in the video data to be played based on the target timestamp includes: For any video frame in the video data to be played, when the timestamp of the video frame in the video data to be played is within the time range corresponding to the target timestamp, the playback speed corresponding to the video frame is determined to be the first speed; When the timestamp of the video frame in the video data to be played is not within the time range corresponding to the target timestamp, the playback speed corresponding to the video frame is determined to be the second speed; wherein, the first speed is less than the second speed.

8. A video playback device, characterized in that, include: The data acquisition module is used to acquire the video data to be played and the thumbnail data corresponding to the video data to be played; The timestamp determination module is used to determine the target timestamp based on the thumbnail data and user preference information; The video playback module is used to play the video data to be played based on the target timestamp.

9. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the video playback method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to perform the steps of the video playback method according to any one of claims 1 to 7.

Images