Video playback method and system, and device

Abstract

Disclosed in embodiments of the present invention are a video playback method and system, and a device. The method comprises: a first device sending, to a second device, a first message comprising a user ID; receiving a second message sent by the second device, wherein the second message comprises a first video source of at least one first video, a second video ID of at least one second video corresponding to the user ID, and a first relationship between the first video and the at least one second video, and the second video is a video obtained by acquiring a second video source from the second device and caching second video content to the first device on the basis of the second video source; acquiring first video content on the basis of the first video source; and playing back the first video on the basis of the first video content and the first relationship. When a video that has not been watched by a user is played back, a video that has been watched by the user can be fully utilized, thereby improving the user's video viewing experience.

Description

A video playback method, system, and device

[0001] This application claims priority to Chinese Patent Application No. 202411178099.4, filed with the State Intellectual Property Office of China on August 26, 2024, entitled “A Video Playback Method, System and Device”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of computer technology, and in particular to a video playback method, system, and device. Background Technology

[0003] The terminal device adaptively selects the video bitrate based on the current network conditions and requests the next set of videos from the server. If the videos in the next set are low-definition, it performs edge-side blind super-resolution to improve the visual effect. However, the effect of edge-side blind super-resolution is limited, and the effect of super-resolution for low-bitrate videos differs significantly from that of high-bitrate videos. Furthermore, after video playback is complete, it is cached locally for a period of time to avoid requiring users to re-request videos upon re-viewing, thus saving bandwidth. However, locally cached videos are only used when the user views videos they have already watched. Therefore, this approach does not fully utilize locally cached videos already viewed by the user, resulting in a lower user viewing experience. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a video playback method, system, and device that can make full use of locally cached videos that the user has already watched when playing videos that the user has not yet watched, thereby improving the user's video viewing experience.

[0005] In a first aspect, embodiments of the present invention provide a video playback method applied to a first device, the method comprising:

[0006] Send a first message to the second device, the first message containing the user ID;

[0007] The system receives a second message sent by the second device, the second message containing a first video source of at least one first video, a second video ID of at least one second video, and a first relationship between the first video and the at least one second video; the second video is a video for which a second video source is obtained from the second device and the content of the second video is cached on the first device according to the second video source.

[0008] Obtain the first video content based on the first video source;

[0009] The first video is played according to the first video content and the first relationship.

[0010] After receiving the first message from the first device, the second device in this embodiment of the invention not only transmits the first video source of the first video to the first device, but also transmits the second video ID of the second video and the first relationship between the first video and at least one second video. This allows the first device to utilize the first relationship between the first video and at least one second video when playing the first video, thereby improving the user's viewing experience of the first video.

[0011] This invention, when playing videos that a user has not yet watched, utilizes a first relationship between unwatched videos and locally cached videos that have already been watched to improve the image quality of unwatched videos, thereby enhancing the user's video viewing experience. Furthermore, compared to locally cached videos that have already been watched and are only used for viewing previously viewed videos, this invention makes fuller use of locally cached videos that have already been watched. Therefore, the video playback method provided by this invention can more fully utilize locally cached videos that the user has already watched, improving the user's video viewing experience.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, playing the first video based on the first video content and the first relationship includes:

[0013] The second video content is obtained based on the second video ID contained in the first relationship;

[0014] The first video is played based on the second video content, the first video content, and the first relationship.

[0015] In this embodiment of the invention, the second video ID contained in the second message may exist together with the first relationship in the second message, that is, the first relationship contains the second video ID. Therefore, when the first device obtains the second video content according to the second video ID in the second message, it specifically obtains the second video content according to the second video ID contained in the first relationship.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the first relationship includes: a similarity relationship between the segments of the first video and the segments of the at least one second video.

[0017] In this embodiment of the invention, the first relationship between the first video and the at least one second video specifically refers to the similarity relationship between the segments of the first video and the segments of the at least one second video, that is, the similarity relationship between the segments of videos that the user has not watched and the segments of at least one locally cached video that the user has watched.

[0018] In this embodiment of the invention, video segments can be understood as video shots.

[0019] In some possible embodiments, the similarity between two video segments is determined by whether the similarity between the two video segments is greater than or equal to a second threshold. If the similarity between the two video segments is greater than or equal to the second threshold, the two video segments are considered to be similar; if the similarity between the two video segments is less than the second threshold, the two video segments are considered to be dissimilar.

[0020] In some possible embodiments, when determining whether two video segments are similar, it is first necessary to determine whether the similarity between the two video segments is greater than or equal to a second threshold, such as 0.85. If the similarity is greater than or equal to 0.85, it is also necessary to check whether the segment is an isolated segment. If the segment is an isolated segment, it will not be considered a similar segment even if the similarity is greater than or equal to 0.85. If the segment is not an isolated segment, it can be considered a similar segment when the similarity is greater than or equal to 0.85.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the step of playing the first video based on the second video content, the first video content, and the first relationship includes:

[0022] The first relationship includes a second video segment that does not have a segment similar to the first video segment. Blind super-resolution is performed on the first video content to obtain third video content, and the first video is played based on the third video content; or

[0023] The first relationship includes the existence of a second video segment that is similar to a segment of the first video. The second video segment that is similar to a segment of the first video is obtained from the second video content. The first video is then guided by super-resolution based on the obtained second video segment to obtain a fourth video content. The first video is then played based on the fourth video content.

[0024] For example, assume that at least one second video includes video A, video B, and video C, and the first video is video D. Each of videos A, B, C, and D contains three segments. The first segment of video A is similar to the first segment of video D, the second segment of video B is similar to the second segment of video D, and the third segment of video C is similar to the third segment of video D. Videos A, B, and C all have a resolution of 1080p, while video D has a resolution of 480p. When performing super-resolution on video D, the first segment of video D is enhanced using the first segment of video A, the second segment of video D is enhanced using the second segment of video B, and the third segment of video D is enhanced using the third segment of video C, resulting in the super-resolution video E, which has a resolution of 1080p.

[0025] For example, the video playback method provided in this embodiment of the invention can be applied to short video playback scenarios. Before sending the next set of videos to the terminal device, the server determines whether the next set of videos has similar segments to videos already watched by the user. If similar segments are found, the server sends the next set of videos to the terminal device along with the similarity relationship between the next set of videos and the videos already watched by the user. When the next set of videos has similar videos already watched by the user, the terminal device retrieves the similar videos from its local cache, performs on-device guided super-resolution on the next set of videos, and then plays them. When the next set of videos does not have similar videos already watched by the user, the terminal device performs blind super-resolution on the next set of videos and then plays them.

[0026] Therefore, this embodiment of the invention introduces a similarity judgment between locally cached videos that the user has already watched and unwatched videos (videos to be refreshed next). When the locally cached videos that the user has already watched are similar to the unwatched videos, the locally cached videos that the user has already watched are used to enhance the unwatched videos, upgrading the terminal-side "blind super-resolution" to the terminal-side "guided super-resolution". The terminal-side guided super-resolution has a reference video, and the effect is better than that of blind super-resolution. It can provide a better picture quality experience when there is no high-definition video source.

[0027] In conjunction with the first aspect, in some implementations of the first aspect, the second message further includes: the video quality score of the first video source;

[0028] Playing the first video based on the second video content, the first video content, and the first relationship includes:

[0029] The first video is played based on the video quality score, the second video content, the first video content, and the first relationship.

[0030] The video playback method provided in this embodiment of the invention also considers the video quality of unwatched videos before playing them.

[0031] In conjunction with the first aspect, in some implementations of the first aspect, playing the first video based on the video quality score, the second video content, the first video content, and the first relationship includes:

[0032] If the video quality score is greater than a first threshold, the first video is played according to the first video content; or

[0033] The video quality score is less than or equal to a first threshold, and the first relationship includes a second video segment that does not have a segment similar to the first video segment. Blind super-resolution is then performed on the first video content to obtain third video content, and the first video is played based on the third video content; or

[0034] The video quality score is less than or equal to a first threshold, and the first relationship includes a segment of the second video that is similar to a segment of the first video. The segment of the second video that is similar to a segment of the first video is obtained from the second video content. The first video content is guided to be super-scored based on the obtained segment of the second video to obtain a fourth video content. The first video is played based on the fourth video content.

[0035] In this embodiment of the invention, the unwatched video is subjected to guided super-resolution, blind super-resolution, or no super-resolution based on the similarity between the unwatched video and the locally cached video that has been watched, and the quality of the unwatched video. Specifically, if the quality of the unwatched video is high, no super-resolution is applied, and the video is directly displayed to the user. If the quality of the unwatched video is low and there are no segments similar to the locally cached video that the user has watched, the client-side blind super-resolution model is invoked to perform blind super-resolution on the unwatched video, and the super-resolution video is displayed to the user. If the quality of the unwatched video is low and there are segments similar to the locally cached video that the user has watched, a similar segment of the locally cached video is found as a reference video based on a first relationship, and the client-side guided super-resolution model is invoked to perform guided super-resolution on the unwatched video, and the super-resolution video is displayed to the user.

[0036] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes:

[0037] The first video content is cached locally.

[0038] In this embodiment of the invention, regardless of whether the first video content is super-resolution enhanced, the first video content will be cached so that the locally cached video can be used to super-resolution enhance the unwatched video later.

[0039] In this embodiment of the invention, the first device can also cache the over-score video content locally, so that users do not need to undergo over-score again when watching it again. The missing video quality score of the over-score video content can be calculated by the first device, including model scoring or original score mapping methods, and can be used as a reference video to guide over-score of unwatched videos. Here, the video quality score refers to a quantitative score obtained by comprehensively analyzing multiple dimensions such as video clarity, signal-to-noise ratio, color, and brightness through a series of algorithms, and is used to evaluate the overall quality of the video.

[0040] In conjunction with the first aspect, in some implementations of the first aspect, after caching the first video content locally, the method further includes:

[0041] Delete the first video content;

[0042] A third message is sent to the second device, the third message being used to instruct the second device to delete the first video ID of the first video from the first list, the first list being used to store the IDs of videos for which video sources are obtained from the second device and whose video content is cached to the first device according to the video sources.

[0043] In this embodiment of the invention, if a user deletes a locally cached video that has been viewed, the terminal device will notify the server, and the server will delete the first video ID from the first list, thereby enabling the terminal device and the server to synchronously record the cached videos that the user has viewed.

[0044] Secondly, embodiments of the present invention provide a video playback method applied to a second device, the method comprising:

[0045] Receive a first message sent by a first device, the first message containing a user ID;

[0046] Based on the first message, at least one first video source of a first video, at least one second video ID of a second video, and a first relationship between the first video and the at least one second video are obtained; the second video is a video for which the second video source is obtained from the second device and the content of the second video is cached on the first device based on the second video source;

[0047] A second message is sent to the first device, the second message containing at least one second video ID, a first video source of the at least one first video, and the first relationship, so that the first device obtains the first video content according to the first video source and plays the first video according to the first video content and the first relationship.

[0048] After receiving the first message from the first device, the second device in this embodiment of the invention not only transmits the first video source of the first video to the first device, but also transmits the second video ID of the second video and the first relationship between the first video and at least one second video. This allows the first device to utilize the first relationship between the first video and at least one second video when playing the first video, thus making use of the second video and improving the user's viewing experience of the first video.

[0049] Therefore, when playing videos that the user has not yet watched, this embodiment of the invention utilizes the first relationship between unwatched videos and locally cached videos that have already been watched to improve the picture quality of unwatched videos, thereby enhancing the user's video viewing experience. Furthermore, compared to locally cached videos that have already been watched only being used for viewing videos already viewed, this embodiment of the invention makes fuller use of locally cached videos that have already been watched. Therefore, the video playback method provided by this embodiment of the invention can more fully utilize locally cached videos that the user has already watched, improving the user's video viewing experience.

[0050] In conjunction with the second aspect, in some implementations of the second aspect, obtaining, based on the first message, a first video source of at least one first video, a second video ID of at least one second video corresponding to the user ID, and a first relationship between the first video and the at least one second video includes:

[0051] Based on the first message, obtain the at least one first video and the second video ID of the at least one second video corresponding to the user ID;

[0052] The first relationship is obtained based on the first video and at least one of the second video IDs;

[0053] The first video source of the first video is determined based on the first relationship.

[0054] For example, the first message contains the user ID.

[0055] For example, a recommendation system typically includes modules such as recall, ranking (coarse ranking + fine ranking + re-ranking), and screen control. Based on user profiles, viewing history, and other information, it retrieves videos that the user may like from a massive video library and recommends them to the user for viewing.

[0056] For example, the recommendation system of the second device first determines the user profile and other information corresponding to the user ID based on the user ID, and then obtains at least one first video through the recall and sorting stages based on the user profile and other information.

[0057] For example, recall is part of the recommendation process, finding a set of videos that the user might like (e.g., 10,000 videos) from a massive video library (e.g., containing 1 billion videos), thus reducing the processing load for the sorting module.

[0058] For example, after the videos are sorted by the sorting model, some processing is done before they are displayed to the user—this is called screen control. The screen control module contains many modules. For example, the deduplication module can prevent two identical videos from being displayed to the user one after the other, which would lead to a poor user experience.

[0059] In some possible embodiments, if the first message does not contain a second video ID, the second device obtains a first list corresponding to the user ID in the first message, and obtains at least one second video ID from the first list.

[0060] In some possible embodiments, the first message contains at least one second video ID, and the second device obtains at least one second video ID from the first message based on the first message.

[0061] In conjunction with the second aspect, in some implementations of the second aspect, obtaining the first relationship based on the first video and at least one second video ID includes:

[0062] Based on the second video ID and the first video ID of the first video, query the database for similarity relationships between the segments of the second video and the segments of the first video;

[0063] or,

[0064] The second video is obtained based on the second video ID; the first video and the second video are segmented to obtain segments of the first video and segments of the second video; several first keyframes are extracted from the segments of the first video and several second keyframes are extracted from the segments of the second video; the similarity relationship between the segments of the first video and the segments of the second video is obtained by performing similarity determination on the first keyframes and the second keyframes.

[0065] For example, if a video has already undergone similarity assessment during the data entry stage, then the similarity relationship can be queried simply by using the video ID. Information feed applications typically have a massive video library on the cloud side, containing videos provided by creators. Data entry refers to the stage from when a creator publishes a video to when it is added to the distributable library, which usually involves some content understanding-related work.

[0066] For example, the second processing module of the second device queries the database, inputs the video ID of video 1 and the video ID of video 3, and outputs that video 3 has a total of 10 segments, segments 1-5 are similar to segments 2-6 of video 1, and segments 6-10 are similar to segments 3-7 of video 2.

[0067] For example, if the video has not undergone similarity judgment during the data entry stage, it needs to be calculated in real time. The general process is to first segment the video into multiple segments, extract several key frames from each segment, and then perform similarity judgment between the key frames of two videos to obtain the similarity relationship between the video segments.

[0068] In conjunction with the second aspect, in some implementations of the second aspect, determining the first video source of the first video based on the first relationship includes:

[0069] The first relationship includes the existence of a second video segment that is similar to a segment of the first video, determining that the bitrate of the first video source is a first bitrate or determining that the resolution of the first video source is a first resolution; or

[0070] The first relationship includes the absence of a second video segment that is similar to the first video segment, and the bitrate of the first video source is determined to be a second bitrate or the resolution of the first video source is determined to be a second resolution.

[0071] Wherein, the second bitrate is higher than the first bitrate; the second resolution is higher than the first resolution.

[0072] For example, if a locally cached video similar to the video to be distributed has been viewed by a user, the second device will send a low-bitrate or low-resolution video source to the first device to save bandwidth; conversely, if no locally cached video similar to the video to be distributed has been viewed by a user, the second device will send a high-bitrate or high-resolution video source to the first device.

[0073] Assuming video 4 has a maximum resolution of 1080p and a high quality score, if video 4 has no similar segments to videos 1 and 2, the second device sends a high-bitrate video source of video 4, such as 1080p, to the first device, which can then play it directly. If video 4 has similar segments to both videos 1 and 2, the second device sends a low-bitrate video source of video 4, such as 480p, to the first device.

[0074] In conjunction with the second aspect, in some implementations of the second aspect, after sending the second message to the first device, the method further includes:

[0075] The first video ID is stored in a first list, which is used to store the IDs of videos for which video sources are obtained from the second device and video content is cached on the first device according to the video sources;

[0076] The system receives a third message sent by the first device and deletes the first video ID from the first list according to the third message; the third message is generated by the first device deleting the first video content of the first video.

[0077] In this embodiment of the invention, after the second device distributes the first video source of the first video to the first device, the second device adds the first video ID to the first list. The first device obtains the first video content according to the first video source and caches the first video content locally, thereby realizing the simultaneous recording of cached videos watched by the user by the terminal device and the server.

[0078] In this embodiment of the invention, if a user deletes a locally cached video that has been viewed, the terminal device will notify the server, and the server will delete the first video ID from the first list, thereby enabling the terminal device and the server to synchronously record the cached videos that the user has viewed.

[0079] In conjunction with the second aspect, in some implementations of the second aspect, the first message also includes the network quality of the first device;

[0080] Obtaining the at least one first video and the first relationship based on the first message includes:

[0081] Retrieve multiple fifth videos based on the user ID;

[0082] A second relationship is obtained based on the IDs of the fifth video and the at least one second video; the second relationship includes similarity relationships between the segments of the fifth video and the segments of the at least one second video.

[0083] The at least one first video and the first relationship are obtained based on the network quality, the second relationship, and the plurality of fifth videos.

[0084] For example, network quality refers to the performance of a network in terms of data transmission speed, stability, reliability, and security. It directly affects the user's network experience and is used as an important indicator to measure the level of information infrastructure. Network quality assessment standards typically include bandwidth, latency, jitter, packet loss rate, network availability, and connection establishment time.

[0085] The recommendation system of the second device first determines the user profile and other information corresponding to the user ID based on the user ID. Then, based on the user profile and other information, it obtains a preliminary set of videos to be distributed through a recall phase. This set contains multiple fifth videos. The second device obtains a second relationship between the fifth video and at least one second video ID. The recommendation system of the second device then adjusts the selection of videos entering the ranking phase based on network quality and the first relationship, filtering out at least one first video and its first relationship from the multiple fifth videos and the second relationship.

[0086] In conjunction with the second aspect, in certain implementations of the second aspect, obtaining the at least one first video based on the network quality, the second relationship, and the plurality of fifth videos includes:

[0087] When the network quality is good, the fifth video, which is dissimilar to the segment of the second video, is preferentially acquired as the first video; or

[0088] When the network quality is poor, the fifth video, which is similar to the segment of the second video, is preferentially acquired as the first video.

[0089] For example, when network quality is poor, the recommendation system may give higher weight to the fifth video that has similar segments to at least one second video, prioritizing videos with similar segments. For instance, the fifth video obtained by the recommendation system based on the user's long / short-term interests is usually highly similar to the second video.

[0090] For example, when the network quality is good, the recommendation system will prioritize the fifth video, which has no similar segments to at least one second video. For instance, the fifth video obtained by the recommendation system based on sudden hot topics and interest exploration usually has a low similarity to the second video.

[0091] After the ranking phase, the recommendation system obtains a set of videos to be distributed. This set of videos contains at least one first video and its first relationship. The first relationship includes the similarity relationships between the segments of the first video and the segments of at least one second video.

[0092] The video playback method provided in this invention combines the similarity relationship between unwatched videos and locally cached videos that have already been watched with a recommendation system to balance user experience and video quality. For example, in scenarios with frequent fluctuations in network quality (such as on a high-speed train where the signal is intermittent), the priority of the recommendation system can be adjusted based on network quality and similarity relationships. For instance, when network quality is poor, the recommendation system prioritizes the fifth video, which is similar to the second video segment, and distributes the fifth video segment that is similar to the second video segment first; when network quality is good, it prioritizes the fifth video segment that is not similar to the second video segment, thereby ensuring the user's video information stream experience.

[0093] The video playback method provided in this invention allows users to smoothly watch videos without sacrificing image quality even when the network fluctuates, while also taking into account user preferences; by introducing similarity relationships into the recommendation system, it achieves the effect of balancing user experience and video quality.

[0094] Thirdly, embodiments of the present invention provide a first device, including a processor and a memory, wherein the memory is used to store a program, and when the processor runs the program, the first device performs the operations involved in the method described above.

[0095] Fourthly, embodiments of the present invention provide a second device, including a processor and a memory, wherein the memory is used to store a program, and when the processor runs the program, the second device performs the operations involved in the method described above.

[0096] Fifthly, embodiments of the present invention provide a video playback system, the system including the first device and the second device described above.

[0097] In a sixth aspect, embodiments of the present invention provide a readable storage medium storing a program that, when run by a first device, causes the first device to perform operations as described above.

[0098] In a seventh aspect, embodiments of the present invention provide a readable storage medium storing a program that, when run by a second device, causes the second device to perform operations as described above.

[0099] Eighthly, embodiments of the present invention provide a program product comprising a program that, when run on a first device or any at least one processor, causes the first device to perform the operations involved in the method described above.

[0100] In a ninth aspect, embodiments of the present invention provide a program product comprising a program that, when run on a second device or any at least one processor, causes the second device to perform the operations involved in the method described above.

[0101] The technical solutions for the video playback method, system, and device provided in this invention include: a first device sending a first message containing a user ID to a second device; receiving a second message sent by the second device, the second message containing a first video source of at least one first video, a second video ID of at least one second video corresponding to the user ID, and a first relationship between the first video and at least one second video; the second video being a video for which a second video source is obtained from the second device and the second video content is cached on the first device according to the second video source; obtaining the first video content according to the first video source; and playing the first video according to the first video content and the first relationship. This method can fully utilize videos already watched by the user when playing videos that the user has not yet watched, thereby improving the user's video viewing experience. Attached Figure Description

[0102] Figure 1 is a schematic diagram of a video playback method;

[0103] Figure 2 is a schematic diagram of the structure of a device provided in an embodiment of the present invention;

[0104] Figure 3 is a software structure block diagram of the device 100 according to an embodiment of the present invention;

[0105] Figure 4 is a schematic diagram of a video playback system provided in an embodiment of the present invention;

[0106] Figure 5 is a signaling interaction diagram of a video playback method provided in an embodiment of the present invention;

[0107] Figure 6 is a specific signaling interaction diagram of one of the video playback methods in Figure 5;

[0108] Figure 7 is a schematic diagram of the similarity relationship between the segments of the first video and the segments of the second video in an embodiment of the present invention;

[0109] Figure 8 is a schematic diagram of obtaining the fourth video content by guiding super-resolution of the first video based on at least one second video segment in an embodiment of the present invention;

[0110] Figure 9 is a schematic diagram of a short video playback scenario in an embodiment of the present invention;

[0111] Figure 10 shows a specific signaling interaction diagram for another video playback method shown in Figure 5;

[0112] Figure 11 is a signaling interaction diagram of another video playback method provided in an embodiment of the present invention;

[0113] Figure 12 is a specific signaling interaction diagram of the video playback method in Figure 11;

[0114] Figure 13 is a flowchart of a video playback method provided in an embodiment of the present invention;

[0115] Figure 14 is a flowchart of another video playback method provided by an embodiment of the present invention;

[0116] Figure 15 is a schematic diagram of the structure of a device provided in an embodiment of the present invention. Detailed Implementation

[0117] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0118] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0119] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0120] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0121] To better understand the embodiments of this application, the terms or concepts that may be involved in the embodiments are explained below.

[0122] 1. Blind Super-resolution of Video: Super-resolution without reference. The input is a video, i.e. the video to be processed, and the output is a video, i.e. the video after blind super-resolution.

[0123] 2. Video-guided super-resolution: Super-resolution with reference. The input consists of two videos, namely the video to be processed and the high-definition reference video, and the output is a single video, namely the video after guided super-resolution.

[0124] 3. Request the next refresh video set: The next refresh will display videos. In a news feed scenario, when a user swipes the screen to trigger a refresh, the terminal device will send a video request to the server. The recommendation system will calculate the set of videos that the user may be interested in within a few hundred milliseconds. This set of videos is the next refresh video set, or simply the next refresh. Then the terminal device plays the videos in sequence, giving the user the feeling that there is never a time limit.

[0125] Short video apps always strive to provide users with a better video quality experience using less data. They often cache videos that users have already watched locally for easy rewatching.

[0126] Figure 1 illustrates a video playback method. As shown in Figure 1, the terminal device adaptively selects the video bitrate based on the current network conditions and requests the next set of videos from the server. If the videos in the next set are high-definition, they are played directly; if they are low-definition, on-device super-resolution is performed to improve the visual effect. After the video playback is complete, it is cached locally for a period of time to avoid requiring users to re-request the video upon re-viewing, thus saving bandwidth.

[0127] However, the video playback method shown in Figure 1 has the following drawbacks:

[0128] 1) The end-side super-resolution in Figure 1 is blind super-resolution, and the super-resolution effect is limited. The effect of super-resolution on low bitrate video is very different from that on high bitrate video.

[0129] 2) High bitrate videos consume a lot of data. When the network is poor, playback will either be choppy or you will have to switch to a low bitrate version, which will significantly reduce the picture quality and result in a poor user experience.

[0130] 3) The local cache is only used when the user views a video that they have already watched, and it is not fully utilized.

[0131] In summary, the system did not make full use of locally cached videos that users had already watched, resulting in a poor user experience when watching videos.

[0132] To address the aforementioned shortcomings, the inventors of this application discovered that: due to the trending effect, a large number of short videos with similar visuals will be uploaded in a short period of time; secondly, users' interests are relatively stable over a period of time, and the videos they watch will have a lot of commonalities, which further increases the proportion of unwatched videos with similar visuals to those in their viewing history (for example, clips from TV series A will be used in short videos on different themes, such as plot explanations of TV series A, information on period dramas, etc.); if the locally cached videos that users have already watched can be fully utilized to enhance unwatched videos, it can not only improve the visual effects of the videos, but also save data traffic and provide users with a better video streaming experience.

[0133] Based on this, embodiments of the present invention provide a video playback system that can fully utilize locally cached videos already watched by the user when playing videos that the user has not yet watched, thereby improving the user's video viewing experience. The video playback system provided in these embodiments includes a first device and a second device, which are connected via wired or wireless means.

[0134] Figure 2 is a schematic diagram of the structure of a device provided in an embodiment of the present invention.

[0135] As shown in Figure 2, device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a user identification module card interface 195, etc.

[0136] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on device 100. In other embodiments of this application, device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0137] Processor 110 may include one or more processing units, such as application processors, modem processors, graphics processors, image signal processors, controllers, video codecs, digital signal processors, baseband processors, and / or neural network processors. These different processing units may be independent devices or integrated into one or more processors.

[0138] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0139] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0140] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include integrated circuit interfaces, integrated circuit built-in audio interfaces, pulse code modulation interfaces, universal asynchronous transceiver interfaces, mobile industry processor interfaces, universal input / output interfaces, user identity module interfaces, and / or universal serial bus interfaces, etc.

[0141] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the device 100. In other embodiments of this application, the device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0142] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0143] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0144] The wireless communication function of device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0145] Device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0146] Device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0147] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the device 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0148] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, general-purpose flash memory, etc. Processor 110 executes various functional applications and data processing of device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory disposed in the processor.

[0149] Device 100 can implement audio functions, such as music playback and recording, through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor.

[0150] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Device 100 can receive button input and generate key signal inputs related to user settings and function control of device 100.

[0151] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0152] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0153] The software system of device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses a layered Android system as an example to illustrate the software structure of device 100.

[0154] Figure 3 is a software structure block diagram of the device 100 according to an embodiment of the present invention.

[0155] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0156] The application layer can include a series of application packages.

[0157] As shown in Figure 3, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0158] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0159] As shown in Figure 3, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0160] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0161] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0162] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0163] The phone manager is used to provide communication functions for device 100. This includes managing call status (including connection and disconnection).

[0164] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, etc.

[0165] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0166] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0167] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0168] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0169] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0170] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0171] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0172] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0173] A 2D graphics engine is a drawing engine for 2D drawing.

[0174] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0175] Figure 4 is a schematic diagram of a video playback system provided in an embodiment of the present invention. As shown in Figure 4, the video playback system provided in this embodiment of the present invention includes: a first device 210 and a second device 220. The first device 210 and the second device 220 are connected by wired or wireless means.

[0176] The hardware and software structures of the first device 210 and the second device 220 provided in this embodiment of the invention can be found in the relevant descriptions of device 100 in Figures 2 and 3.

[0177] For example, as shown in FIG4, the first device 210 includes a first processing module 211, a first communication module 212, a local cache module 213, a super-resolution module 214, and a display module 215.

[0178] The second device 220 includes a second communication module 221, a second processing module 222, and a recommendation system 223.

[0179] For example, the first device 210 includes a terminal device, such as a handheld terminal, wearable device, or entertainment device.

[0180] For example, the second device 220 includes a server.

[0181] A server is a device with centralized computing capabilities. For example, a server can be implemented through devices such as servers, virtual machines, the cloud, or robots.

[0182] When the server-side component includes servers, the type of server includes, but is not limited to, general-purpose computers, dedicated server computers, blade servers, etc. This application does not impose a strict limit on the number of servers included in the server-side component; the number can be one or multiple (such as a server cluster).

[0183] A virtual machine is a computing module simulated by software, possessing full hardware system functionality and running in a completely isolated environment. Of course, besides virtual machines, servers can also be implemented using other computing instances, such as containers.

[0184] The cloud is a software platform that uses application virtualization technology, enabling one or more software programs or applications to be developed and run in an independent virtualized environment. Optionally, when the server is implemented through the cloud, the cloud can be deployed on a public cloud, private cloud, or hybrid cloud.

[0185] To address the aforementioned issues of underutilizing locally cached videos already watched by users and resulting in a poor user experience, the embodiments of the present invention need to solve the following three technical problems:

[0186] 1. How to determine the similarity between locally cached videos that a user has already watched and unwatched videos (videos to be refreshed next time);

[0187] 2. Given the similarity between locally cached videos that a user has already watched and unwatched videos (videos to be refreshed next), how can the locally cached videos that the user has already watched be used to enhance the image quality of unwatched videos?

[0188] 3. In situations of severe network fluctuations, how to coordinate video similarity relationships with recommendation systems to achieve a balance between user video quality experience and distribution effectiveness.

[0189] Based on the system architecture shown in Figure 4, this embodiment of the invention provides a video playback method that can make full use of locally cached videos that the user has already watched when playing videos that the user has not yet watched, thereby improving the user's video viewing experience.

[0190] Figure 5 is a signaling interaction diagram of a video playback method provided in an embodiment of the present invention, and Figure 6 is a specific signaling interaction diagram of a video playback method in Figure 5. As shown in Figure 5, the method includes: steps 302-318.

[0191] Step 302: The first device responds to the video refresh event and generates a first message containing the user ID.

[0192] In this step, as shown in Figure 6, the first processing module of the first device responds to the video refresh event and generates the first message.

[0193] For example, a video refresh event may be triggered by a user performing a video refresh operation, or it may be triggered automatically by the first device.

[0194] For example, the first device includes a first application. For instance, the first application is a short video application, where the user triggers a video refresh event by swiping up to get the next video while watching a short video, or the first application automatically triggers a video refresh event after startup to get the next video.

[0195] For example, the user ID is the ID used by the first device to log in to the first application.

[0196] For example, the first message may also include user click behavior data on the previous video in the previous video collection.

[0197] For example, the click behavior data for the previous video includes the duration of the user's viewing of the previous video. If the user liked or saved the video while watching it, the click behavior data also includes the user's liking or saving of the previous video.

[0198] Step 304: The first device sends a first message to the second device.

[0199] In some possible embodiments, as shown in FIG6, step 304 specifically includes: step 304a-step 304b.

[0200] Step 304a: The first processing module sends a first message to the first communication module.

[0201] In this step, as shown in Figure 6, the first processing module of the first device sends a first message to the first communication module of the first device.

[0202] Step 304b: The first communication module sends a first message to the second communication module.

[0203] In this step, as shown in Figure 6, the first communication module of the first device sends a first message to the second communication module of the second device.

[0204] Step 306: The second device obtains the second video ID of at least one second video based on the first message.

[0205] In some possible embodiments, as shown in FIG6, step 306 specifically includes: step 306a-step 306b.

[0206] Step 306a: The second communication module sends the first message to the second processing module.

[0207] In this step, as shown in Figure 6, the second communication module of the second device sends a first message to the second processing module of the second device.

[0208] Step 306b: The second processing module obtains the second video ID of at least one second video corresponding to the user ID based on the first message.

[0209] In this step, as shown in Figure 6, the second processing module of the second device obtains the second video ID of at least one second video corresponding to the user ID based on the first message.

[0210] For example, the second video is a video for which a second video source is obtained from a second device and the second video content is cached on the first device based on the second video source. Therefore, the second video is a video that the user has already watched and cached on the first device.

[0211] For example, suppose the second device obtains the second video IDs of two second videos based on the first message, such as the video ID of video 1 and the video ID of video 2. Therefore, video 1 and video 2 are videos that the user has watched and cached on the first device.

[0212] For example, each time the first device retrieves the next batch of videos from the second device, the second device records the IDs of the videos that have been recommended to the first device in a first list.

[0213] For example, the first list is used to store the IDs of videos for which video sources are obtained from the second device and whose video content is cached on the first device based on the video sources.

[0214] In some possible embodiments, if the first message does not contain a second video ID, then the second processing module of the second device obtains a first list corresponding to the user ID in the first message, and obtains at least one second video ID from the first list.

[0215] In some possible embodiments, the first message contains at least one second video ID, and the second processing module of the second device obtains at least one second video ID from the first message according to the first message.

[0216] Step 308: The second device obtains at least one first video based on the user ID.

[0217] In some possible embodiments, as shown in FIG6, step 308 specifically includes: step 308a-step 308c.

[0218] Step 308a: The second processing module sends the user ID to the recommendation system.

[0219] In this step, as shown in Figure 6, the second processing module of the second device sends the user ID to the recommendation system of the second device.

[0220] Step 308b: The recommendation system retrieves at least one first video based on the user ID.

[0221] In this step, as shown in Figure 6, the recommendation system of the second device obtains at least one first video based on the user ID.

[0222] For example, a recommendation system typically includes modules such as recall, ranking (coarse ranking + fine ranking + re-ranking), and screen control. Based on user profiles, viewing history, and other information, it retrieves videos that the user may like from a massive video library and recommends them to the user for viewing.

[0223] For example, the recommendation system first determines the user profile and other information corresponding to the user ID based on the user ID, and then obtains at least one first video, i.e. the video set to be distributed, through the recall and sorting stages based on the user profile and other information. Assume that the video set to be distributed contains two first videos, denoted as video 3 and video 4.

[0224] For example, recall is part of the recommendation process, finding a set of videos that the user might like (e.g., 10,000 videos) from a massive video library (e.g., containing 1 billion videos), thus reducing the processing load for the sorting module.

[0225] For example, after the videos are sorted by the sorting model, some processing is done before they are displayed to the user—this is called screen control. The screen control module contains many modules. For example, the deduplication module can prevent two identical videos from being displayed to the user one after the other, which would lead to a poor user experience.

[0226] Step 308c: The recommendation system sends at least one first video to the second processing module.

[0227] In this step, as shown in Figure 6, the recommendation system of the second device sends at least one first video to the second processing module of the second device.

[0228] Step 310: The second device obtains a first relationship between the first video and at least one second video based on the first video and at least one second video ID.

[0229] In this step, as shown in Figure 6, the second processing module of the second device obtains the first relationship between the first video and at least one second video based on the first video and at least one second video ID.

[0230] For example, the first relationship includes the similarity relationship between a segment of a first video and at least one segment of a second video. In this embodiment of the invention, the first relationship between the first video and the at least one second video specifically refers to the similarity relationship between the segments of the first video and the segments of the at least one second video, that is, the similarity relationship between a segment of a video that the user has not watched and a segment of at least one locally cached video that the user has watched.

[0231] For example, Figure 7 is a schematic diagram of the similarity relationship between the segments of the first video and the segments of the second video in an embodiment of the present invention. Assume the first video is video #1 and the second video is video #2. Assume both video #1 and video #2 have 10 segments (also called "shots"). As shown in Figure 7, segments 3, 4, and 5 of video #1 are similar to segments 4, 5, and 6 of video #2, respectively, with a similarity of 0.85; segments 8 and 9 of video #1 are similar to segments 10 and 9 of video #2, respectively, with a similarity of 0.85; segments 1, 2, 6, 7, and 10 of video #1 are not similar to segments of video #2, but may be similar to other videos; one video can have a similarity relationship with multiple videos.

[0232] In some possible embodiments, the similarity between two video segments is determined by whether the similarity between the two video segments is greater than or equal to a second threshold. If the similarity between the two video segments is greater than or equal to the second threshold, the two video segments are considered to be similar; if the similarity between the two video segments is less than the second threshold, the two video segments are considered to be dissimilar.

[0233] In some possible embodiments, when determining whether two video segments are similar, in addition to determining whether the similarity between the two video segments is greater than or equal to a second threshold, such as 0.85, if the similarity is greater than or equal to 0.85, it is also necessary to check whether the segment is an isolated segment. If the segment is an isolated segment, it will not be considered a similar segment even if the similarity is greater than or equal to 0.85; if the segment is not an isolated segment, it can be considered a similar segment when the similarity is greater than or equal to 0.85.

[0234] For example, as shown in Figure 7, the similarity between segment 4 of video #1 and segment 2 of video #2 is 0.9. Although the similarity is greater than 0.85, segment 2 of video #2 is an isolated segment. That is, segments 1 and 3 of video #2, which are adjacent to segment 2, have a similarity of 0 with segment 4 of video #1. Therefore, even though the similarity between segment 4 of video #1 and segment 2 of video #2 is greater than 0.85, segment 4 of video #1 and segment 2 of video #2 will not be considered similar. Similarly, even though the similarity between segment 7 of video #1 and segment 4 of video #2 is greater than 0.85, segment 7 of video #1 and segment 4 of video #2 will not be considered similar.

[0235] For example, if a video has already undergone similarity assessment during the data entry stage, then the similarity relationship can be queried simply by using the video ID. Information feed applications typically have a massive video library on the cloud side, containing videos provided by creators. Data entry refers to the stage from when a creator publishes a video to when it is added to the distributable library, which usually involves some content understanding-related work.

[0236] For example, the second processing module of the second device queries the database, inputs the video ID of video 1 and the video ID of video 3, and outputs that video 3 has a total of 10 segments, segments 1-5 are similar to segments 2-6 of video 1, and segments 6-10 are similar to segments 3-7 of video 2.

[0237] In some possible embodiments, step 310 specifically includes: the second device querying the database for similarity relationships between the segments of the second video and the segments of the first video based on the second video ID and the first video ID of the first video.

[0238] For example, if the video has not undergone similarity judgment during the data entry stage, it needs to be calculated in real time. The general process is to first segment the video into multiple segments, extract several key frames from each segment, and then perform similarity judgment between the key frames of two videos to obtain the similarity relationship between the video segments.

[0239] In some possible embodiments, step 310 specifically includes steps 3102-3108.

[0240] Step 3102: The second device obtains the second video based on the second video ID.

[0241] In this step, the second processing module of the second device obtains the second video based on the second video ID.

[0242] Step 3104: The second device performs shot segmentation on the first video and the second video to obtain segments of the first video and the second video.

[0243] In this step, the second processing module of the second device performs shot segmentation on the first video and the second video to obtain segments of the first video and the second video.

[0244] Step 3106: The second device extracts several first keyframes from the segments of the first video and several second keyframes from the segments of the second video.

[0245] In this step, the second processing module of the second device extracts several first keyframes from the segments of the first video and several second keyframes from the segments of the second video.

[0246] Step 3108: The second device obtains the similarity relationship between the segments of the first video and the segments of the second video by performing similarity determination on the first keyframe and the second keyframe.

[0247] In this step, the second device determines the similarity relationship between the segments of the first video and the segments of the second video by performing a similarity judgment on the first keyframe and the second keyframe.

[0248] Step 312: The second device determines the first video source of the first video based on the first relationship.

[0249] In this step, as shown in Figure 6, the second processing module of the second device determines the first video source of the first video based on the first relationship.

[0250] For example, if a locally cached video similar to the video to be distributed has been viewed by a user, the second device will send a low-bitrate or low-resolution video source to the first device to save bandwidth; conversely, if no locally cached video similar to the video to be distributed has been viewed by a user, the second device will send a high-bitrate or high-resolution video source to the first device.

[0251] In some possible embodiments, step 310 specifically includes: the first relationship includes the existence of a second video segment similar to the segment of the first video, determining the bitrate of the first video source as a first bitrate or determining the resolution of the first video source as a first resolution; or the first relationship includes the absence of a second video segment similar to the segment of the first video, determining the bitrate of the first video source as a second bitrate or determining the resolution of the first video source as a second resolution.

[0252] Among them, the second bit rate is higher than the first bit rate; the second resolution is higher than the first resolution.

[0253] Assuming video 4 has a maximum resolution of 1080p and a high quality score, if video 4 has no similar segments to videos 1 and 2, the second device sends a high-bitrate video source of video 4, such as 1080p, to the first device, which can then play it directly. If video 4 has similar segments to both videos 1 and 2, the second device sends a low-bitrate video source of video 4, such as 480p, to the first device.

[0254] In some possible embodiments, when determining the first video source of the first video based on the first relationship, segments in the first video that are dissimilar to at least one segment of the second video can be selected from high bitrate or high resolution video sources; segments in the first video that have a similar relationship to at least one segment of the second video can be selected from low bitrate or low resolution video sources. Step 314: The second device sends a second message to the first device, the second message including at least one second video ID, at least one first video source of the first video, and the first relationship.

[0255] In some possible embodiments, as shown in FIG6, step 314 specifically includes:

[0256] Step 314a: The second processing module generates a second message based on at least one second video ID, at least one first video source of the first video, and a first relationship.

[0257] In this step, as shown in Figure 6, the second processing module of the second device generates a second message based on at least one second video ID, at least one first video source of the first video, and a first relationship.

[0258] Step 314b: The second processing module sends a second message to the second communication module.

[0259] In this step, as shown in Figure 6, the second processing module of the second device sends a second message to the second communication module of the second device.

[0260] Step 314c: The second communication module sends a second message to the first communication module.

[0261] In this step, as shown in Figure 6, the second communication module of the second device sends a second message to the first communication module of the first device.

[0262] Step 316: The first device obtains the first video content and the first video ID of the first video based on the first video source, and caches the first video content locally.

[0263] In some possible embodiments, as shown in FIG6, step 316 specifically includes: steps 316a-316d.

[0264] Step 316a: The first communication module sends a second message to the first processing module.

[0265] In this step, as shown in Figure 6, the first communication module of the first device sends a second message to the first processing module of the first device.

[0266] Step 316b: The first processing module obtains the first video content and the first video ID based on the first video source.

[0267] In this step, as shown in Figure 6, the first processing module of the first device obtains the first video content and the first video ID based on the first video source.

[0268] Step 316c: The first processing module sends the first video content and the first video ID to the local cache module.

[0269] In this step, as shown in Figure 6, the first processing module of the first device sends the first video content and the first video ID to the local cache module of the first device.

[0270] Step 316d: The local caching module caches the first video content and the first video ID locally.

[0271] In this step, as shown in Figure 6, the local caching module of the first device caches the first video content and the first video ID locally.

[0272] For example, the local caching module of the first device manages the local cache video index and records information such as video ID, local cache path, and video segment quality.

[0273] In this embodiment of the invention, regardless of whether the first video content is super-resolution enhanced, the first video content will be cached so that the locally cached video can be used to super-resolution enhance the unwatched video later.

[0274] Step 318: The first device obtains the second video content based on the second video ID, and plays the first video based on the second video content, the first video content, and the first relationship.

[0275] In some possible embodiments, as shown in FIG6, step 318 specifically includes:

[0276] Step 318a: The first processing module obtains the second video content based on the second video ID and continues to execute step 318b or step 318f.

[0277] In this step, as shown in Figure 6, the first processing module of the first device obtains the second video content according to the second video ID and continues to execute step 318b or step 318f.

[0278] In some possible embodiments, the second video ID contained in the second message may exist together with the first relationship in the second message, that is, the first relationship contains the second video ID. Step 318a specifically includes: the first processing module obtains the second video content according to the second video ID contained in the first relationship. Therefore, when the first device obtains the second video content according to the second video ID in the second message, it specifically obtains the second video content according to the second video ID contained in the first relationship.

[0279] Step 318b: When the first processing module contains a second video segment that does not have a segment similar to the first video segment, it sends a fourth message to the super-resolution module.

[0280] In this step, as shown in Figure 6, when the first processing module of the first device sends a fourth message to the super-resolution module of the first device when the first relationship includes a second video segment that does not have a segment similar to the first video segment.

[0281] For example, the fourth message is used to instruct the super-resolution module to perform blind super-resolution on the first video content.

[0282] Step 318c: The super-resolution module performs blind super-resolution on the first video content based on the fourth message to obtain the third video content.

[0283] For example, the main function of the super-resolution module of the first device is to enhance the image quality of the video. Taking super-resolution as an example, it may also include functions such as contrast enhancement and color enhancement, and needs to support both referenceless super-resolution and referenced super-resolution.

[0284] In this step, as shown in Figure 6, the super-resolution module of the first device performs blind super-resolution on the first video content according to the fourth message to obtain the third video content.

[0285] Step 318d: The super-resolution module sends the third video content to the display module.

[0286] In this step, as shown in Figure 6, the super-resolution module of the first device sends the third video content to the display module of the first device.

[0287] Step 318e: The display module plays the first video based on the content of the third video. The process ends.

[0288] In this step, as shown in Figure 6, the display module of the first device plays the first video according to the content of the third video.

[0289] In this embodiment of the invention, the first device can cache the third video content obtained by blind super-resolution locally, so that users do not need to super-resolution again when watching it again. The missing video quality score of the super-resolution video content can be calculated by the first device. The calculation includes model scoring or original score mapping, etc., and can be used as a reference video to guide super-resolution of unwatched videos.

[0290] Step 318f: When the first relationship contains a second video segment that is similar to the first video segment, the first processing module obtains the second video segment that is similar to the first video segment from the second video content.

[0291] In this step, as shown in Figure 6, when the first relationship includes a second video segment that is similar to the first video segment, the first processing module of the first device obtains the second video segment that is similar to the first video segment from the second video content.

[0292] Step 318g: The first processing module sends a fifth message to the super-resolution module. The fifth message contains the acquired segments of the second video.

[0293] In this step, as shown in Figure 6, the first processing module of the first device sends a fifth message to the super-resolution module of the first device. The fifth message contains the acquired segments of the second video.

[0294] Step 318h: The super-resolution module performs super-resolution on the first video based on the obtained segments of the second video to obtain the fourth video content.

[0295] In this step, as shown in Figure 6, the super-resolution module of the first device performs super-resolution on the first video based on the acquired segments of the second video to obtain the fourth video content.

[0296] For example, Figure 8 is a schematic diagram of guiding super-resolution of a first video to obtain a fourth video content based on at least one segment of a second video in an embodiment of the present invention. As shown in Figure 8, at least one second video includes three videos on the left side of Figure 8, namely video A, video B, and video C, and the first video is video D located in the lower middle of Figure 8. Each of videos A, B, C, and D contains three segments. The first segment of video A is similar to the first segment of video D, the second segment of video B is similar to the second segment of video D, and the third segment of video C is similar to the third segment of video D. The resolution of videos A, B, and C is 1080p, and the resolution of video D is 480p. When performing super-resolution on video D, the first segment of video A is used to enhance the first segment of video D, the second segment of video B is used to enhance the second segment of video D, and the third segment of video C is used to enhance the third segment of video D, resulting in the super-resolution video E with a resolution of 1080p.

[0297] Step 318i: The super-resolution module sends the fourth video content to the display module.

[0298] In this step, as shown in Figure 6, the super-resolution module of the first device sends the fourth video content to the display module of the first device.

[0299] Step 318j: The display module plays the first video based on the content of the fourth video. The process ends.

[0300] In this step, as shown in Figure 6, the display module of the first device plays the first video according to the fourth video content.

[0301] In this embodiment of the invention, the first device can cache the fourth video content obtained by guided super-resolution locally, so that the user does not need to undergo super-resolution again when watching it again. The missing video quality score of the super-resolution video content can be calculated by the first device, including model scoring or original score mapping methods, and can be used as a reference video for subsequent guided super-resolution of unwatched videos.

[0302] For example, account A on a short video app uploads a 1080p video, and account B uploads a 480p video. These two videos are similar. If we watch account A first and then account B, account B's local cache contains similar high-definition videos. Therefore, account B's 480p video should have better image quality after uploading. If we delete account B's locally cached high-definition videos, the image quality of account B's 480p video should decrease.

[0303] For example, the video playback method provided in this embodiment of the invention can be applied to short video playback scenarios. Figure 9 is a schematic diagram of a short video playback scenario in this embodiment of the invention. As shown in Figure 9, before the server transmits the next set of videos to the terminal device, it determines whether the next set of videos has similar segments to videos already watched by the user. If there are similar segments, the server transmits the similarity relationship between the next set of videos and the videos already watched by the user to the terminal device at the same time as transmitting the next set of videos to the terminal device. When the next set of videos has similar videos already watched by the user, the terminal device retrieves the similar videos already watched by the user from the local cache and performs on-side guided super-resolution before playback; when the next set of videos does not have similar videos already watched by the user, the terminal device performs blind super-resolution before playback.

[0304] Therefore, this embodiment of the invention introduces a similarity judgment between locally cached videos that the user has already watched and unwatched videos (videos to be refreshed next). When the locally cached videos that the user has already watched are similar to the unwatched videos, the locally cached videos that the user has already watched are used to enhance the unwatched videos, thereby saving data and improving visual effects, and improving the user's video watching experience.

[0305] In some possible embodiments, the second message may further include: the video quality score of the first video source; in step 318, the first device plays the first video according to the second video content, the first video content, and the first relationship, specifically including: step 318A, step 318B, or step 318C.

[0306] Step 318A: If the video quality score is greater than the first threshold, the first device plays the first video according to the first video content.

[0307] In this step, when the video quality score is greater than the first threshold, the first processing module of the first device directly sends the first video content to the display module, and the display module plays the first video according to the first video content.

[0308] For example, a video quality score is a quantitative rating derived from a series of algorithms that comprehensively analyze multiple dimensions of a video, such as sharpness, signal-to-noise ratio, color, and brightness. This score is used to evaluate the overall quality of the video. Video quality scores are positively correlated with resolution and bitrate.

[0309] For example, the first device receives information such as the first relationship, the first video source, and the video quality score provided by the second device, requests the first video content and decodes it. If the quality of the first video content is high, the video is directly displayed to the user.

[0310] Step 318B: If the video quality score is less than or equal to the first threshold and the first relationship includes a second video segment that is not similar to the first video segment, perform blind super-resolution on the first video content to obtain the third video content, and the first device plays the first video based on the third video content.

[0311] In this step, when the video quality score of the first device is less than or equal to the first threshold and the first relationship includes the absence of a second video segment similar to the first video segment, the first processing module of the first device sends the first video content to the super-resolution module. The super-resolution module performs blind super-resolution on the first video content to obtain the third video content. The super-resolution module sends the third video content to the display module, and the display module plays the first video based on the third video content.

[0312] For example, the first device receives information such as the first relationship, the first video source, and the video quality score provided by the second device, requests the first video content and decodes it. If the quality of the first video content is low and there are no segments similar to the locally cached video that the user has already watched, the device calls the edge-side blind super-resolution model to perform blind super-resolution on the first video content and displays the blindly super-resolution video to the user.

[0313] Step 318C: The video quality score is less than or equal to the first threshold, and the first relationship includes a second video segment that is similar to the first video segment. The first device obtains the second video segment that is similar to the first video segment from the second video content, and performs super-resolution on the first video content based on the obtained second video segment to obtain the fourth video content. The first video is then played based on the fourth video content.

[0314] In this step, when the video quality score of the first device is less than or equal to the first threshold and the first relationship includes the absence of a second video segment similar to the first video segment, the first processing module of the first device sends the first video content to the super-resolution module. The super-resolution module performs blind super-resolution on the first video content to obtain the third video content. The super-resolution module sends the third video content to the display module, and the display module plays the first video based on the third video content.

[0315] For example, the first device receives information such as the first relationship, the first video source, and the video quality score provided by the second device, requests the first video content and decodes it. If the quality of the first video content is low and there are segments similar to the locally cached video that the user has already watched, then the similar locally cached video segments are found as reference videos based on the first relationship, and the terminal-side guided super-resolution model is called to guide super-resolution of the first video content, and the guided super-resolution video is displayed to the user.

[0316] Assuming video 3 has a maximum resolution of 480p and a low video quality score, if video 3 has no similar segments to video 1 and video 2, it will enter the blind super-resolution branch, and the enhanced effect will be 480p + blind super-resolution. If video 3 has similar segments to both video 1 and video 2, and the resolution of video 1 and video 2 is 1080p, it will enter the guided super-resolution branch, and the enhanced effect will be 480p + 1080p reference + guided super-resolution.

[0317] Assuming video 4 has a maximum resolution of 1080p and a high quality score, if video 4 has no similar segments to videos 1 and 2, the second device sends a high-bitrate video source of video 4, such as 1080p, to the first device, which can then play it directly. However, if video 4 has similar segments to both videos 1 and 2, the second device sends a low-bitrate video source of video 4, such as 480p, to the first device. In this case, the guided super-resolution branch will be entered. Since videos 1 and 2 both have a resolution of 1080p, the enhanced effect should be 480p + 1080p reference + guided super-resolution.

[0318] Compared to the video playback method shown in Figure 1, the video playback method provided in this embodiment of the invention has the following three improvements:

[0319] (1) End-side super-resolution can use guided super-resolution, i.e. reference super-resolution. Using the already watched high-definition local cached video to guide the super-resolution of the unwatched video is better than blind super-resolution. In the case of no high-definition video source for the unwatched video, a better picture quality experience can be obtained, thereby improving the user's video viewing experience.

[0320] (2) When there are similar segments between the unwatched video and the locally cached video that has been watched, the unwatched video can use a low bitrate source or a low resolution source to effectively reduce traffic consumption and significantly improve the user experience in poor network conditions. The video playback method shown in Figure 1 cannot achieve this effect.

[0321] (3) Make fuller use of the local cached video that has been watched. In the video playback method shown in Figure 1, the local cached video that has been watched is only used for the user to view the video that has been watched. In this embodiment, the local cached video that has been watched is also used as the reference video for the end-side super-resolution.

[0322] The beneficial effects of the video playback method provided in this embodiment of the invention are as follows:

[0323] 1) Higher image quality: The embodiments of the present invention can provide reference videos for edge super-resolution, upgrading edge "blind super-resolution" to edge "guided super-resolution". Edge guided super-resolution has reference videos, and the effect is better than blind super-resolution. It can provide a better image quality experience when there is no high-definition video source.

[0324] For example, the highest resolution of video 3 is 480p. The effect of the video playback method shown in Figure 1 is 480p + blind super-resolution. In the embodiment of the present invention, the effect is 480p + 1080p reference + guided super-resolution. When the reference video is similar, the enhancement effect of 480p + 1080p reference + guided super-resolution is much higher than that of 480p + blind super-resolution.

[0325] 2) Saves more data: When there are similar segments between unwatched videos and locally cached videos that have already been watched, similar segments can use low bitrate or low resolution video sources, effectively reducing data consumption and significantly improving the user experience in poor network environments.

[0326] For example, if the highest resolution of video 4 is 1080p, the video playback method shown in Figure 1 requires downloading the original 1080p video. In cases of poor network conditions, the video will either stutter and not play smoothly, or the resolution will be reduced for playback. However, with the video playback method improved in this embodiment of the invention, when the reference video is similar, only a 480p video needs to be downloaded. Furthermore, the enhancement effect of 480p + 1080p reference + guided super-resolution is approximately equivalent to the effect of the original 1080p video.

[0327] In some possible embodiments, the first relationship includes the second video ID. Step 318 specifically includes:

[0328] Step 3182: The first device obtains the second video content based on the second video ID contained in the first relationship.

[0329] In this step, the first processing module of the first device obtains the second video content based on the second video ID contained in the first relationship. Therefore, the first device only obtains the second video content containing the second video ID contained in the first relationship.

[0330] Step 3184: The first device plays the first video based on the second video content, the first video content, and the first relationship.

[0331] For a description of step 3184, please refer to the records of steps 318b-318j above, which will not be repeated here.

[0332] Optionally, when the first message does not contain the second video ID, as shown in Figure 5, step 314 is followed by step 320; and step 318 is followed by steps 322-326.

[0333] Step 320: The second device adds the first video ID of the first video to the first list.

[0334] In this step, as shown in Figure 6, the second processing module of the second device adds the first video ID of the first video to the first list.

[0335] In this embodiment of the invention, after the second device distributes the first video source of the first video to the first device, the second device adds the first video ID to the first list. The first device obtains the first video content according to the first video source and caches the first video content locally, thereby realizing the simultaneous recording of cached videos watched by the user by the terminal device and the server.

[0336] Step 322: The first device responds to the deletion event of the first video content and generates a third message.

[0337] In some possible embodiments, Figure 10 is a specific signaling interaction diagram of another video playback method shown in Figure 5. In this step, as shown in Figure 10, the first processing module of the first device generates a third message containing the first video ID in response to the deletion event of the first video content.

[0338] Step 324: The first device sends a third message to the second device, the third message containing the first video ID.

[0339] In some possible embodiments, as shown in FIG10, step 324 specifically includes:

[0340] Step 324a: The first processing module sends a third message to the first communication module.

[0341] In this step, as shown in Figure 6, the first processing module of the first device sends a third message to the first communication module of the first device.

[0342] Step 324b: The first communication module sends a third message to the second communication module.

[0343] In this step, as shown in Figure 6, the first communication module of the first device sends a third message to the second communication module of the second device.

[0344] Step 326: The second device deletes the first video ID from the first list according to the third message.

[0345] In some possible embodiments, as shown in FIG10, step 326 specifically includes:

[0346] Step 326a: The second communication module sends a third message to the second processing module.

[0347] In this step, as shown in Figure 6, the second communication module of the second device sends a third message to the second processing module of the second device.

[0348] Step 326b: The second processing module deletes the first video ID from the first list based on the third message.

[0349] In this step, as shown in Figure 6, the second processing module of the second device deletes the first video ID from the first list according to the third message.

[0350] In this embodiment of the invention, if a user deletes a locally cached video that has been viewed, the terminal device will notify the server, and the server will delete the first video ID from the first list, thereby enabling the terminal device and the server to synchronously record the cached videos that the user has viewed.

[0351] In summary, the video playback method provided by the embodiments of the present invention can solve the first two of the above three technical problems.

[0352] 1. How to determine the similarity between locally cached videos that a user has already watched and unwatched videos (videos to be refreshed next)?

[0353] In this embodiment of the invention, the second device receives a first message sent by the first device, and obtains the IDs of the first video (a video the user has not watched) and the second video (the IDs of locally cached videos the user has watched) based on the first message. The second device queries the video similarity database at the time of access based on the first video ID and the second video ID to obtain the similarity relationship between the videos, or it can calculate the similarity relationship between the videos in real time. If the unwatched video and the watched locally cached video are similar, the second device can send the low-definition video source of the unwatched video to the first device, which can save data traffic.

[0354] 2. Given the similarity between locally cached videos that a user has already watched and unwatched videos (videos to be refreshed next), how can the locally cached videos that the user has already watched be used to enhance the image quality of unwatched videos?

[0355] In this embodiment of the invention, the unwatched video is subjected to guided super-resolution, blind super-resolution, or no super-resolution based on the similarity between the unwatched video and the locally cached video that has been watched, and the quality of the unwatched video. Specifically, if the quality of the unwatched video is high, no super-resolution is applied, and the video is directly displayed to the user. If the quality of the unwatched video is low and there are no segments similar to the locally cached video that the user has watched, the client-side blind super-resolution model is invoked to perform blind super-resolution on the unwatched video, and the super-resolution video is displayed to the user. If the quality of the unwatched video is low and there are segments similar to the locally cached video that the user has watched, a similar segment of the locally cached video is found as a reference video based on a first relationship, and the client-side guided super-resolution model is invoked to perform guided super-resolution on the unwatched video, and the super-resolution video is displayed to the user.

[0356] Figure 11 is a signaling interaction diagram of another video playback method provided in an embodiment of the present invention, and Figure 12 is a specific signaling interaction diagram of the video playback method in Figure 11. As shown in Figure 11, the method includes:

[0357] Step 402: In response to the video refresh event, the first device generates a first message containing the user ID and the network quality of the first device.

[0358] In this step, as shown in Figure 12, the first processing module of the first device responds to the video refresh event and generates a first message containing the user ID and the network quality of the first device.

[0359] For example, network quality refers to the performance of a network in terms of data transmission speed, stability, reliability, and security. It directly affects the user's network experience and is used as an important indicator to measure the level of information infrastructure. Network quality assessment standards typically include bandwidth, latency, jitter, packet loss rate, network availability, and connection establishment time.

[0360] Step 404: The first device sends a first message to the second device.

[0361] For a description of step 404, please refer to the description of step 304 in the embodiment shown in Figure 5 above, which will not be repeated here.

[0362] Step 406: The second device obtains the second video ID of at least one second video based on the first message.

[0363] For a description of step 406, please refer to the description of step 306 in the embodiment shown in Figure 5 above, which will not be repeated here.

[0364] Step 408: The second device obtains multiple fifth videos based on the user ID.

[0365] In some possible embodiments, as shown in FIG12, step 408 specifically includes: steps 408a-408c.

[0366] Step 408a: The second processing module sends the user ID and network quality to the recommendation system.

[0367] In this step, as shown in Figure 12, the second processing module of the second device sends the user ID and network quality to the recommendation system of the second device.

[0368] Step 408b: The recommendation system retrieves multiple fifth videos based on the user ID.

[0369] In this step, as shown in Figure 12, the recommendation system of the second device obtains multiple fifth videos based on the user ID.

[0370] For example, the recommendation system first determines the user profile and other information corresponding to the user ID based on the user ID, and then obtains a set of preliminary screening videos to be distributed through the recall phase based on the user profile and other information. This set contains multiple fifth videos.

[0371] Step 408c: The recommendation system sends multiple fifth videos to the second processing module.

[0372] In this step, as shown in Figure 12, the recommendation system of the second device sends multiple fifth videos to the second processing module of the second device.

[0373] Step 410: The second device obtains a second relationship between the fifth video and at least one second video ID based on the fifth video and at least one second video ID.

[0374] In this step, as shown in Figure 12, the second processing module of the second device obtains a second relationship between the fifth video and at least one second video ID.

[0375] For example, the second relationship includes a similarity relationship between a segment of the fifth video and at least one segment of the second video.

[0376] For an explanation of how the second device obtains the second relationship in step 410, please refer to the description of how the second device obtains the first relationship in step 310 of the embodiment shown in Figure 5 above, which will not be repeated here.

[0377] Step 412: The second device obtains at least one first video and its first relationship based on network quality, the second relationship, and multiple fifth videos.

[0378] In some possible embodiments, as shown in FIG12, step 412 specifically includes:

[0379] Step 412a: The second processing module sends the second relationships corresponding to multiple fifth videos to the recommendation system.

[0380] In this step, as shown in Figure 12, the second processing module of the second device sends the second relationship corresponding to multiple fifth videos to the recommendation system.

[0381] Step 412b: The recommendation system obtains at least one first video and its first relationship based on network quality, second relationship, and multiple fifth videos.

[0382] In this step, as shown in Figure 12, the recommendation system of the second device adjusts the selection of videos entering the sorting stage based on network quality and first relationship, and selects at least one first video and its first relationship from multiple fifth videos and second relationships.

[0383] For example, when network quality is poor, the recommendation system may give higher weight to the fifth video that has similar segments to at least one second video, prioritizing videos with similar segments. For instance, the fifth video obtained by the recommendation system based on the user's long / short-term interests is usually highly similar to the second video.

[0384] For example, when the network quality is good, the recommendation system will prioritize the fifth video, which has no similar segments to at least one second video. For instance, the fifth video obtained by the recommendation system based on sudden hot topics and interest exploration usually has a low similarity to the second video.

[0385] After the ranking phase, the recommendation system obtains a set of videos to be distributed. This set of videos contains at least one first video and its first relationship. The first relationship includes the similarity relationships between the segments of the first video and the segments of at least one second video.

[0386] The video playback method shown in Figure 1, under network fluctuations, either maintains high-definition video sources but experiences playback stuttering, or sacrifices image quality to ensure smooth playback; it cannot simultaneously guarantee both image quality and smooth playback. The video playback method provided in this embodiment combines the similarity relationship between unwatched videos and locally cached videos that have already been watched with a recommendation system to balance user experience and video quality. For example, in scenarios with frequent network quality fluctuations (such as on a high-speed train where the signal is intermittent), the priority of the recommendation system can be adjusted based on network quality and similarity relationships. For instance, when network quality is poor, the recommendation system prioritizes the fifth video segment, which is similar to the second video segment, and distributes the fifth video segment first; when network quality is good, it prioritizes the fifth video segment, which is not similar to the second video segment, thereby ensuring a smooth video streaming experience for the user.

[0387] Compared to the method embodiments shown in Figures 5-10, the video playback method provided by this invention allows users to smoothly watch videos without sacrificing image quality even when the network fluctuates, while also taking into account user preferences; by introducing similarity relationships into the recommendation system, it achieves the effect of balancing user experience and video quality.

[0388] Step 412c: The recommendation system sends at least one first video and its first relationship to the second processing module.

[0389] In this step, as shown in Figure 12, the recommendation system of the second device sends at least one first video and its first relationship to the second processing module.

[0390] Step 414: The second device determines the first video source of the first video based on the first relationship.

[0391] For a description of step 414, please refer to the description of step 312 in the embodiment shown in Figure 5 above, which will not be repeated here.

[0392] Step 416: The second device sends a second message to the first device. The second message includes at least one second video ID, at least one first video source of the first video, and a first relationship.

[0393] For a description of step 416, please refer to the description of step 314 in the embodiment shown in Figure 5 above, which will not be repeated here.

[0394] Step 418: The first device obtains the first video content and the first video ID of the first video based on the first video source, and caches the first video content and the first video ID locally.

[0395] For a description of step 418, please refer to the description of step 316 in the embodiment shown in Figure 5 above, which will not be repeated here.

[0396] Step 420: The first device obtains the second video content based on the second video ID, and plays the first video based on the second video content, the first video content, and the first relationship.

[0397] For a description of step 420, please refer to the description of step 318 in the embodiment shown in Figure 5 above, which will not be repeated here.

[0398] Optionally, when the first message does not contain the second video ID, as shown in Figure 11, step 416 is followed by step 422; and step 420 is followed by steps 424-428.

[0399] Step 422: The second device adds the first video ID of the first video to the first list.

[0400] For a description of step 422, please refer to the description of step 320 in the embodiment shown in Figure 5 above, which will not be repeated here.

[0401] Step 424: The first device responds to the deletion event of the first video content and generates a third message.

[0402] For a description of step 424, please refer to the description of step 322 in the embodiment shown in Figure 5 above, which will not be repeated here.

[0403] Step 426: The first device sends a third message to the second device, the third message containing the first video ID.

[0404] For a description of step 426, please refer to the description of step 324 in the embodiment shown in Figure 5 above, which will not be repeated here.

[0405] Step 428: The second device deletes the first video ID from the first list according to the third message.

[0406] For a description of step 428, please refer to the description of step 326 in the embodiment shown in Figure 5 above, which will not be repeated here.

[0407] In summary, the video playback method provided by the embodiments of the present invention can solve the above three technical problems.

[0408] 1. How to determine the similarity between locally cached videos that a user has already watched and unwatched videos (videos to be refreshed next)?

[0409] In this embodiment of the invention, the second device receives a first message sent by the first device, and obtains the IDs of the first video (a video the user has not watched) and the second video (the IDs of locally cached videos the user has watched) based on the first message. The second device queries the video similarity database at the time of access based on the first video ID and the second video ID to obtain the similarity relationship between the videos, or it can calculate the similarity relationship between the videos in real time. If the unwatched video and the watched locally cached video are similar, the second device can send the low-definition video source of the unwatched video to the first device, which can save data traffic.

[0410] 2. Given the similarity between locally cached videos that a user has already watched and unwatched videos (videos to be refreshed next), how can the locally cached videos that the user has already watched be used to enhance the image quality of unwatched videos?

[0411] In this embodiment of the invention, the unwatched video is subjected to guided super-resolution, blind super-resolution, or no super-resolution based on the similarity between the unwatched video and the locally cached video that has been watched, and the quality of the unwatched video. Specifically, if the quality of the unwatched video is high, no super-resolution is applied, and the video is directly displayed to the user. If the quality of the unwatched video is low and there are no segments similar to the locally cached video that the user has watched, the client-side blind super-resolution model is invoked to perform blind super-resolution on the unwatched video, and the super-resolution video is displayed to the user. If the quality of the unwatched video is low and there are segments similar to the locally cached video that the user has watched, a similar segment of the locally cached video is found as a reference video based on a first relationship, and the client-side guided super-resolution model is invoked to perform guided super-resolution on the unwatched video, and the super-resolution video is displayed to the user.

[0412] 3. In situations of severe network fluctuations, how to coordinate video similarity relationships with recommendation systems to achieve a balance between user video quality experience and distribution effectiveness.

[0413] This invention utilizes the similarity between unwatched videos and watched locally cached videos to adjust the recommendation system. When network quality is good, it prioritizes recommending unwatched videos that are not similar to watched locally cached videos; when network quality is poor, it prioritizes recommending unwatched videos that are similar to watched locally cached videos. This allows for a smooth high-definition video experience and a better user experience, without affecting the distribution effect.

[0414] Figure 13 is a flowchart of a video playback method provided by an embodiment of the present invention. As shown in Figure 13, the method includes:

[0415] Step 502: The first device sends a first message to the second device. The first message contains the user ID.

[0416] Step 504: The second device obtains at least one first video source of a first video, at least one second video ID corresponding to a user ID, and a first relationship between the first video and at least one second video based on the first message; the second video is a video for which the second video source is obtained from the second device and the content of the second video is cached to the first device based on the second video source.

[0417] Step 506: The second device sends a second message to the first device. The second message contains the first video source, the second video ID, and the first relationship.

[0418] Step 508: The first device obtains the first video content based on the first video source.

[0419] Step 510: The first device plays the first video according to the first video content and the first relationship.

[0420] In the technical solution of the video playback method provided in this embodiment of the invention, the method includes: a first device sending a first message to a second device, the first message containing a user ID; the second device obtaining, based on the first message, a first video source of at least one first video, a second video ID of at least one second video corresponding to the user ID, and a first relationship between the first video and the at least one second video; the second video being a video for which a second video source is obtained from the second device and the second video content is cached on the first device based on the second video source; the second device sending a second message to the first device, the second message containing the first video source, the second video ID, and the first relationship; the first device obtaining the first video content based on the first video source, and playing the first video based on the first video content and the first relationship, thereby fully utilizing locally cached videos already watched by the user when playing videos that the user has not yet watched, thus improving the user's video viewing experience.

[0421] After receiving the first message from the first device, the second device in this embodiment of the invention not only transmits the first video source of the first video to the first device, but also transmits the second video ID of the second video and the first relationship between the first video and at least one second video. This allows the first device to utilize the first relationship between the first video and at least one second video when playing the first video, thus making use of the second video and improving the user's viewing experience of the first video.

[0422] This invention, when playing videos that a user has not yet watched, utilizes a first relationship between unwatched videos and locally cached videos that have already been watched to improve the image quality of unwatched videos, thereby enhancing the user's video viewing experience. Furthermore, compared to locally cached videos that have already been watched and are only used for viewing previously viewed videos, this invention makes fuller use of locally cached videos that have already been watched. Therefore, the video playback method provided by this invention can more fully utilize locally cached videos that the user has already watched, improving the user's video viewing experience.

[0423] Figure 14 is a flowchart of another video playback method provided by an embodiment of the present invention. As shown in Figure 14, the method includes:

[0424] Step 602: The first device sends a first message to the second device. The first message contains the user ID.

[0425] Step 604: The second device obtains at least one first video source of a first video, at least one second video ID corresponding to a user ID, and a first relationship between the first video and at least one second video according to the first message; the second video is a video that obtains the second video source from the second device and caches the second video content to the first device according to the second video source.

[0426] For example, the first relationship includes the similarity relationship between the segments of the first video and the segments of the second video.

[0427] In some possible embodiments, the first message does not include the network quality of the first device; step 604 specifically includes: steps 6042-6046.

[0428] Step 6042: The second device obtains at least one first video and at least one second video ID corresponding to the user ID based on the first message.

[0429] In some possible embodiments, the second device acquires at least one first video based on the first message, including: the second device acquires the at least one first video based on a user ID.

[0430] Step 6044: The second device obtains the first relationship based on the first video and at least one second video ID.

[0431] In some possible embodiments, step 6044 specifically includes either step 604a or step 604b.

[0432] Step 604a: The second device queries the database for the similarity relationship between the segments of the second video and the segments of the first video based on the second video ID and the first video ID of the first video.

[0433] Step 604b: The second device obtains the second video based on the second video ID; performs shot segmentation on the first video and the second video to obtain segments of the first video and segments of the second video; extracts several first keyframes from the segments of the first video and extracts several second keyframes from the segments of the second video; and obtains the similarity relationship between the segments of the first video and the segments of the second video by performing similarity determination on the first keyframes and the second keyframes.

[0434] Step 6046: The second device determines the first video source of the first video based on the first relationship.

[0435] In some possible embodiments, step 6046 specifically includes either step 604A or step 604B.

[0436] Step 604A: The first relationship includes the existence of a second video segment that is similar to the segment of the first video. The second device determines that the bitrate of the first video source is a first bitrate or determines that the resolution of the first video source is a first resolution.

[0437] Step 604B: The first relationship includes a second video segment that does not have a segment similar to the first video segment. The second device determines that the bitrate of the first video source is the second bitrate or determines that the resolution of the first video source is the second resolution.

[0438] Among them, the second bit rate is higher than the first bit rate; the second resolution is higher than the first resolution.

[0439] In some possible embodiments, the first message contains the network quality of the first device; in step 604, the second device obtains at least one first video and a first relationship based on the first message, including: steps 604c-604e.

[0440] Step 604c: The second device obtains multiple fifth videos based on the user ID;

[0441] Step 604d: The second device obtains a second relationship based on the IDs of the fifth video and at least one second video; the second relationship includes the similarity relationship between the segments of the fifth video and the segments of the second video.

[0442] Step 604e: The second device obtains at least one first video and a first relationship based on network quality, the second relationship, and multiple fifth videos.

[0443] In some possible embodiments, step 604e includes: when the network quality is good, the second device preferentially acquires a fifth video that is not similar to a segment of the second video as the first video; or when the network quality is poor, it preferentially acquires a fifth video that is similar to a segment of the second video as the first video.

[0444] Step 606: The second device sends a second message to the first device. The second message contains the first video source, the second video ID, and the first relationship.

[0445] Step 608: The second device adds the first video ID of the first video to the first list.

[0446] For example, the first list is used to store the IDs of videos for which video sources are obtained from the second device and whose video content is cached on the first device based on the video sources.

[0447] Step 610: The first device obtains the first video content based on the first video source.

[0448] Step 612: The first device plays the first video according to the first video content and the first relationship.

[0449] In some possible embodiments, the first relationship includes the second video ID. Step 612 specifically includes:

[0450] Step 6122: The first device obtains the second video content based on the second video ID contained in the first relationship.

[0451] Step 6124: The first device plays the first video based on the second video content, the first video content, and the first relationship.

[0452] In some possible embodiments, step 6124 specifically includes either step 612a or step 612b.

[0453] Step 612a: The first relationship includes a second video segment that does not have a segment similar to the first video segment. The first device performs blind super-resolution on the first video content to obtain the third video content, and plays the first video based on the third video content; or

[0454] Step 612b: The first relationship includes a second video segment that is similar to the first video segment. The first device obtains the second video segment that is similar to the first video segment from the second video content. Based on the obtained second video segment, the first video is guided to perform super-resolution to obtain the fourth video content. The first video is then played based on the fourth video content.

[0455] In some possible embodiments, the second message further includes: the video quality score of the first video source; step 6124 includes: playing the first video based on the video quality score, the second video content, the first video content, and the first relationship.

[0456] In some possible embodiments, step 6124 specifically includes: step 612A, step 612B, or step 612C.

[0457] Step 612A: If the video quality score is greater than the first threshold, the first device plays the first video according to the first video content.

[0458] Step 612B: If the video quality score is less than or equal to the first threshold and the first relationship includes a second video segment that is not similar to the first video segment, perform blind super-resolution on the first video content to obtain the third video content, and the first device plays the first video based on the third video content.

[0459] Step 612C: The video quality score is less than or equal to the first threshold, and the first relationship includes a second video segment that is similar to the first video segment. The first device obtains the second video segment that is similar to the first video segment from the second video content, and performs super-resolution on the first video content based on the obtained second video segment to obtain the fourth video content. The first video is then played based on the fourth video content.

[0460] Step 614: In response to the deletion event of the first video content, the first device sends a third message to the second device, the third message containing the first video ID.

[0461] Step 616: The second device deletes the first video ID from the first list according to the third message.

[0462] Figure 15 is a schematic diagram of the structure of a device provided in an embodiment of the present invention. It should be understood that the device 700 is capable of performing the various steps in the above method embodiments, and will not be described in detail here to avoid repetition. The device 700 includes a processor 701 and a memory 702.

[0463] This application embodiment also provides a first device, including at least one processor 701 and at least one memory 702, wherein the at least one memory 702 is used to store at least one program, and when the at least one processor 701 runs the at least one program, the first device performs the operations involved in the first device as described in the above method embodiment.

[0464] This application embodiment also provides a second device, including at least one processor 701 and at least one memory 702, wherein the at least one memory 702 is used to store at least one program, and when the at least one processor 701 runs the at least one program, the second device performs the operations involved in the second device as described in the above method embodiment.

[0465] This application provides a readable storage medium storing a program that, when run by a first device, causes the first device to perform operations as described in the above method embodiments.

[0466] This application provides a readable storage medium storing a program that, when run by a second device, causes the second device to perform operations as described in the above method embodiments.

[0467] This application also provides a program product containing instructions that, when run on a first device or any at least one processor, cause the first device to perform the operations involved in the first device as described in the above method embodiments.

[0468] This application also provides a program product containing instructions that, when run on a second device or any at least one processor, cause the second device to perform the operations involved in the second device as described in the above method embodiments.

[0469] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0470] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0471] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A video playback method, characterized in that, Applied to a first device, the method includes: Send a first message to the second device, the first message containing the user ID; The system receives a second message sent by the second device, the second message containing a first video source of at least one first video, a second video ID of at least one second video, and a first relationship between the first video and the at least one second video; the second video is a video for which a second video source is obtained from the second device and the content of the second video is cached on the first device according to the second video source. Obtain the first video content based on the first video source; The first video is played according to the first video content and the first relationship.

2. The method according to claim 1, characterized in that, Playing the first video according to the first video content and the first relationship includes: The second video content is obtained based on the second video ID contained in the first relationship; The first video is played based on the second video content, the first video content, and the first relationship.

3. The method according to claim 1 or 2, characterized in that, The first relationship includes the similarity relationship between the segments of the first video and the segments of the at least one second video.

4. The method according to claim 2, characterized in that, Playing the first video based on the second video content, the first video content, and the first relationship includes: The first relationship includes a second video segment that does not have a segment similar to the first video segment. Blind super-resolution is performed on the first video content to obtain third video content, and the first video is played based on the third video content; or The first relationship includes the existence of a second video segment that is similar to a segment of the first video. The second video segment that is similar to a segment of the first video is obtained from the second video content. The first video is then guided by super-resolution based on the obtained second video segment to obtain a fourth video content. The first video is then played based on the fourth video content.

5. The method according to claim 2, characterized in that, The second message also includes: the video quality score of the first video source; Playing the first video based on the second video content, the first video content, and the first relationship includes: The first video is played based on the video quality score, the second video content, the first video content, and the first relationship.

6. The method according to claim 5, characterized in that, Playing the first video based on the video quality score, the second video content, the first video content, and the first relationship includes: If the video quality score is greater than a first threshold, the first video is played according to the first video content; or The video quality score is less than or equal to a first threshold, and the first relationship includes a second video segment that does not have a segment similar to the first video segment. Blind super-resolution is then performed on the first video content to obtain third video content, and the first video is played based on the third video content; or The video quality score is less than or equal to a first threshold, and the first relationship includes a segment of the second video that is similar to a segment of the first video. The segment of the second video that is similar to a segment of the first video is obtained from the second video content. The first video content is guided to be super-scored based on the obtained segment of the second video to obtain a fourth video content. The first video is played based on the fourth video content.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: The first video content is cached locally.

8. The method according to claim 7, characterized in that, After caching the first video content locally, the method further includes: Delete the first video content; A third message is sent to the second device, the third message being used to instruct the second device to delete the first video ID of the first video from the first list, the first list being used to store the IDs of videos for which video sources are obtained from the second device and whose video content is cached to the first device according to the video sources.

9. A video playback method, characterized in that, Applied to a second device, the method includes: Receive a first message sent by a first device, the first message containing a user ID; Based on the first message, at least one first video source of a first video, at least one second video ID of a second video, and a first relationship between the first video and the at least one second video are obtained; the second video is a video for which the second video source is obtained from the second device and the content of the second video is cached on the first device based on the second video source; A second message is sent to the first device, the second message containing at least one second video ID, a first video source of the at least one first video, and the first relationship, so that the first device obtains the first video content according to the first video source and plays the first video according to the first video content and the first relationship.

10. The method according to claim 9, characterized in that, The step of obtaining a first video source of at least one first video, a second video ID of at least one second video corresponding to the user ID, and a first relationship between the first video and the at least one second video based on the first message includes: Based on the first message, obtain the at least one first video and the second video ID of the at least one second video corresponding to the user ID; The first relationship is obtained based on the first video and at least one of the second video IDs; The first video source of the first video is determined based on the first relationship.

11. The method according to claim 10, characterized in that, The step of obtaining the first relationship based on the first video and at least one of the second video IDs includes: Based on the second video ID and the first video ID of the first video, query the database for similarity relationships between the segments of the second video and the segments of the first video; or, The second video is obtained based on the second video ID; the first video and the second video are segmented to obtain segments of the first video and segments of the second video; several first keyframes are extracted from the segments of the first video and several second keyframes are extracted from the segments of the second video; the similarity relationship between the segments of the first video and the segments of the second video is obtained by performing similarity determination on the first keyframes and the second keyframes.

12. The method according to claim 10 or 11, characterized in that, Determining the first video source of the first video based on the first relationship includes: The first relationship includes the existence of a second video segment that is similar to a segment of the first video, determining that the bitrate of the first video source is a first bitrate or determining that the resolution of the first video source is a first resolution; or The first relationship includes the absence of a second video segment that is similar to the first video segment, and the bitrate of the first video source is determined to be a second bitrate or the resolution of the first video source is determined to be a second resolution. Wherein, the second bitrate is higher than the first bitrate; the second resolution is higher than the first resolution.

13. The method according to any one of claims 9-12, characterized in that, After sending the second message to the first device, the method further includes: The first video ID is stored in a first list, which is used to store the IDs of videos for which video sources are obtained from the second device and video content is cached on the first device according to the video sources; The system receives a third message sent by the first device and deletes the first video ID from the first list according to the third message; the third message is generated by the first device deleting the first video content of the first video.

14. The method according to claim 9, characterized in that, The first message also includes the network quality of the first device; Obtaining the at least one first video and the first relationship based on the first message includes: Retrieve multiple fifth videos based on the user ID; A second relationship is obtained based on the IDs of the fifth video and the at least one second video; the second relationship includes similarity relationships between the segments of the fifth video and the segments of the at least one second video. The at least one first video and the first relationship are obtained based on the network quality, the second relationship, and the plurality of fifth videos.

15. The method according to claim 14, characterized in that, The step of obtaining the at least one first video based on the network quality, the second relationship, and the plurality of fifth videos includes: When the network quality is good, the fifth video, which is dissimilar to the segment of the second video, is preferentially acquired as the first video; or When the network quality is poor, the fifth video, which is similar to the segment of the second video, is preferentially acquired as the first video.

16. A first device, characterized in that, It includes a processor and a memory, wherein the memory is used to store a program that, when the processor runs the program, causes the first device to perform the steps of the method as described in any one of claims 1-8.

17. A second device, characterized in that, It includes a processor and a memory, wherein the memory is used to store a program that, when the processor runs the program, causes the second device to perform the steps of the method as described in any one of claims 9-15.

18. A video playback system, characterized in that, It includes the first device in the method of claim 16 and the second device in the method of claim 17.

19. A readable storage medium, characterized in that, The readable storage medium stores a program that, when executed by the device, causes the device to perform the method as described in any one of claims 1-8 or the method as described in any one of claims 9-15.

20. A program product, characterized in that, The program product stores a program that, when the program is run by the device, causes the device to perform the method as described in any one of claims 1-8 or the method as described in any one of claims 9-15.

Images