Video playback method, device, equipment, readable storage medium and program product

Abstract

The embodiments of the present application provide a video playback method, device, equipment, readable storage medium and program product, which relate to the fields of media broadcast control, and the application scenarios include but are not limited to the scenario of synchronous video playback of multiple terminals. The method is executed by a terminal, and includes: receiving a first message sent by a server, the first message is used to notify the terminal to play each video in a video set; based on the first message, playing the video frame of each video, and sending the video sequence number of each video and the video frame sequence number of each video frame to the server; receiving the maximum sequence number of the video frame and the minimum sequence number of the video frame sent by the server; based on the video frame sequence number, the maximum sequence number of the video frame and the minimum sequence number of the video frame, adjusting the video playback mode of the terminal to synchronize the video playback between the terminal and other terminals; in this way, multiple terminals do not rely on time synchronization, and achieve frame-level playback synchronization, thereby improving the accuracy of multi-terminal video playback synchronization.

Description

Technical Field

[0001] The present application relates to the field of computer technology. Specifically, the present application relates to a video playback method, device, equipment, readable storage medium and program product. Background Art

[0002] In the prior art, in some business scenarios, media broadcast control needs to play videos synchronously on multiple terminals in the same space. The server sends the program to each terminal in the multiple terminals for program delivery. After each terminal downloads the video locally, each terminal plays the video content according to a unified delivery time. Since the synchronous video playback of multiple terminals depends on the time synchronization between each terminal and the server, but the time synchronization is often not completely synchronized due to network factors, the difference in hardware performance between each terminal will make it difficult to ensure time synchronization; for example, the difference in hardware performance of each terminal will result in different decoding time for each frame of video; thus, the synchronous video playback of multiple terminals is affected, that is, the accuracy of the synchronous video playback of multiple terminals is reduced. Summary of the invention

[0003] In view of the shortcomings of the existing methods, the present application proposes a video playback method, device, equipment, computer-readable storage medium and computer program product to solve the problem of how to improve the accuracy of video synchronization of multi-terminal playback.

[0004] In a first aspect, the present application provides a video playback method, which is executed by a terminal, comprising:

[0005] receiving a first message sent by the server, where the first message is used to notify the terminal to play each video in the video set;

[0006] Based on the first message, play the video frame of each video, and send the video sequence number of each video and the video frame sequence number of each video frame to the server; the video sequence number is used to represent the play order of each video, and the video frame sequence number is used to represent the play order of the video frame of each video;

[0007] The maximum sequence number of the video frame and the minimum sequence number of the video frame sent by the receiving server; the maximum sequence number of the video frame is the maximum sequence number of the video frame sequence numbers sent by the terminal and other terminals to the server at the first time point; the minimum sequence number of the video frame is the minimum sequence number of the video frame sequence numbers sent by the terminal and other terminals to the server at the first time point;

[0008] Based on the video frame sequence number, the maximum video frame sequence number, and the minimum video frame sequence number, the video playback mode of the terminal is adjusted to synchronize the video playback between the terminal and other terminals.

[0009] In one embodiment, before receiving the first message sent by the server, the method further includes:

[0010] Receiving a program to be played sent by a server, the program to be played includes program information and program content, the program information includes a start time point of the program to be played, and the program content includes a video set corresponding to the program to be played;

[0011] At a second time point before the start time point, receiving a second message sent by the server, the second message being used to notify the terminal to prepare for playback;

[0012] Based on the second message, within the time period formed by the start time point and the second time point, multiple original video frames in the first video with a higher playback order in the video collection are decoded to obtain multiple video frames of the first video, and the multiple video frames are cached in the cache queue of the terminal.

[0013] In one embodiment, adjusting the video playback mode of the terminal based on the video frame sequence number, the maximum video frame sequence number, and the minimum video frame sequence number includes:

[0014] The video playback mode of the terminal is adjusted based on the video frame sequence number, the maximum video frame sequence number, the minimum video frame sequence number and a preset synchronization error threshold.

[0015] In one embodiment, adjusting the video playback mode of the terminal based on the video frame sequence number, the maximum video frame sequence number, the minimum video frame sequence number, and a preset synchronization error threshold includes any one of the following:

[0016] If the video frame sequence number is equal to the maximum video frame sequence number, and the difference between the video frame sequence number and the minimum video frame sequence number is greater than or equal to a preset synchronization error threshold, then the video frame corresponding to the video frame sequence number is replayed;

[0017] If the video frame sequence number is less than the maximum video frame sequence number, and the difference between the maximum video frame sequence number and the video frame sequence number is greater than or equal to the synchronization error threshold, then after playing the video frame corresponding to the video frame sequence number, the video frame corresponding to the maximum video frame sequence number is played;

[0018] If the video frame sequence number is less than the maximum video frame sequence number, and the difference between the maximum video frame sequence number and the video frame sequence number is less than the synchronization error threshold, then after the video frame corresponding to the video frame sequence number is played, the next video frame of the video frame corresponding to the video frame sequence number will be played.

[0019] In one embodiment, if the terminal is restarted at a third time point, a video sequence number of a second video in a video set played by other terminals at the third time point and a maximum sequence number of video frames of other terminals at the third time point are obtained;

[0020] Based on the video sequence number of the second video and the maximum sequence number of the video frames of other terminals at the third time point, the video playback mode of the restarted terminal is adjusted to synchronize the video playback between the terminal and the other terminals.

[0021] In one embodiment, adjusting the video playback mode of the restarted terminal based on the video sequence number of the second video and the maximum sequence number of the video frames of other terminals at the third time point includes:

[0022] Determine, based on the video sequence number of the second video and the maximum sequence number of video frames of other terminals at a third time point, a video frame sequence number corresponding to the I frame in the second video;

[0023] If the video frame sequence number corresponding to the I frame is greater than the maximum sequence number of the video frames of other terminals at the third time point, the I frame is decoded to obtain the video frame corresponding to the I frame, and the video frame corresponding to the I frame is cached in the cache queue of the terminal;

[0024] If the maximum sequence number of the video frame of other terminals at the fourth time point is equal to the video frame sequence number corresponding to the I frame, the video frame corresponding to the I frame is played; the fourth time point is after the third time point.

[0025] In a second aspect, the present application provides a video playback method, which is executed by a server and includes:

[0026] Sending a first message to at least two terminals, where the first message is used to notify the at least two terminals to play each video in the video set;

[0027] receiving a video sequence number of each video and a video frame sequence number of a video frame of each video sent by each of at least two terminals;

[0028] Based on the video sequence number of each video sent by each terminal and the video frame sequence number of each video frame, determine the maximum video frame sequence number and the minimum video frame sequence number; the maximum video frame sequence number is the maximum sequence number of the video frame sequence numbers sent to the server by at least two terminals at a first time point; the minimum video frame sequence number is the minimum sequence number of the video frame sequence numbers sent to the server by at least two terminals at a first time point;

[0029] Send the maximum video frame sequence number and the minimum video frame sequence number to each terminal; so that each terminal adjusts the video playback mode of each terminal based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so that the video playback between the terminals remains synchronized.

[0030] In a third aspect, the present application provides a video playback device, applied to a terminal, comprising:

[0031] A first processing module, used for receiving a first message sent by the server, where the first message is used for notifying the terminal to play each video in the video set;

[0032] A second processing module is used to play the video frame of each video based on the first message, and send the video sequence number of each video and the video frame sequence number of each video frame to the server; the video sequence number is used to represent the play order of each video, and the video frame sequence number is used to represent the play order of the video frame of each video;

[0033] The third processing module is used to receive the maximum sequence number and the minimum sequence number of the video frame sent by the server; the maximum sequence number of the video frame is the maximum sequence number of the video frame sequence numbers sent by the terminal and other terminals to the server at the first time point; the minimum sequence number of the video frame is the minimum sequence number of the video frame sequence numbers sent by the terminal and other terminals to the server at the first time point;

[0034] The fourth processing module is used to adjust the video playing mode of the terminal based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so as to keep the video playing between the terminal and other terminals synchronized.

[0035] In a fourth aspect, the present application provides a video playback device, applied to a server, comprising:

[0036] A fifth processing module, configured to send a first message to at least two terminals, where the first message is used to notify the at least two terminals to play each video in the video set;

[0037] A sixth processing module, configured to receive a video serial number of each video and a video frame serial number of a video frame of each video sent by each of at least two terminals;

[0038] A seventh processing module, for determining a maximum video frame sequence number and a minimum video frame sequence number based on a video sequence number of each video and a video frame sequence number of each video sent by each terminal; the maximum video frame sequence number is the maximum sequence number of video frame sequence numbers sent to the server by at least two terminals at a first time point; the minimum video frame sequence number is the minimum sequence number of video frame sequence numbers sent to the server by at least two terminals at a first time point;

[0039] The eighth processing module is used to send the maximum video frame sequence number and the minimum video frame sequence number to each terminal; so that each terminal adjusts the video playback mode of each terminal based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so that the video playback between the terminals remains synchronized.

[0040] In a fifth aspect, the present application provides an electronic device, including: a processor, a memory, and a bus;

[0041] A bus, used to connect the processor and memory;

[0042] A memory, used for storing operation instructions;

[0043] The processor is used to execute the video playback method of the first aspect of the present application by calling an operation instruction.

[0044] In a sixth aspect, the present application provides a computer-readable storage medium storing a computer program, which is used to execute the video playback method of the first aspect of the present application.

[0045] In a seventh aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the video playback method in the first aspect of the present application.

[0046] The technical solution provided in the embodiments of the present application has at least the following beneficial effects:

[0047] The terminal receives a first message sent by the server, and the first message is used to notify the terminal to play each video in the video set; based on the first message, the terminal plays the video frame of each video, and sends the video serial number of each video and the video frame serial number of each video frame to the server; the terminal receives the maximum video frame serial number and the minimum video frame serial number sent by the server; the terminal adjusts the video playback mode of the terminal based on the video frame serial number, the maximum video frame serial number and the minimum video frame serial number, so as to keep the video playback between the terminal and other terminals synchronized; in this way, multiple terminals do not rely on time synchronization, and frame-level playback synchronization is achieved, thereby improving the accuracy of multi-terminal video playback synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0049] Figure 1 A schematic diagram of the architecture of a video playback system provided in an embodiment of the present application;

[0050] Figure 2 A flowchart of a video playback method provided in an embodiment of the present application;

[0051] Figure 3 A flowchart of another video playback method provided in an embodiment of the present application;

[0052] Figure 4 A flowchart of another video playback method provided in an embodiment of the present application;

[0053] Figure 5A schematic diagram of the structure of a video playback device provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of the structure of another video playback device provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0057] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" indicates that it is implemented as "A", or implemented as "B", or implemented as "A and B".

[0058] It can be understood that in the specific implementation of the present application, data related to video playback is involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0059] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0060] The embodiment of the present application is a video playback method provided by a video playback system, and the video playback method relates to fields such as media playback control.

[0061] In order to better understand and illustrate the solutions of the embodiments of the present application, some technical terms involved in the embodiments of the present application are briefly explained below.

[0062] I frame: I frame is an intra-frame coded frame, also known as a full-frame compressed coded frame; I frame is usually the first frame of each GOP (a video compression technology used by MPEG). After moderate compression, it can be used as a reference point for random access and can be regarded as a static image.

[0063] The solution provided by the embodiment of the present application relates to video technology. The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0064] In order to better understand the solution provided by the embodiment of the present application, the solution is described below in conjunction with a specific application scenario.

[0065] In one embodiment, Figure 1 FIG. 1 shows a schematic diagram of the architecture of a video playback system applicable to an embodiment of the present application. It can be understood that the video playback method provided in the embodiment of the present application can be applicable to but not limited to applications such as Figure 1 In the application scenario shown.

[0066] In this example, Figure 1 As shown, the architecture of the video playback system in this example may include but is not limited to a server 10, a terminal 20 and a database 30. The server 10, the terminal 20 and the database 30 may interact with each other through a network 40.

[0067] The terminal 20 receives a first message sent by the server 10, and the first message is used to notify the terminal to play each video in the video set; based on the first message, the terminal 20 plays the video frame of each video, and sends the video sequence number of each video and the video frame sequence number of each video to the server 10; the video sequence number is used to represent the play order of each video, and the video frame sequence number is used to represent the play order of the video frame of each video; the terminal 20 receives the maximum sequence number of the video frame and the minimum sequence number of the video frame sent by the server 10; the maximum sequence number of the video frame is the maximum sequence number of the video frame sequence numbers sent to the server 10 by the terminal 20 and other terminals 20 at the first time point; the minimum sequence number of the video frame is the minimum sequence number of the video frame sequence numbers sent to the server by the terminal 20 and other terminals 20 at the first time point; the terminal 20 adjusts the video play mode of the terminal 20 based on the video frame sequence number, the maximum sequence number of the video frame and the minimum sequence number of the video frame, so that the video play between the terminal 20 and the other terminals 20 remains synchronized. The server 10 sends the maximum sequence number of the video frame and the minimum sequence number of the video frame to the database 30 for storage through the network 40.

[0068] It is understandable that the above is only an example and this embodiment is not limited here.

[0069] Among them, terminals include but are not limited to smart phones (such as Android phones, iOS phones, etc.), mobile phone simulators, tablet computers, laptops, digital broadcast receivers, MIDs (Mobile Internet Devices), PDAs (Personal Digital Assistants), intelligent voice interaction devices, smart home appliances, car terminals, etc.

[0070] The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0071] The above network may include but is not limited to: wired network, wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, Wi-Fi and other networks that realize wireless communication. The specific can also be determined based on the actual application scenario requirements and is not limited here.

[0072] See also Figure 2 , Figure 2 The flowchart of a video playback method provided by an embodiment of the present application is shown, wherein the method can be executed by any electronic device, such as a terminal; as an optional implementation, the method can be executed by a terminal. For the convenience of description, in the description of some optional embodiments below, the terminal will be used as an example to illustrate the execution subject of the method. Figure 2 As shown, the video playback method provided in the embodiment of the present application includes the following steps:

[0073] S201, receiving a first message sent by a server, where the first message is used to notify a terminal to play each video in a video set.

[0074] Specifically, the web page of the server edits the program to be played, and the server sends the same program to be played to different terminals, that is, multiple terminals; the program to be played includes at least one video material, and each video material in the at least one video material can be a video, that is, the video set is a video material set. The program to be played includes program information and program content; wherein the program information includes the start time point of the program to be played (the delivery time of the program to be played, such as 10:00 a.m. on February 3, 2022), the playback time of the program to be played (for example, the playback time of the program to be played is 2 hours), the playback time of each video material in the program to be played (for example, the program to be played includes 3 video materials, and the total playback time of one video material is 30 minutes. The video material can be set to play only 10 minutes, that is, the playback time of the video material is 10 minutes), the position of the program to be played in the display of the terminal, etc.; the program content includes the video set corresponding to the program to be played, and the video set is a video material set.

[0075] The start time of the program to be played (the time of the program to be played) is based on the server time (the date and time displayed on the server); at a second time point before the start time point, the server sends a second message (a "playback preparation" message, i.e., the second message is used to notify the terminal to prepare for playback) to all terminals, and each terminal performs playback preparation after receiving the second message, wherein the time period formed by the second time point and the start time point is Tp, and Tp is, for example, 100ms (milliseconds). The terminal's playback preparation work includes (1) and (2):

[0076] (1) The terminal loads the player in advance. According to the position of the program to be played on the display of the terminal, the player is loaded at the corresponding position of the display. The player is loaded in a hidden manner and is not displayed.

[0077] (2) A video set may include multiple videos, and the video sequence number of each video is used to represent the playback order of each video; for example, a video set includes 3 videos, and the video sequence numbers of these 3 videos are 1, 2, and 3 respectively. The video corresponding to video sequence number 1 is played first, followed by the video corresponding to video sequence number 2, and finally the video corresponding to video sequence number 3. Each video includes multiple video frames, and the video frame sequence number of each video frame is used to represent the playback order of each video frame; for example, a video includes 100 video frames, and the video frame sequence numbers of these 100 video frames are 1, 2, 3, ... 100 respectively. The video frame corresponding to video frame sequence number 1, the video frame corresponding to video frame sequence number 2, the video frame corresponding to video frame sequence number 3, ... video frame 100 corresponding to video frame sequence number 100 are played in sequence.

[0078] For example, a video set includes three videos, and the video sequence numbers of these three videos are 1, 2 and 3 respectively. Multiple original video frames in the first video of the program to be played (for example, the video corresponding to the video sequence number 1) are decoded to obtain multiple video frames of the first video, and the multiple video frames are cached in the cache queue of the terminal; the above steps only perform video decoding, but do not display the played video; the number of multiple video frames can be 5-10 frames, that is, some original video frames (multiple original video frames) in the first video are decoded to obtain multiple video frames and cache them, and when multiple video frames are played, other original video frames in the first video are decoded.

[0079] It should be noted that the number of multiple video frames is related to the decoding speed of the terminal's hardware performance. If the hardware performance is high and the decoding speed is fast, more video frames are cached. If the hardware performance is average or ultra-high-definition video is used, fewer video frames are cached. Tp can be determined based on the terminal's decoding speed. If the terminal's decoding speed is fast, Tp can be set smaller; if the terminal's decoding speed is slow, Tp can be set larger.

[0080] When the start time of the program to be played arrives, the server sends a first message ("start playing" message, the first message is used to notify the terminal to play each video in the video set) to all terminals. After each terminal receives the first message, each terminal displays the player and displays and plays the video frame. The player immediately sends the video frame to the display thread for display.

[0081] S202, based on the first message, play the video frame of each video, and send the video serial number of each video and the video frame serial number of each video frame to the server; the video serial number is used to represent the playback order of each video, and the video frame serial number is used to represent the playback order of each video frame.

[0082] Specifically, when the start time of the program to be played arrives, the server sends a first message (a "start playing" message, which is used to notify the terminal to play each video in the video set) to all terminals. After each terminal receives the first message, each terminal displays a player and displays and plays the video frames. The player immediately sends the video frames to the display thread for display.

[0083] It should be noted that a waiting signal mechanism is added between video decoding and video display and playback, and the waiting signal mechanism includes: only when each terminal receives the first message ("start playing" message) sent by the server, the display and playback thread of each terminal starts to extract video frames from the cache queue of each terminal for display and playback; in this way, the start playback time of the video frames of all terminals is completely consistent, thereby eliminating the display and playback start error caused by the decoding time difference between the terminals, and at the same time eliminating the problem of increasing accumulated display and playback errors caused by different hardware performance and inconsistent decoding speeds of each terminal.

[0084] S203, receiving the maximum sequence number and the minimum sequence number of the video frame sent by the server; the maximum sequence number of the video frame is the maximum sequence number among the video frame sequence numbers sent to the server by the terminal and other terminals at the first time point; the minimum sequence number of the video frame is the minimum sequence number among the video frame sequence numbers sent to the server by the terminal and other terminals at the first time point.

[0085] Specifically, after all terminals (terminals and other terminals) start playing, when each terminal in all terminals renders and displays each video frame in video A in the video set, each terminal sends the video sequence number Vi of video A and the video frame sequence number Fi of the video frame of video A to the server. For example, at the same time point (for example, the first time point), terminal A sends the video sequence number 1 of video A and the video frame sequence number 100 of the video frame of video A to the server, terminal B sends the video sequence number 1 of video A and the video frame sequence number 98 of the video frame of video A to the server, and terminal C sends the video sequence number 1 of video A and the video frame sequence number 95 of the video frame of video A to the server; that is, at the same time point, terminal A plays the 100th video frame of video A, terminal B plays the 98th video frame of video A, and terminal C plays the 95th video frame of video A.

[0086] After the server receives the video serial number of video A and the video frame serial number of the video frame of video A reported by each terminal among all the terminals, the server puts each video frame serial number into a queue, updates the queue in time each time it receives data reported by each terminal, and then sorts the video frame serial numbers in the queue at intervals of M (M is for example 40ms) to determine the maximum video frame serial number and the minimum video frame serial number; among which, the terminal corresponding to the maximum video frame serial number is the terminal that plays the video the fastest among all the terminals, and the terminal corresponding to the minimum video frame serial number is the terminal that plays the video the slowest among all the terminals. For example, at the same time point (for example, the first time point), terminal A sends the video sequence number 1 of video A and the video frame sequence number 100 of the video frame of video A to the server, terminal B sends the video sequence number 1 of video A and the video frame sequence number 98 of the video frame of video A to the server, and terminal C sends the video sequence number 1 of video A and the video frame sequence number 95 of the video frame of video A to the server; the server sorts the video frame sequence number 100, the video frame sequence number 98, and the video frame sequence number 95, and determines that the maximum video frame sequence number Fmax is 100 and the minimum video frame sequence number Fmin is 95; the server sends the maximum video frame sequence number 100 and the minimum video frame sequence number 95 to all terminals.

[0087] It should be noted that M can be adjusted according to actual conditions over a period of time. If the terminal performance varies greatly or ultra-high-definition video is decoded, M can be set to be smaller than 40ms. In this way, M performs multi-terminal synchronization every certain period of time, that is, multi-terminal synchronization is triggered in a timely manner.

[0088] S204: Based on the video frame sequence number, the maximum video frame sequence number, and the minimum video frame sequence number, adjust the video play mode of the terminal to synchronize the video play between the terminal and other terminals.

[0089] Specifically, based on the video frame sequence number Fi, the maximum video frame sequence number Fmax and the minimum video frame sequence number Fmin, the video playback mode of the terminal is adjusted, so that each terminal can always maintain a synchronization error of no more than 2 frames; because the terminal with faster playback will wait for the terminal with slower playback, the terminal with lower hardware performance will not have the phenomenon of fast-forwarding the video due to catching up with the playback.

[0090] In an embodiment of the present application, a terminal receives a first message sent by a server, and the first message is used to notify the terminal to play each video in a video set; based on the first message, the terminal plays the video frame of each video, and sends the video serial number of each video and the video frame serial number of each video frame to the server; the terminal receives the maximum video frame serial number and the minimum video frame serial number sent by the server; the terminal adjusts the video playback mode of the terminal based on the video frame serial number, the maximum video frame serial number and the minimum video frame serial number, so as to keep the video playback between the terminal and other terminals synchronized; in this way, multiple terminals achieve frame-level playback synchronization without relying on time synchronization, thereby improving the accuracy of multi-terminal video playback synchronization.

[0091] In one embodiment, before receiving the first message sent by the server, steps A1 to A3 are also included:

[0092] Step A1, receiving a program to be played sent by a server, wherein the program to be played includes program information and program content, wherein the program information includes a start time point of the program to be played, and the program content includes a video set corresponding to the program to be played.

[0093] Specifically, the server sends the same program to be played to different terminals, that is, multiple terminals; the program to be played includes at least one video material, and each video material in the at least one video material can be a video, that is, the video set is a video material set. The program to be played includes program information and program content; wherein the program information includes the start time point of the program to be played (the delivery time of the program to be played, such as 10:00 a.m. on February 3, 2022), the playback time of the program to be played (for example, the playback time of the program to be played is 2 hours), the playback time of each video material in the program to be played (for example, the program to be played includes 3 video materials, and the total playback time of one video material is 30 minutes. The video material can be set to play only 10 minutes, that is, the playback time of the video material is 10 minutes), the position of the program to be played in the display of the terminal, etc.; the program content includes the video set corresponding to the program to be played, and the video set is a video material set.

[0094] Step A2: at a second time point before the start time point, receiving a second message sent by the server, where the second message is used to notify the terminal to prepare for playing.

[0095] Specifically, the start time of the program to be played (the time of delivery of the program to be played) is based on the server time (the date and time displayed on the server); at a second time point before the start time point, the server sends a second message (a "playback preparation" message, i.e., the second message is used to notify the terminal to prepare for playback) to all terminals, and each terminal performs playback preparation after receiving the second message, wherein the time period formed by the second time point and the start time point is Tp, and Tp is, for example, 100 ms (milliseconds). The terminal's playback preparation includes (1) and (2):

[0096] (1) The terminal loads the player in advance. According to the position of the program to be played on the display of the terminal, the player is loaded at the corresponding position of the display. The player is loaded in a hidden manner and is not displayed.

[0097] (2) A video set may include multiple videos, and the video sequence number of each video is used to represent the playback order of each video; for example, a video set includes 3 videos, and the video sequence numbers of these 3 videos are 1, 2, and 3 respectively. The video corresponding to video sequence number 1 is played first, followed by the video corresponding to video sequence number 2, and finally the video corresponding to video sequence number 3. Each video includes multiple video frames, and the video frame sequence number of each video frame is used to represent the playback order of each video frame; for example, a video includes 100 video frames, and the video frame sequence numbers of these 100 video frames are 1, 2, 3, ... 100 respectively. The video frame corresponding to video frame sequence number 1, the video frame corresponding to video frame sequence number 2, the video frame corresponding to video frame sequence number 3, ... video frame 100 corresponding to video frame sequence number 100 are played in sequence.

[0098] For example, the video set includes 3 videos, and the video sequence numbers of these 3 videos are 1, 2 and 3 respectively. Multiple original video frames in the first video of the program to be played (for example, the video corresponding to the video sequence number 1) are decoded to obtain multiple video frames of the first video, and the multiple video frames are cached in the cache queue of the terminal; the above steps only perform video decoding, but do not display the played video. The number of multiple video frames can be 5-10 frames, that is, some original video frames (multiple original video frames) in the first video are decoded to obtain multiple video frames and cache them. When multiple video frames are played, other original video frames in the first video are decoded.

[0099] Step A3, based on the second message, within the time period formed by the start time point and the second time point, multiple original video frames in the first video with a higher playback order in the video collection are decoded to obtain multiple video frames of the first video, and the multiple video frames are cached in the cache queue of the terminal.

[0100] Specifically, for example, the video set includes 3 videos, and the video sequence numbers of these 3 videos are 1, 2 and 3 respectively. The playback order of these 3 videos is the video corresponding to video sequence number 1, the video corresponding to video sequence number 2, and the video corresponding to video sequence number 3; the terminal first plays the video corresponding to video sequence number 1, that is, the video corresponding to video sequence number 1 is the first video; after the terminal plays the video corresponding to video sequence number 1, the terminal plays the video corresponding to video sequence number 2, that is, the video corresponding to video sequence number 2 is the first video. Decode multiple original video frames in the first video of the program to be played to obtain multiple video frames of the first video, and cache the multiple video frames in the cache queue of the terminal; the above steps only perform video decoding, but do not display the played video. The number of multiple video frames can be 5-10 frames, that is, decode part of the original video frames (multiple original video frames) in the first video to obtain multiple video frames and cache them, and when multiple video frames are played, decode other original video frames in the first video.

[0101] In one embodiment, adjusting the video playback mode of the terminal based on the video frame sequence number, the maximum video frame sequence number, and the minimum video frame sequence number includes:

[0102] The video playback mode of the terminal is adjusted based on the video frame sequence number, the maximum video frame sequence number, the minimum video frame sequence number and a preset synchronization error threshold.

[0103] Specifically, based on the video frame sequence number Fi, the maximum video frame sequence number Fmax, the minimum video frame sequence number Fmin and the preset synchronization error threshold, the video playback mode of the terminal is adjusted; wherein the preset synchronization error threshold can be 2, so that each terminal can always maintain a synchronization error of no more than 2 frames.

[0104] In one embodiment, adjusting the video playback mode of the terminal based on the video frame sequence number, the maximum video frame sequence number, the minimum video frame sequence number, and a preset synchronization error threshold includes any one of the following:

[0105] If the video frame sequence number is equal to the maximum video frame sequence number, and the difference between the video frame sequence number and the minimum video frame sequence number is greater than or equal to a preset synchronization error threshold, then the video frame corresponding to the video frame sequence number is replayed;

[0106] If the video frame sequence number is less than the maximum video frame sequence number, and the difference between the maximum video frame sequence number and the video frame sequence number is greater than or equal to the synchronization error threshold, then after playing the video frame corresponding to the video frame sequence number, the video frame corresponding to the maximum video frame sequence number is played;

[0107] If the video frame sequence number is less than the maximum video frame sequence number, and the difference between the maximum video frame sequence number and the video frame sequence number is less than the synchronization error threshold, then after playing the video frame corresponding to the video frame sequence number, the next video frame corresponding to the video frame sequence number is played.

[0108] Specifically, the terminal receives the maximum video frame sequence number Fmax and the minimum video frame sequence number Fmin sent by the server, and the terminal compares the video frame sequence number Fi with Fmax and Fmin respectively.

[0109] If Fi = Fmax and Fi - Fmin >= the synchronization error threshold (for example, the synchronization error threshold is 2), that is, Fi - Fmin >= 2, then the video played by this terminal is faster than that played by other terminals. When the Fi-th video frame (video frame sequence number Fi) of this terminal is played, this terminal repeats playing the Fi-th video frame to wait for other terminals with slower playback.

[0110] If Fi < Fmax and Fmax - Fi >= the synchronization error threshold (for example, the synchronization error threshold is 2), that is, Fmax - Fi >= 2, then the video played by this terminal is slower than that played by other terminals. Discard the corresponding frame to catch up with other terminals with faster playback. After this terminal plays the Fi-th video frame, this terminal directly jumps to play the Fmax-th video frame.

[0111] If Fi < Fmax and Fmax - Fi < the synchronization error threshold (for example, the synchronization error threshold is 2), that is, Fmax - Fi < 2, then after this terminal plays the Fi-th video frame, this terminal normally plays the next frame, that is, the (Fi + 1)-th video frame.

[0112] For example, three terminals (Terminal 1, Terminal 2, and Terminal 3) play a video synchronously. The server puts the sequence numbers of each video frame into a queue. The data changes in this queue can reflect the synchronization process of these three terminals, as shown in Table 1 below:

[0113] Table 1 Synchronization process of multiple terminals (three terminals)

[0114]

[0115]

[0116] It should be noted that based on the video frame sequence number Fi, the maximum video frame sequence number Fmax, the minimum video frame sequence number Fmin, and the synchronization error threshold 2, the video playback method of the terminal is adjusted; in this way, the synchronization error of each terminal can always be kept within 2 frames; since the terminal with faster playback will wait for the terminal with slower playback, for terminals with lower hardware performance, there will be no phenomenon of fast-forward playback of the video due to continuous chasing playback.

[0117] In one embodiment, if the terminal is restarted at a third time point, a video sequence number of a second video in a video set played by other terminals at the third time point and a maximum sequence number of video frames of other terminals at the third time point are obtained;

[0118] Based on the video sequence number of the second video and the maximum sequence number of the video frames of other terminals at the third time point, the video playback mode of the restarted terminal is adjusted to synchronize the video playback between the terminal and the other terminals.

[0119] Specifically, if a terminal goes offline during playback, and if the terminal is restarted and goes online, it is necessary to synchronize the playback between the terminal and other terminals that are already playing. The implementation process includes (1) and (2):

[0120] (1) After the terminal is restarted and goes online, the terminal loads the player at the corresponding position of the display according to the position of the program to be played in the display of the terminal. The player is loaded in a hidden manner and is not displayed. The terminal obtains from the server the video sequence number Vi (the video sequence number of the second video) of the video being played in the program to be played at the current (third time point) and the maximum sequence number Fmax of the video frames of other terminals at the current (third time point).

[0121] (2) When the terminal decodes the Vith video, it does not start decoding from the first video frame in the Vith video, but from the I frame. For example, the maximum serial number Fmax of the video frame of other terminals at the current (third time point) is 100. The terminal goes offline after playing the 50th video frame in the Vith video. When the terminal goes online again, it needs to synchronize with other terminals. Other terminals play to the 100th video frame (the maximum serial number Fmax of the video frame is 100), but the 100th video frame is not an I frame. The terminal needs to determine the nearest backward I frame to the Fmax frame (the 100th video frame) (for example, the I frame is the 102nd video frame) for decoding, because video decoding must start from the I frame. The original I frame is decoded to obtain the first video frame (I frame), and the first video frame is placed in the cache queue without displaying it. If the Fmax sent by the server to the terminal (the maximum video frame sequence number of other terminals at the fourth time point is 102, that is, Fmax is 102) is equal to the video frame sequence number of the first video frame in the cache queue (the video frame sequence number of the I frame is 102), the terminal starts playing the video, and the terminal maintains playback synchronization with other terminals that are currently playing the fastest.

[0122] In one embodiment, adjusting the video playback mode of the restarted terminal based on the video sequence number of the second video and the maximum sequence number of the video frames of other terminals at the third time point includes:

[0123] Determine, based on the video sequence number of the second video and the maximum sequence number of video frames of other terminals at a third time point, a video frame sequence number corresponding to the I frame in the second video;

[0124] If the video frame sequence number corresponding to the I frame is greater than the maximum sequence number of the video frames of other terminals at the third time point, the I frame is decoded to obtain the video frame corresponding to the I frame, and the video frame corresponding to the I frame is cached in the cache queue of the terminal;

[0125] If the maximum sequence number of the video frame of other terminals at the fourth time point is equal to the video frame sequence number corresponding to the I frame, the video frame corresponding to the I frame is played; the fourth time point is after the third time point.

[0126] Specifically, the terminal obtains from the server the video serial number Vi (the video serial number of the second video) of the video being played in the program to be played at the current time (third time point), as well as the maximum serial number Fmax of the video frames of other terminals at the current time (third time point); for example, the maximum serial number Fmax of the video frames of other terminals at the current time (third time point) is 100, and the terminal goes offline after playing the 50th video frame in the Vith video. When the terminal goes online again, it needs to synchronize with other terminals. Other terminals play the 100th video frame (the maximum serial number Fmax of the video frame is 100), but the 100th video frame is not an I frame. The terminal needs to determine the backward I frame closest to the Fmax frame (the 100th video frame) (for example, the I frame is the 102nd video frame) for decoding, because video decoding must start from the I frame.

[0127] The original I frame is decoded to obtain the first video frame (I frame), and the first video frame is placed in a cache queue without being displayed.

[0128] If the Fmax sent by the server to the terminal (the maximum video frame sequence number of other terminals at the fourth time point is 102, that is, Fmax is 102) is equal to the video frame sequence number of the first video frame in the cache queue (the video frame sequence number of the I frame is 102), the terminal starts playing the video, and the terminal maintains playback synchronization with other terminals that are currently playing the fastest.

[0129] In one embodiment, when a program to be played contains multiple videos, when the terminal is playing the first video, when there is still 100ms (the time 100ms is consistent with Tp) before the first video ends, that is, before the second video starts 100ms, the terminal needs to prepare to play the second video in a new thread, and the terminal's preparation for playing the second video is the same as that for playing the first video. If the terminal detects that the first video has finished playing, the terminal extracts the video frame of the second video from the terminal's cache queue and plays the video frame of the second video.

[0130] The application of the embodiments of the present application has at least the following beneficial effects:

[0131] The video frame start playback time of all terminals is completely consistent, thereby eliminating the display playback start error caused by the decoding time difference between the terminals, and at the same time eliminating the problem of increasing accumulated display playback errors caused by different hardware performance and inconsistent decoding speeds of each terminal. The terminal adjusts the terminal's video playback mode based on the video frame sequence number, the maximum video frame sequence number, and the minimum video frame sequence number to synchronize the video playback between the terminal and other terminals; in this way, multiple terminals do not rely on time synchronization and achieve frame-level playback synchronization, thereby improving the accuracy of multi-terminal video synchronization. For terminals that are restarted during playback, they are decoded by jumping to the key frame (I frame) near the fastest playback frame, and waiting for the arrival of the video frame played by other playing terminals, and then synchronized playback is performed; in this way, the frame synchronization playback of the restarted terminal and other playing terminals is guaranteed.

[0132] See also Figure 3 , Figure 3 The flowchart of a video playback method provided by an embodiment of the present application is shown, wherein the method can be executed by any electronic device, such as a server; as an optional implementation, the method can be executed by a server. For the convenience of description, in the description of some optional embodiments below, the server will be used as an example to illustrate the execution subject of the method. Figure 3 As shown, the video playback method provided in the embodiment of the present application includes the following steps:

[0133] S301, sending a first message to at least two terminals, where the first message is used to notify the at least two terminals to play each video in a video set.

[0134] Specifically, the server edits the program to be played on the web page, and the server sends the same program to be played to different terminals, that is, at least two terminals. When the start time of the program to be played arrives, the server sends a first message (a "start playing" message, which is used to notify the terminal to play each video in the video set) to all terminals.

[0135] S302: Receive a video sequence number of each video and a video frame sequence number of each video frame sent by each of at least two terminals.

[0136] Specifically, for example, at the same time point (for example, the first time point), the server receives the video sequence number 1 of video A and the video frame sequence number 100 of the video frame of video A sent by terminal A, the server receives the video sequence number 1 of video A and the video frame sequence number 98 of the video frame of video A sent by terminal B, and the server receives the video sequence number 1 of video A and the video frame sequence number 95 of the video frame of video A sent by terminal C.

[0137] S303, based on the video sequence number of each video sent by each terminal and the video frame sequence number of each video frame, determine the maximum video frame sequence number and the minimum video frame sequence number; the maximum video frame sequence number is the maximum sequence number among the video frame sequence numbers sent to the server by at least two terminals at a first time point; the minimum video frame sequence number is the minimum sequence number among the video frame sequence numbers sent to the server by at least two terminals at a first time point.

[0138] Specifically, after the server receives the video sequence number of video A and the video frame sequence number of the video frame of video A reported by each terminal among all the terminals, the server puts each video frame sequence number into a queue, updates the queue in time each time it receives data reported by each terminal, and then sorts the video frame sequence numbers in the queue at intervals of M (M, for example, 40ms) to determine the maximum video frame sequence number and the minimum video frame sequence number.

[0139] S304, sending the maximum video frame sequence number and the minimum video frame sequence number to each terminal; so that each terminal adjusts the video playback mode of each terminal based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so that the video playback between the terminals remains synchronized.

[0140] Specifically, the terminal adjusts the video playback mode of the terminal based on the video frame sequence number Fi, the maximum video frame sequence number Fmax and the minimum video frame sequence number Fmin, so that each terminal can always maintain a synchronization error of no more than 2 frames; because the terminal with faster playback will wait for the terminal with slower playback, the terminal with lower hardware performance will not experience the phenomenon of fast-forwarding the video due to catching up with the playback.

[0141] In an embodiment of the present application, the server sends a first message to at least two terminals; the server receives the video sequence number of each video and the video frame sequence number of each video frame sent by each of the at least two terminals; the server determines the maximum video frame sequence number and the minimum video frame sequence number based on the video sequence number of each video and the video frame sequence number of each video frame sent by each terminal; the server sends the maximum video frame sequence number and the minimum video frame sequence number to each terminal; so that each terminal adjusts the video playback mode of each terminal based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so that the video playback between the terminals remains synchronized; in this way, multiple terminals do not rely on time synchronization, and frame-level playback synchronization is achieved, thereby improving the accuracy of multi-terminal video playback synchronization.

[0142] In order to better understand the method provided in the embodiment of the present application, the scheme of the embodiment of the present application is further explained below with reference to examples of specific application scenarios.

[0143] In a specific application scenario embodiment, for example, a scenario where multiple terminals play videos synchronously, see Figure 4 , shows a processing flow of a video playback method, such as Figure 4 As shown, the processing flow of the video playback method provided in the embodiment of the present application includes the following steps:

[0144] S401, the server sends the same program to be played to multiple terminals.

[0145] S402, multiple terminals receive the program to be played sent by the server.

[0146] S403: The server sends a second message to multiple terminals at a second time point before the start time point of playing the program to be played.

[0147] Specifically, the second message is a "playing preparation" message, and the second message is used to notify each terminal to prepare for playing. The time period formed by the second time point and the start time point is Tp, and Tp is, for example, 100ms.

[0148] S404, each of the multiple terminals receives the second message and performs play preparation work within the time period Tp.

[0149] Specifically, each terminal loads a player in advance, and according to the position of the program to be played in the display of each terminal, the player is loaded at the corresponding position of the display. The player is loaded in a hidden manner and is not displayed. For example, the video set corresponding to the program to be played includes 3 videos, and the video sequence numbers of these 3 videos are 1, 2 and 3 respectively. The multiple original video frames in the first video of the program to be played (for example, the video corresponding to the video sequence number 1) are decoded to obtain multiple video frames of the first video, and the multiple video frames are cached in the cache queue of the terminal; the above steps only perform video decoding, but do not display the played video; the number of multiple video frames can be 5-10 frames, that is, some original video frames (multiple original video frames) in the first video are decoded to obtain multiple video frames and cache them, and when the multiple video frames are played, the other original video frames in the first video are decoded.

[0150] S405: After the start time of the program to be played arrives, the server sends a first message to multiple terminals.

[0151] Specifically, the first message is a "start playing" message, and the first message is used to notify each terminal to play each video in the video set.

[0152] S406, each terminal receives the first message sent by the server, and based on the first message, plays (renders and displays) a video frame in a certain video, and sends the video serial number of the video and the video frame serial number of the video frame to the server.

[0153] S407, the server receives the video sequence number of the video and the video frame sequence number of the video frame sent by each terminal, puts the video frame sequence number sent by each terminal into a queue, and updates the queue.

[0154] S408: At regular intervals, the server sorts the sequence numbers of the video frames in the queue to determine the maximum sequence number and the minimum sequence number of the video frames.

[0155] S409: The server sends the maximum video frame sequence number and the minimum video frame sequence number to multiple terminals.

[0156] S410, each terminal adjusts its own video playing mode based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so that the video playing between multiple terminals remains synchronized.

[0157] Specifically, any terminal among the multiple terminals receives the maximum video frame sequence number Fmax and the minimum video frame sequence number Fmin sent by the server, and the terminal compares the video frame sequence number Fi with Fmax and Fmin respectively.

[0158] If Fi = Fmax, and Fi - Fmin >= synchronization error threshold (for example, the synchronization error threshold is 2), that is, Fi - Fmin >= 2, then the video played by this terminal is faster than the videos played by other terminals. When the Fi-th video frame (video frame sequence number Fi) of this terminal is played, this terminal repeats playing the Fi-th video frame to wait for other terminals with slower playback.

[0159] If Fi < Fmax, and Fmax – Fi >= synchronization error threshold (for example, the synchronization error threshold is 2), that is, Fmax – Fi >= 2, then the video played by this terminal is slower than the videos played by other terminals. Discard the corresponding frame to catch up with other terminals with faster playback. After this terminal plays the Fi-th video frame, this terminal directly jumps to play the Fmax-th video frame.

[0160] If Fi < Fmax, and Fmax – Fi < synchronization error threshold (for example, the synchronization error threshold is 2), that is, Fmax – Fi < 2, then after this terminal plays the Fi-th video frame, this terminal normally plays the next frame, that is, the (Fi + 1)-th video frame.

[0161] Applying the embodiments of the present application has at least the following beneficial effects:

[0162] It realizes that the start times of video frames of multiple terminals are exactly the same, thereby eliminating the display and playback start errors caused by the decoding time differences among the terminals in multiple terminals. At the same time, it eliminates the problem that the accumulated display and playback errors become larger and larger due to different hardware performances and inconsistent decoding speeds of each terminal. Each terminal adjusts its own video playback method based on the video frame sequence number, the maximum video frame sequence number, and the minimum video frame sequence number to keep the video playback among multiple terminals synchronized; in this way, multiple terminals do not rely on time synchronization and achieve frame-level playback synchronization, thereby improving the accuracy of video playback synchronization among multiple terminals.

[0163] The embodiments of the present application also provide a video playback device, which is applied to a terminal. The structural schematic diagram of the video playback device is as Figure 5 shown. The video playback device 50 includes a first processing module 501, a second processing module 502, a third processing module 503, and a fourth processing module 504.

[0164] The first processing module 501 is configured to receive a first message sent by the server, and the first message is used to notify the terminal to play each video in the video set;

[0165] The second processing module 502 is configured to play the video frames of each video based on the first message, and send the video sequence number of each video and the video frame sequence number of the video frames of each video to the server; the video sequence number is used to represent the playback order of each video, and the video frame sequence number is used to represent the playback order of the video frames of each video.

[0166] The third processing module 503 is used to receive the maximum sequence number and the minimum sequence number of the video frame sent by the server; the maximum sequence number of the video frame is the maximum sequence number of the video frame sequence numbers sent by the terminal and other terminals to the server at the first time point; the minimum sequence number of the video frame is the minimum sequence number of the video frame sequence numbers sent by the terminal and other terminals to the server at the first time point;

[0167] The fourth processing module 504 is used to adjust the video playing mode of the terminal based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so as to synchronize the video playing between the terminal and other terminals.

[0168] In one embodiment, the first processing module 501 is further configured to:

[0169] Receiving a program to be played sent by a server, the program to be played includes program information and program content, the program information includes a start time point of the program to be played, and the program content includes a video set corresponding to the program to be played;

[0170] At a second time point before the start time point, receiving a second message sent by the server, the second message being used to notify the terminal to prepare for playback;

[0171] Based on the second message, within the time period formed by the start time point and the second time point, multiple original video frames in the first video with a higher playback order in the video collection are decoded to obtain multiple video frames of the first video, and the multiple video frames are cached in the cache queue of the terminal.

[0172] In one embodiment, the fourth processing module 504 is specifically configured to:

[0173] The video playback mode of the terminal is adjusted based on the video frame sequence number, the maximum video frame sequence number, the minimum video frame sequence number and a preset synchronization error threshold.

[0174] In one embodiment, the fourth processing module 504 is specifically configured to perform any one of the following:

[0175] If the video frame sequence number is equal to the maximum video frame sequence number, and the difference between the video frame sequence number and the minimum video frame sequence number is greater than or equal to a preset synchronization error threshold, then the video frame corresponding to the video frame sequence number is replayed;

[0176] If the video frame sequence number is less than the maximum video frame sequence number, and the difference between the maximum video frame sequence number and the video frame sequence number is greater than or equal to the synchronization error threshold, then after playing the video frame corresponding to the video frame sequence number, the video frame corresponding to the maximum video frame sequence number is played;

[0177] If the video frame sequence number is less than the maximum video frame sequence number, and the difference between the maximum video frame sequence number and the video frame sequence number is less than the synchronization error threshold, then after the video frame corresponding to the video frame sequence number is played, the next video frame of the video frame corresponding to the video frame sequence number will be played.

[0178] In one embodiment, the fourth processing module 504 is further configured to:

[0179] If the terminal is restarted at the third time point, obtaining the video sequence number of the second video in the video set played by other terminals at the third time point, and the maximum sequence number of the video frames of other terminals at the third time point;

[0180] Based on the video sequence number of the second video and the maximum sequence number of the video frames of other terminals at the third time point, the video playback mode of the restarted terminal is adjusted to synchronize the video playback between the terminal and the other terminals.

[0181] In one embodiment, the fourth processing module 504 is specifically configured to:

[0182] Determine, based on the video sequence number of the second video and the maximum sequence number of video frames of other terminals at a third time point, a video frame sequence number corresponding to the I frame in the second video;

[0183] If the video frame sequence number corresponding to the I frame is greater than the maximum sequence number of the video frames of other terminals at the third time point, the I frame is decoded to obtain the video frame corresponding to the I frame, and the video frame corresponding to the I frame is cached in the cache queue of the terminal;

[0184] If the maximum sequence number of the video frame of other terminals at the fourth time point is equal to the video frame sequence number corresponding to the I frame, the video frame corresponding to the I frame is played; the fourth time point is after the third time point.

[0185] The application of the embodiments of the present application has at least the following beneficial effects:

[0186] The terminal receives a first message sent by the server, and the first message is used to notify the terminal to play each video in the video set; based on the first message, the terminal plays the video frame of each video, and sends the video serial number of each video and the video frame serial number of each video frame to the server; the terminal receives the maximum video frame serial number and the minimum video frame serial number sent by the server; the terminal adjusts the video playback mode of the terminal based on the video frame serial number, the maximum video frame serial number and the minimum video frame serial number, so as to keep the video playback between the terminal and other terminals synchronized; in this way, multiple terminals do not rely on time synchronization, and frame-level playback synchronization is achieved, thereby improving the accuracy of multi-terminal video playback synchronization.

[0187] The embodiment of the present application also provides a video playback device applied to a server. The structural diagram of the video playback device is shown in FIG. Figure 6 As shown, the video playback device 60 includes a fifth processing module 601 , a sixth processing module 602 , a seventh processing module 603 and an eighth processing module 604 .

[0188] A fifth processing module 601 is used to send a first message to at least two terminals, where the first message is used to notify the at least two terminals to play each video in the video set;

[0189] A sixth processing module 602 is configured to receive a video serial number of each video and a video frame serial number of a video frame of each video sent by each of at least two terminals;

[0190] The seventh processing module 603 is used to determine a maximum video frame sequence number and a minimum video frame sequence number based on the video sequence number of each video and the video frame sequence number of each video frame sent by each terminal; the maximum video frame sequence number is the maximum sequence number of the video frame sequence numbers sent to the server by at least two terminals at a first time point; the minimum video frame sequence number is the minimum sequence number of the video frame sequence numbers sent to the server by at least two terminals at the first time point;

[0191] The eighth processing module 604 is used to send the maximum video frame sequence number and the minimum video frame sequence number to each terminal; so that each terminal adjusts the video playback mode of each terminal based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so that the video playback between the terminals remains synchronized.

[0192] The application of the embodiments of the present application has at least the following beneficial effects:

[0193] The server sends a first message to at least two terminals; the server receives the video sequence number of each video and the video frame sequence number of each video frame sent by each of the at least two terminals; the server determines the maximum video frame sequence number and the minimum video frame sequence number based on the video sequence number of each video and the video frame sequence number of each video frame sent by each terminal; the server sends the maximum video frame sequence number and the minimum video frame sequence number to each terminal; so that each terminal adjusts the video playback mode of each terminal based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so that the video playback between the terminals remains synchronized; in this way, multiple terminals do not rely on time synchronization, and frame-level playback synchronization is achieved, thereby improving the accuracy of multi-terminal video playback synchronization.

[0194] The present application also provides an electronic device. The structural diagram of the electronic device is as follows: Figure 7 As shown, Figure 7The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0195] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0196] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0197] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0198] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0199] Among them, electronic equipment includes but is not limited to: terminals, servers, etc.

[0200] The application of the embodiments of the present application has at least the following beneficial effects:

[0201] Multiple terminals do not rely on time synchronization and achieve frame-level playback synchronization, thereby improving the accuracy of multi-terminal video playback synchronization.

[0202] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0203] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0204] Based on the same principle as the method provided in the embodiment of the present application, the embodiment of the present application further provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in any optional embodiment of the present application.

[0205] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0206] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A video playback method, executed by a terminal, characterized in that: include: receiving a first message sent by a server, where the first message is used to notify the terminal to play each video in the video set; Based on the first message, play the video frame of each video, and send the video sequence number of each video and the video frame sequence number of the video frame of each video to the server; the video sequence number is used to represent the play order of each video, and the video frame sequence number is used to represent the play order of the video frame of each video; receiving a maximum video frame sequence number and a minimum video frame sequence number sent by a server; the maximum video frame sequence number is the maximum sequence number of video frame sequence numbers sent to the server by the terminal and other terminals at a first time point; the minimum video frame sequence number is the minimum sequence number of video frame sequence numbers sent to the server by the terminal and other terminals at the first time point; Based on the video frame sequence number, the maximum video frame sequence number, and the minimum video frame sequence number, the video play mode of the terminal is adjusted to synchronize the video play between the terminal and the other terminals.

2. The method according to claim 1, characterized in that Before receiving the first message sent by the server, the method further includes: Receiving a program to be played sent by the server, wherein the program to be played includes program information and program content, wherein the program information includes a start time point of playing the program to be played, and the program content includes the video set corresponding to the program to be played; At a second time point before the start time point, receiving a second message sent by the server, wherein the second message is used to notify the terminal to prepare for playback; Based on the second message, within the time period formed by the start time point and the second time point, multiple original video frames in the first video with a higher playback order in the video collection are decoded to obtain multiple video frames of the first video, and the multiple video frames are cached in the cache queue of the terminal.

3. The method according to claim 1, characterized in that The adjusting the video playback mode of the terminal based on the video frame sequence number, the maximum video frame sequence number, and the minimum video frame sequence number includes: The video playback mode of the terminal is adjusted based on the video frame sequence number, the maximum video frame sequence number, the minimum video frame sequence number and a preset synchronization error threshold.

4. The method according to claim 3, characterized in that The adjusting the video playback mode of the terminal based on the video frame sequence number, the maximum video frame sequence number, the minimum video frame sequence number and a preset synchronization error threshold includes any one of the following: If the video frame sequence number is equal to the maximum video frame sequence number, and the difference between the video frame sequence number and the minimum video frame sequence number is greater than or equal to a preset synchronization error threshold, then replaying the video frame corresponding to the video frame sequence number; If the video frame sequence number is less than the maximum video frame sequence number, and the difference between the maximum video frame sequence number and the video frame sequence number is greater than or equal to the synchronization error threshold, then after playing the video frame corresponding to the video frame sequence number, the video frame corresponding to the maximum video frame sequence number is played; If the video frame sequence number is less than the maximum video frame sequence number, and the difference between the maximum video frame sequence number and the video frame sequence number is less than the synchronization error threshold, then after the video frame corresponding to the video frame sequence number is played, the next video frame of the video frame corresponding to the video frame sequence number will be played.

5. The method according to claim 1, characterized in that: Also includes: If the terminal is restarted at a third time point, obtaining a video sequence number of a second video in the video set played by the other terminal at the third time point, and a maximum sequence number of a video frame of the other terminal at the third time point; Based on the video serial number of the second video and the maximum serial number of the video frame of the other terminal at the third time point, the video playback mode of the terminal after restarting is adjusted to synchronize the video playback between the terminal and the other terminal.

6. The method according to claim 5, characterized in that The adjusting the video playback mode of the restarted terminal based on the video sequence number of the second video and the maximum sequence number of the video frame of the other terminal at the third time point includes: Determine, based on the video sequence number of the second video and the maximum sequence number of the video frames of the other terminal at the third time point, a video frame sequence number corresponding to the I frame in the second video; If the video frame sequence number corresponding to the I frame is greater than the maximum sequence number of the video frame of the other terminal at the third time point, the I frame is decoded to obtain the video frame corresponding to the I frame, and the video frame corresponding to the I frame is cached in the cache queue of the terminal; If the maximum sequence number of the video frame of the other terminal at the fourth time point is equal to the video frame sequence number corresponding to the I frame, the video frame corresponding to the I frame is played; the fourth time point is after the third time point.

7. A video playback method, executed by a server, characterized in that: include: Sending a first message to at least two terminals, where the first message is used to notify the at least two terminals to play each video in the video set; Receiving a video sequence number of each video and a video frame sequence number of a video frame of each video sent by each of the at least two terminals; Determine a maximum video frame sequence number and a minimum video frame sequence number based on the video sequence number of each video and the video frame sequence number of each video sent by each terminal; the maximum video frame sequence number is the maximum sequence number of the video frame sequence numbers sent to the server by the at least two terminals at a first time point; the minimum video frame sequence number is the minimum sequence number of the video frame sequence numbers sent to the server by the at least two terminals at the first time point; Send the maximum video frame sequence number and the minimum video frame sequence number to each terminal; so that each terminal adjusts the video playback mode of each terminal based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so that the video playback between the terminals remains synchronized.

8. A video playback device, applied to a terminal, characterized in that: include: A first processing module, configured to receive a first message sent by a server, wherein the first message is used to notify the terminal to play each video in the video set; A second processing module is used to play the video frame of each video based on the first message, and send the video sequence number of each video and the video frame sequence number of the video frame of each video to the server; the video sequence number is used to represent the play order of each video, and the video frame sequence number is used to represent the play order of the video frame of each video; The third processing module is used to receive the maximum sequence number and the minimum sequence number of the video frame sent by the server; the maximum sequence number of the video frame is the maximum sequence number of the video frame sequence numbers sent to the server by the terminal and other terminals at a first time point; the minimum sequence number of the video frame is the minimum sequence number of the video frame sequence numbers sent to the server by the terminal and other terminals at the first time point; The fourth processing module is used to adjust the video playback mode of the terminal based on the video frame sequence number, the maximum video frame sequence number and the minimum video frame sequence number, so as to synchronize the video playback between the terminal and the other terminals.

9. A video playback device, applied to a server, characterized in that: include: A fifth processing module, configured to send a first message to at least two terminals, where the first message is used to notify the at least two terminals to play each video in the video set; A sixth processing module, configured to receive a video serial number of each video and a video frame serial number of a video frame of each video sent by each of the at least two terminals; a seventh processing module, configured to determine a maximum video frame sequence number and a minimum video frame sequence number based on the video sequence number of each video sent by each terminal and the video frame sequence number of the video frame of each video; the maximum video frame sequence number is the maximum sequence number of the video frame sequence numbers sent to the server by the at least two terminals at a first time point; the minimum video frame sequence number is the minimum sequence number of the video frame sequence numbers sent to the server by the at least two terminals at the first time point; The eighth processing module is used to send the maximum sequence number of the video frame and the minimum sequence number of the video frame to each terminal; so that each terminal adjusts the video playback mode of each terminal based on the video frame sequence number, the maximum sequence number of the video frame and the minimum sequence number of the video frame, so that the video playback between the terminals remains synchronized.

10. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images