Multi-channel video playback control method, device, electronic equipment and storage medium

Abstract

The embodiments of the present application disclose a playback control method, device, electronic device and storage medium for multiple videos, and relate to the fields of cloud technology, video processing and big data. The method comprises: in response to a received video playback request, obtaining at least two video streams corresponding to the video playback request, and at least two video streams are video streams corresponding to the same scene; calling a player control plug-in to create at least two players, and at least two players correspond one to one to at least two video streams; through the player control plug-in, controlling at least two players to synchronously play their respective corresponding video streams on the user interface of the client. By adopting the technical solution in the present application, in the scenario of synchronous playback of multiple video streams, the complexity of the business logic of synchronous playback of multiple video streams is reduced, and the difficulty of client development is reduced.

Description

Technical Field

[0001] The present application relates to fields such as cloud technology, video processing and big data, and in particular to a playback control method, device, electronic device and storage medium for multiple-channel videos. Background Art

[0002] At present, in some scenarios, with the increasing demand for live broadcasting, for example, in sports competitions such as games, football, basketball, etc., higher requirements are also put forward for live broadcasting.

[0003] In order to provide users with more interactive experience, the multi-channel video streaming function has begun to appear in the current live broadcast application. The client can pull different video streams to watch, which allows the audience to watch the live broadcast from multiple perspectives to a certain extent, simulating the immersive effect. In the scenario of playing multiple video streams at the same time, it is particularly important to achieve multi-perspective picture synchronization. In the existing technology, the business logic for achieving multi-perspective picture synchronization is relatively complex and the effect is not ideal. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device and storage medium for controlling the playback of multiple videos. In the scenario of synchronous playback of multiple video streams, the complexity of the business logic of synchronous playback of multiple video streams is reduced, and the difficulty of client development is reduced.

[0005] On the one hand, an embodiment of the present application provides a method for controlling the playback of multiple videos, the method comprising:

[0006] In response to the received video playback request, obtaining at least two video streams corresponding to the video playback request, wherein the at least two video streams are video streams corresponding to the same scene;

[0007] Calling a player control plug-in to create at least two players, wherein the at least two players correspond one-to-one to the at least two video streams;

[0008] The at least two players are controlled by the player control plug-in to synchronously play the corresponding video streams on the user interface of the client.

[0009] On the one hand, an embodiment of the present application provides a playback control device for multiple videos, the device comprising:

[0010] A video stream acquisition module, configured to acquire, in response to a received video playback request, at least two video streams corresponding to the video playback request, wherein the at least two video streams are video streams corresponding to the same scene;

[0011] A creation module, used for calling a player control plug-in to create at least two players, wherein the at least two players correspond one to one to the at least two video streams;

[0012] The video stream playback control module is used to control the at least two players to synchronously play their corresponding video streams on the user interface of the client through the player control plug-in.

[0013] In an optional embodiment, when calling the player control plug-in, the video stream playback control module is used to:

[0014] A thread corresponding to the player control plug-in is created, and the player control plug-in is called through the thread corresponding to the player control plug-in, wherein the thread corresponding to the player control plug-in and the thread corresponding to the player belong to the same process.

[0015] In an optional embodiment, when the video stream playback control module controls the at least two players to synchronously play their respective corresponding video streams on the user interface of the client through the player control plug-in, it is used to:

[0016] Repeat the following operations through the above player control plug-in until the playback end condition is met:

[0017] Obtain the playback progress of the video stream played by each of the above players;

[0018] The slowest playback progress among the above-mentioned playback progress is taken as the benchmark playback progress, and the playback progress of the video stream played by the above-mentioned other players is controlled according to the playback progress difference between the corresponding playback progress of each other player and the benchmark playback progress, wherein the above-mentioned other players refer to each player among the above-mentioned at least two players except the benchmark player, and the above-mentioned benchmark player is the player corresponding to the above-mentioned benchmark playback progress.

[0019] In an optional embodiment, the video stream playback control module is used to control the playback progress of the video streams played by the other players:

[0020] For each of the other players, if the playback progress difference corresponding to the player is greater than or equal to a first threshold, or if the playback progress difference corresponding to the player is less than or equal to a second threshold, the video stream corresponding to the player is played according to the playback progress of the player, wherein the first threshold is greater than the second threshold;

[0021] For each of the other players, if the playback progress difference corresponding to the player is smaller than the first threshold and larger than the second threshold, the playback progress of the player is controlled according to the playback progress difference corresponding to the player.

[0022] In an optional embodiment, for each of the other players, the video stream playback control module, when controlling the playback progress of the player according to the playback progress difference corresponding to the player, is used to:

[0023] If the player is in the first state, the video stream of the player is controlled to be in the second state, wherein the maximum duration of the player in the second state is equal to the play progress difference, the first state is the play state, and the second state is the pause state;

[0024] If the player is in the second state, the player is kept in the second state.

[0025] In an optional embodiment, for each video stream, the video stream includes video data and audio data, the audio data and the video data have respective corresponding timestamps, and the video stream playback control module, when acquiring the playback progress of the video stream played by each of the players, is used to:

[0026] For each player, the player control plug-in performs the following operations to obtain the playback progress of the video stream played by the player:

[0027] Get the first timestamp of the audio data and the second timestamp of the video data of the video stream currently played by the player;

[0028] If the time difference between the first timestamp and the second timestamp is greater than or equal to a third threshold, controlling the synchronization of the audio data and the video data played by the player according to the time difference;

[0029] The playing time information of the synchronized audio data and video data is used as the playing progress of the player.

[0030] In an optional embodiment, for each video stream, the video stream includes video data and audio data, and the video stream playback control module, when calling the player control plug-in to control the at least two players to synchronously play the corresponding video streams on the user interface of the client, is used to:

[0031] Call the above-mentioned player control plug-in to control the player corresponding to the target video stream among the above-mentioned at least two video streams to play the audio data and video data of the target video stream, and control other players except the player corresponding to the above-mentioned target video stream to play the video data of their corresponding video streams, wherein the above-mentioned target video stream is one of the above-mentioned at least two video streams.

[0032] In an optional embodiment, the user interface of the client includes a first display area and at least one second display area, and the video stream playback control module, when controlling the at least two players to synchronously play their respective corresponding video streams on the user interface of the client, is used to:

[0033] Controlling a first player of the at least two players to play a video stream corresponding to the first player in a first display area of ​​a user interface of the client, and controlling each second player except the first player to play a video stream corresponding to the first player in each second display area;

[0034] Wherein, the target video stream is the video stream corresponding to the first display area;

[0035] The above device also includes a video stream switching module, which is used to:

[0036] In response to the user's video stream switching operation, the player control plug-in controls the second player corresponding to the video stream switching operation to play the corresponding video stream in the first display area, and controls the first player to play the corresponding video stream in the second display area.

[0037] On the one hand, an embodiment of the present application provides an electronic device, which includes a processor and a memory, which are interconnected; the memory is used to store computer programs; the processor is configured to execute a method provided by any possible implementation of the above-mentioned multi-channel video playback control method when calling the above-mentioned computer program.

[0038] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement a method provided by any possible implementation of a method for controlling the playback of multiple videos.

[0039] On the one hand, the embodiments of the present application provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided by any possible implementation of the above-mentioned multi-channel video playback control method.

[0040] The beneficial effects of the technical solution provided by this application are:

[0041] In the embodiment of the present application, the playback control method, device, electronic device and storage medium of multiple videos provided in the embodiment of the present application initiate a video playback request through the client, and in response to the video playback request, obtain at least two video streams corresponding to the video playback request, and the at least two video streams are video streams corresponding to the same scene, and then, create at least two players, and the at least two players correspond to the at least two video streams one by one, and then, by calling the player control plug-in, control the at least two players to synchronously play their respective corresponding video streams on the video playback interface of the client. With the above technical solution, by calling the player control plug-in, the synchronous playback of each video stream on the user interface of the client is controlled, and the client only needs to receive the result of the synchronous playback of multiple video streams by the player control plug-in, which reduces the complexity of the business logic of the client to realize the synchronous playback of multiple video streams and reduces the difficulty of client development. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a schematic diagram of an implementation flow of a multi-channel video playback control system provided by an embodiment of the present application in an application scenario;

[0044] Figure 2 It is a flowchart of a method for controlling the playback of multiple videos provided in an embodiment of the present application;

[0045] Figure 3 It is a schematic diagram of the principle of synchronous playback of multiple players provided in an embodiment of the present application;

[0046] Figure 4 This is a flow chart of opening a player control plug-in provided in an embodiment of the present application;

[0047] Figure 5 It is a schematic diagram of displaying multiple video streams on a client user interface provided by an embodiment of the present application;

[0048] Figure 6 is a schematic diagram of the playback progress of a player provided in an embodiment of the present application;

[0049] Figure 7 It is a flow chart of controlling multiple players to play synchronously through a player control plug-in provided in an embodiment of the present application;

[0050] Figure 8It is a structural schematic diagram of a multi-channel video playback control device provided in an embodiment of the present application;

[0051] Fig. 9 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] The multi-channel video playback control method provided in the embodiment of the present application involves various fields of cloud technology, such as cloud computing, cloud services, cloud video, etc. in cloud technology, as well as related data computing and processing fields in the big data field.

[0054] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or a local area network to achieve data calculation, storage, processing and sharing. The multi-channel video playback control method provided in the embodiment of the present application can be implemented based on cloud computing in cloud technology.

[0055] Cloud computing refers to obtaining required resources on demand and in an easily scalable manner through the network. It is the product of the integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0056] Cloud video refers to a video network platform service based on the cloud computing business model. On the cloud platform, all video suppliers, agents, planning service providers, producers, industry associations, management agencies, industry media, legal structures, etc. are integrated into a resource pool. Each resource is displayed and interacted with each other, and communication is done on demand to reach an agreement, thereby reducing costs and improving efficiency. This concept is the concept of cloud video. For example, the video stream in the embodiment of the present application can be a video stream from the cloud platform.

[0057] Big data refers to a collection of data that cannot be captured, managed, and processed by conventional software tools within a certain time frame. It is a massive, high-growth, and diversified information asset that requires new processing models to have stronger decision-making power, insight discovery, and process optimization capabilities. With the advent of the cloud era, big data has also attracted more and more attention. Based on big data, special technologies are required to effectively implement the playback control method of multiple videos provided in this embodiment. Technologies suitable for big data include large-scale parallel processing databases, data mining, distributed file systems, distributed databases, and the above-mentioned cloud computing.

[0058] As an example, Figure 1 FIG. 1 shows a schematic diagram of an implementation flow of a playback control system for multiple video channels applicable to an embodiment of the present application in an application scenario. It can be understood that the playback control method for multiple video channels provided in the embodiment of the present application can be applied to, but not limited to, Figure 1 In the application scenario shown.

[0059] The multi-channel video playback control system in this example may include but is not limited to a user terminal 101, a network 102, and a server 103. The user terminal 101 (such as a user's smart phone) may communicate with the server 103 through the network 102. The user terminal 101 may be installed with a client of a video application, which has a video playback function, and the user may watch the video through the client.

[0060] The multi-channel video playback control method in the embodiment of the present application can be executed by any electronic device, such as a user terminal, or can be completed by the interaction between the user terminal and the server. Optionally, in the following example, it is executed by the user terminal. The specific interaction process of the multi-channel video playback control system in the present application may include steps S1-S3:

[0061] In step S1, the user initiates a video playback request in the client of the video application through the human-computer interaction screen of the user terminal 101. For example, the user opens the client of the video application in the user terminal, clicks on the video of interest to watch on the client, and the video provides at least two video streams, such as sports competition videos.

[0062] The at least two video streams may also be referred to as multiple video streams, which refer to multiple independent video streams displayed on the same video playback interface (i.e., the user interface of the client) through multiple players, and one video stream corresponds to one player. Each video stream in the mutually related multiple video streams corresponds in content and needs to be displayed synchronously.

[0063] In the multi-channel video stream, there is one main video stream and several slave video streams. During the playback of the main video stream, the user can select one or more slave video streams to be displayed synchronously with the main video stream.

[0064] Taking a football match as an example, the main video stream may be a video stream facing the entire football field, the slave video stream may be a video stream facing the audience seats, a video stream facing the studio, a video stream of close-up shots of one or more athletes, and so on, without any limitation.

[0065] Step S2, the user terminal 101 receives the video playback request initiated by the user through the client, and in response to the video playback request, obtains at least two video streams corresponding to the video playback request from the server 103, where the at least two video streams are video streams corresponding to the same scene.

[0066] Step S3: the user terminal 101 calls the player control plug-in to create at least two players, and the at least two players correspond one-to-one to the at least two video streams.

[0067] The player control plug-in is not included in the client, or in other words, the player control plug-in is a plug-in outside the client. The calling here means calling the player control plug-in that is not included in the client.

[0068] When creating at least two players, an additional player control plug-in is called, at least two players are created through the player control plug-in, and the at least two players are controlled to synchronously play their corresponding video streams on the user interface of the client.

[0069] Step S4: the user terminal 101 controls at least two players to synchronously play their corresponding video streams on the user interface of the client by calling the player control plug-in.

[0070] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. The above-mentioned network may include but is not limited to: wired network, wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, Wi-Fi and other networks that realize wireless communication. The user terminal can be a smart phone (such as Android mobile phone, iOS mobile phone, etc.), a tablet computer, a laptop computer, a digital broadcast receiver, MID (Mobile Internet Devices, mobile Internet device), PDA (personal digital assistant), desktop computer, vehicle terminal (such as vehicle navigation terminal), smart speaker, smart watch, etc. The user terminal and the server can be directly or indirectly connected by wired or wireless communication, but it is not limited to this. It can also be determined based on the actual application scenario requirements, and is not limited here.

[0071] Optionally, the server 103 may be a server of the application or a CDN server communicating with the server of the application. The CDN server may receive a video playback request from the server of the application and send at least two video streams corresponding to the video playback request to the application in response to the video playback request.

[0072] See also Figure 2 , Figure 2 1 is a flowchart of a multi-channel video playback control method provided by an embodiment of the present application. The method can be executed by any electronic device, can be executed by a user terminal (including but not limited to a mobile phone, a computer, a television, etc.), can be completed by the interaction between the user terminal and the server, can also be executed by the client, and optionally, can be executed by the user terminal, such as Figure 2 As shown, the playback control method of multiple videos provided in the embodiment of the present application includes the following steps:

[0073] Step S201: in response to a received video playback request, obtaining at least two video streams corresponding to the video playback request, wherein the at least two video streams are video streams corresponding to the same scene.

[0074] Step S202: calling a player control plug-in to create at least two players, wherein the at least two players correspond one-to-one to the at least two video streams.

[0075] Step S203: Control the at least two players to synchronously play their respective corresponding video streams on the user interface of the client through the player control plug-in.

[0076] Optionally, the above-mentioned client can be a web client, a mini-program client or an application (Application, APP for short) client. The client can be a client of a social media application, and the client has the function of playing multiple video streams simultaneously. For example, the client can be a video application client, etc. The embodiments of the present application are not limited here.

[0077] It is understandable that the at least two players mentioned above refer to multiple player instances created by the terminal device, and each player instance can be created by calling the multimedia service component of the operating system. The role of the player is to parse the acquired video stream and play the parsed video stream.

[0078] In some application scenarios, in order to improve the user's viewing experience, multiple perspectives can be provided so that users can watch videos from multiple perspectives. For example, in some sports competitions, users can watch multiple perspectives at the same time. Taking football matches as an example, users can see multiple perspectives switched by the director, such as the audience seats, the entire stadium, the studio, close-ups of players, close-ups of football, etc. In this process, it is particularly important to achieve simultaneous playback of multiple perspectives.

[0079] In order to achieve synchronous playback of multiple video streams, an optional implementation method is to pull multiple video streams from the front-end server (i.e., server 103 described above, that is, the server corresponding to the client) through the client, and then control multiple players to synchronously play their corresponding video streams. In the process of controlling these multiple players to synchronize the players, the client needs to frequently obtain the current playback position of each player with each player in a cross-process communication manner, that is, frequently obtain the current playback progress of each player, and frequently communicate with each player, which greatly increases the system performance loss of the client, increases the business logic of the client, and increases the difficulty of client development.

[0080] In order to solve the above problems and better achieve synchronous playback of multiple video streams, the following takes a live football match as an example. Figure 3 The playback control method of multiple videos in the embodiment of the present application is described. The implementation process of the method may include the following:

[0081] The recording device (such as a camera, etc.) is used to record the stream in real time from multiple different angles. The recording device can also be called an audio and video acquisition device, which has the function of recording video and audio, and collects multiple video streams in real time and transmits them to the front-end server (that is, the server 103 described above, that is, the server corresponding to the client). The front-end server pushes them to various existing networks, and the client pulls the streams of different video streams to watch, and the client synchronously plays the corresponding video streams through multiple players. The specific process is as follows:

[0082] Step 1: At the scene of the football match, multiple live images and live audio of the football match are recorded from multiple different perspectives through multiple recording devices to form corresponding multiple video streams, and the recorded multiple video streams are transmitted to the front-end server.

[0083] For each video stream, the video stream includes audio data and video data, and both the audio data and the video data carry respective corresponding timestamps.

[0084] Among them, the bit rate specifications of the video stream transmitted by the recording device to the front-end server may include 720P30, 1080P50, 1080P60, etc., wherein P means progressive scan, 720 means that the resolution of each frame is 1280*720, 1080 means that the resolution of each frame is 1920*1080, and 30 / 50 / 60 means that 30 / 50 / 60 frames are played per second. Taking the bit rate specification of the video stream as 720P30 as an example, it means that the resolution of each frame in the video stream is 1280*720, and 30 frames are played per second.

[0085] In practical applications, there is no limit on the number of recording devices, and it can be configured according to actual needs or experience. For example, the number of recording devices can be determined according to the number of selected viewing angles, and at least one spare recording device can be provided. As an example, Figure 3 Three recording devices are shown, namely recording device 1, recording device 2, recording device 3 and recording device 4.

[0086] Step 2: The front-end server receives the video stream recorded by the recording device.

[0087] Step 3: The user selects at least two viewing angles of interest through the client interface in the user terminal or directly selects at least two default viewing angles provided by the client, and initiates a video playback request through the client human-computer interaction interface.

[0088] In response to the video playback request, at least two video streams corresponding to the video playback request are pulled from the front-end server, and the two video streams are both video streams in the above-mentioned football game scene.

[0089] Among them, when the user terminal pulls at least two video streams from the front-end server, that is, when the front-end server pushes the video stream to the user terminal, the pushed video stream is a high-quality code stream, and the server maintains efficient and high-stability real-time transmission. The front-end server needs to ensure that the video stream transmitted to the client has low latency and no packet loss through mechanisms such as packet loss retransmission and congestion control.

[0090] The front-end server may be a CDN server, so that the user can obtain the desired content nearby, that is, obtain at least two video streams corresponding to the video playback request nearby, thereby improving the response speed to the user access request.

[0091] Step 4: Call the player control plug-in to create at least two players, wherein the at least two players correspond one-to-one to at least two video streams.

[0092] In order to achieve synchronous playback of multiple video streams, it is also necessary to create at least two players corresponding to at least two video streams. Figure 3 As shown, four players are created, namely player 1, player 2, player 3 and player 4. The at least two players created refer to multiple player instances created by the terminal device, and each player instance can be created by calling the multimedia service component of the operating system. The role of the player is to parse the acquired video stream and play the parsed video stream.

[0093] Step 5: Control at least two players to synchronously play their corresponding video streams on the user interface of the client by calling the player control plug-in.

[0094] The player control plug-in can be understood as a toolkit that has the function of enabling multiple players to synchronously play their respective video streams.

[0095] The detailed process of calling the player control plug-in is as follows Figure 4 As shown, the specific steps are as follows:

[0096] Step S401: The client calls the underlying central control player and starts the multicast screen synchronization function.

[0097] Among them, the bottom-level central control player can be understood as the interface of the player control center hub. The bottom-level central control player and the player control center hub can constitute the above-mentioned player control plug-in. Enabling the multicast screen synchronization function can be understood as, if the client supports the multi-channel video stream synchronous playback function, and the video selected by the user to watch on the client contains multiple video streams, then the multicast screen synchronization function can be enabled in response to the user's video playback request.

[0098] Step S402, detecting whether the multicast screen synchronization mode is turned on, if it is turned on, executing step S503, if it is not turned on, re-enabling the multicast screen synchronization function.

[0099] Step S403, start the multicast screen player control center hub thread, where: Figure 3 The thread corresponding to the player control center hub shown in is the thread corresponding to the player control plug-in.

[0100] In an optional embodiment, the calling of the player control plug-in includes:

[0101] A thread corresponding to the player control plug-in is created, and the player control plug-in is called through the thread corresponding to the player control plug-in, wherein the thread corresponding to the player control plug-in and the thread corresponding to the player belong to the same process.

[0102] Optionally, the player control plug-in can be called by creating a thread corresponding to the player plug-in. For each player, there is a corresponding thread, such as a decoding thread and a rendering thread. The decoding thread corresponding to the player is used to decode the acquired video stream, and the rendering thread is used to render the decoded video stream. In addition, the threads corresponding to each player and the threads corresponding to the player control plug-in belong to the same large process, so that the threads corresponding to the player and the threads corresponding to the player control plug-in can communicate in the same process, reducing cross-process communication and improving communication efficiency.

[0103] like Figure 3 As shown, the player control plug-in can be called to continuously control the above players 1, 2, 3 and 4 to synchronously play the video streams of their respective corresponding video streams in the user interface of the client. This allows viewers to watch the live broadcast from multiple perspectives, simulating an immersive effect. Figure 5 As shown, you can see the following in the client user interface: Figure 5From the display screen shown, it can be seen that the user interface of the client has four display areas, namely display area 51, display area 52, display area 53, and display area 54. The four display areas correspond to the above-mentioned four players respectively. The main screen is displayed in display area 51, and three auxiliary screens (i.e. auxiliary screen 1, auxiliary screen 2, and auxiliary screen 3 shown in the figure) are displayed in display area 52, display area 53, and display area 54 respectively. Moreover, the video streams displayed in these four display areas are different. For example, the main screen can display a panoramic picture of a football field, auxiliary screen 1 can display wonderful shots (such as a player's goal shot, an athlete's dribbling shot, etc.), auxiliary screen 2 can display a close-up shot of an athlete, and auxiliary screen 3 can display an audience seat picture. Among them, there can be only one audio played, such as the audio played can be the live commentary of the host in the studio, which is not limited here.

[0104] Optionally, the video streams displayed in the four display areas match the corresponding video streams, and may be partially the same, completely the same, or completely different, which is not limited in the embodiment of the present application.

[0105] Compared with the implementation of the prior art, it can be seen from the above steps 1 to 5 that in the embodiment of the present application Figure 3 In the implementation shown, when the video streams of at least two players corresponding to each other are played synchronously on the user interface of the client, it is controlled by calling the player control plug-in, which greatly reduces the business logic of the client. It also has the optimization characteristics of convenient acquisition of player information, high timeliness, reduced cross-process communication between the client and the player, and reduced CPU load.

[0106] The following describes in detail the process of using the player control plug-in to achieve multi-screen synchronous playback.

[0107] In an optional embodiment, the controlling the at least two players to synchronously play the video streams of the respective corresponding video streams on the user interface of the client through the player control plug-in includes:

[0108] Repeat the following operations through the above player control plug-in until the playback end condition is met:

[0109] Obtain the playback progress of the video stream played by each of the above players;

[0110] The slowest playback progress among the above-mentioned playback progress is taken as the benchmark playback progress, and the playback progress of the video stream played by the above-mentioned other players is controlled according to the playback progress difference between the corresponding playback progress of each other player and the benchmark playback progress, wherein the above-mentioned other players refer to each player among the above-mentioned at least two players except the benchmark player, and the above-mentioned benchmark player is the player corresponding to the above-mentioned benchmark playback progress.

[0111] Optionally, since playing video streams is a continuous process, multiple video streams need to be played synchronously continuously, so it is necessary to repeatedly play the multiple video streams synchronously through the player control process. In actual applications, the player control process can repeat the synchronous playback of multiple video streams at certain intervals.

[0112] The main principle of controlling at least two players to play synchronously by calling the player control plug-in is: by continuously obtaining the real-time playback position of each player, controlling it to pause or play, and using the slowest player as a reference clock object to achieve the multi-view screen synchronization principle. In one example, the following operations can be repeatedly performed by the player control plug-in until the playback end condition is met:

[0113] Get the playback progress of the video stream currently played by each player, where the playback progress refers to the current playback position of the video stream played by the player. In layman's terms, it refers to where the video stream played by the player is played to, such as Figure 6 As shown, assuming that the playback progress of the video stream currently played by player 1 is moment 1, the playback progress of the video stream currently played by player 2 is moment 2, the playback progress of the video stream currently played by player 3 is moment 3, and the playback progress of the video stream currently played by player 4 is moment 4, it can be seen that among these four players, the playback progress is the slowest in player 1. Therefore, player 1 can be used as a benchmark player, and the playback progress corresponding to the benchmark player can be used as the benchmark playback progress.

[0114] Then, the difference between the playback progress corresponding to player 2, player 3 and player 4 and the benchmark playback progress corresponding to the benchmark player (i.e., player 1) is calculated respectively, that is, the time difference 1 between moment 2 and moment 1, the time difference 2 between moment 3 and moment 1, and the time difference 3 between moment 4 and moment 1 are calculated. The time difference is the playback progress difference described above.

[0115] Then, the playback progress of the video streams played by the player 2, the player 3 and the player 4 is controlled respectively by using the obtained time difference 1, the time difference 2 and the time difference 3.

[0116] Among them, the player control process is a continuous cycle when controlling the synchronous playback of each player. The cycle period is very small, for example, the cycle period can be 20ms (milliseconds) once. The cycle end condition can be receiving a request initiated by the user to stop watching the video, or the video playback ends, which is not limited here.

[0117] Through the embodiments of the present application, the video streams of each player can be continuously synchronized through the player control process. On the one hand, it can provide users with multi-perspective video streams, improving the user's viewing experience. On the other hand, it reduces the business logic of the client.

[0118] The following describes how to control the playback progress of the video stream played by each player other than the base player.

[0119] In an optional embodiment, the calling of the player control plug-in to control the at least two players to synchronously play the corresponding video streams on the user interface of the client includes:

[0120] Repeat the following operations through the above player control plug-in until the playback end condition is met:

[0121] Obtain the playback progress of the video stream played by each of the above players;

[0122] The slowest playback progress among the above-mentioned playback progress is taken as the benchmark playback progress, and the playback progress of the video stream played by the above-mentioned other players is controlled according to the playback progress difference between the corresponding playback progress of each other player and the benchmark playback progress, wherein the above-mentioned other players refer to each player among the above-mentioned at least two players except the benchmark player, and the above-mentioned benchmark player is the player corresponding to the above-mentioned benchmark playback progress.

[0123] Optionally, considering that the human eye's attention span is within the time range of 2 seconds, the human eye can recognize that the images are out of sync when the time is greater than 2 seconds, so the first threshold can be set to 2 seconds. The second threshold can be an empirical value, such as 400ms, which is not limited here.

[0124] For each of the other players, the progress of the video stream played by the player can be controlled in the following ways:

[0125] Scenario 1: For each of the other players, if the difference in playback progress between the playback progress of the player and the benchmark player's playback progress is greater than or equal to the first threshold (2s), since the difference in playback progress between the player and the benchmark player is relatively large, greater than or equal to 2s, the human eye can recognize the asynchrony. At this time, the player can be put into a free playback state. Free playback can be understood as allowing the player to play the video stream corresponding to the player according to its own playback progress.

[0126] For example, Figure 6 As shown, if Figure 6 If the time difference 1 (ie, the play progress difference) between player 2 and the reference player (ie, player 1) is greater than 2s, then player 2 can be put into a free play state.

[0127] Scenario 2: For each of the other players, if the difference in playback progress between the playback progress of the player and the benchmark player's playback progress is less than or equal to the second threshold (400ms), it indicates that the difference in playback progress between the player and the benchmark player is relatively small. People cannot perceive that the player is out of sync with the benchmark player. The player can be put into a normal playback state and continue to play the corresponding video stream according to the playback progress corresponding to the player.

[0128] For example, Figure 6 As shown, if Figure 6 If the time difference 3 (ie, the play progress difference) between player 4 and the reference player (ie, player 1) is less than 400ms, then player 4 can continue to play according to its own play progress.

[0129] Scenario 3: for each of the other players, if the difference in playback progress between the playback progress of the player and the benchmark player's playback progress is greater than the second threshold (400ms) and less than the first threshold (2s), then the player needs to be synchronized for playback. Optionally, the playback progress of the player can be controlled according to the difference in playback progress between the player and the benchmark player. For example, the player can be paused for a certain period of time, and the duration of the pause is not greater than the playback progress difference.

[0130] For example, Figure 6 As shown, if Figure 6 If the time difference 2 (i.e., the playback progress difference) between player 3 and the benchmark player (i.e., player 1) is greater than 400ms and less than 2s, then player 3 can be paused for a certain period of time so that the playback progress difference between the player and the benchmark player can be reduced.

[0131] According to the above-mentioned situations, each player can be controlled to be in a relatively synchronous state.

[0132] Through the embodiments of the present application, the playback progress of other players can be controlled by the difference in playback progress between other players and the benchmark player and the size between the first threshold and the second threshold. The method is simple and flexible, and the efficiency is improved.

[0133] In an optional embodiment, for each of the other players, controlling the playback progress of the player according to the playback progress difference corresponding to the player includes:

[0134] If the player is in the first state, the video stream of the player is controlled to be in the second state, wherein the maximum duration of the player in the second state is equal to the play progress difference, the first state is the play state, and the second state is the pause state;

[0135] If the player is in the second state, the player is kept in the second state.

[0136] Optionally, for each player whose playback progress difference between the playback progress of the player in other players and the playback progress of the reference player is greater than the second threshold and less than the first threshold, the playback progress of the player can be controlled according to the playback progress difference corresponding to the player. This can be achieved in the following manner:

[0137] First, the current state information of the player is obtained. If the player is in the first state (i.e., the playing state), the player can be switched from the first state to the second state, i.e., from the playing state to the paused state, wherein the maximum duration of the player in the paused state is equal to the corresponding playing progress difference. If the player is in the second state (i.e., the paused state), the player can continue to be kept in the paused state.

[0138] Optionally, the following describes in detail the process of controlling each player to play synchronously by calling the player control plug-in in combination with an example. Figure 7 As shown, the specific process is as follows:

[0139] Step 1, the player corresponding to each video stream is enqueued, and the queue is the play queue. At the same time, the specific play time (i.e., play progress) of the play screen (i.e., video stream) of each player in the queue is obtained, and then the specific play time of each player is compared one by one, and the player with the slowest play progress is found as the benchmark player, and the specific play time corresponding to the benchmark player is used as the benchmark clock (i.e., benchmark play progress), and the audio time of the benchmark player is used as the audio reference clock. The audio reference clock can be used as the reference clock of the audio of all players. For users, the sound heard on the client is based on the audio reference clock. Wherein, the clock unit is milliseconds (ms).

[0140] When determining the specific play time of the play screen of each player, the play progress corresponding to each player may be determined according to the timestamp of the corresponding video stream.

[0141] Step 2, after obtaining the reference clock, determine the difference between the playback progress of the video stream played by other players and the reference clock, that is, determine the playback progress difference between each player and the reference player. The playback progress difference can be used as a reference threshold for adjusting the synchronous playback of multiple players, and the value range of the playback progress difference is [400ms, 2s]. That is to say, if the playback progress difference between the playback progress of other players and the playback progress of the reference player is within the range of [400ms, 2s], the playback progress of the other players needs to be adjusted so that the playback progress of the other players and the reference player is in relatively synchronous playback (relative to synchronous playback that cannot be detected by the human eye).

[0142] In an optional example, for each player whose playback progress difference with the benchmark player is greater than 400 milliseconds and less than 2 seconds, the player is considered to be a player ahead of the benchmark player and needs to enter the synchronization link, that is, to determine whether the player is in the playback state (i.e., the Start state). If so, the player is placed in the pause state (i.e., the Pause state). By pausing the player, the playback progress of the player and the benchmark player is synchronized. Since the synchronous playback is always in a loop, the next time the synchronous playback is performed, it can be determined whether the playback progress difference between the player and the benchmark player of the next loop is less than 400ms. If less than, the player can be switched from the Pause state to the Play state in the next loop.

[0143] Among them, since the human eye's attention is within 2 seconds, the human eye can recognize that the picture is out of sync for more than 2 seconds. Therefore, within a range of less than 2 seconds, the picture synchronization is achieved by pausing, which is imperceptible to the user, and there will be no picture freeze, and it will not affect the user's visual experience.

[0144] For each player whose playback progress difference with the benchmark player is greater than 2 seconds, since the human eye's attention is within 2 seconds, the human eye can recognize that the picture is out of sync after 2 seconds. Therefore, for each player whose playback progress difference with the benchmark player is greater than 2 seconds, the player will be put into free playback state and will not be synchronized with other video streams in real time. The reason is that if synchronization is performed, the picture will be paused for too long, resulting in picture freeze, affecting the visual experience.

[0145] For each player whose playing progress difference with the reference player is less than 400ms, no matter whether the player is in the playing state or not, the player can be made to play normally according to its corresponding playing progress.

[0146] In an optional embodiment, for each video stream, the video stream includes video data and audio data, the audio data and the video data have respective corresponding timestamps, and the obtaining of the playback progress of the video stream played by each of the players includes:

[0147] For each player, the player control plug-in performs the following operations to obtain the playback progress of the video stream played by the player:

[0148] Get the first timestamp of the audio data and the second timestamp of the video data of the video stream currently played by the player;

[0149] If the time difference between the first timestamp and the second timestamp is greater than or equal to a third threshold, controlling the synchronization of the audio data and the video data played by the player according to the time difference;

[0150] The playing time information of the synchronized audio data and video data is used as the playing progress of the player.

[0151] Optionally, each video stream includes video data and audio data, as well as timestamps corresponding to the video data and the audio data. For each player, in addition to ensuring synchronous playback with other players, the player itself also needs to ensure the synchronization of audio and video of the player, that is, to ensure synchronous playback of the audio data and video data corresponding to the player. The specific implementation process is as follows:

[0152] For each video stream corresponding to a player, perform the following operations through the player control plug-in:

[0153] Obtain the first timestamp of the audio data and the second timestamp of the video data of the video stream currently played by the player, determine the time difference between the first timestamp of the audio data and the second timestamp of the video data of the video stream currently played by the player, if the time difference between the first timestamp and the second timestamp is greater than or equal to the third threshold value (such as 50ms), it indicates that the playback progress of the video data is slower than the playback progress of the audio data, and a synchronization strategy of dropping some video frames to catch up with the audio clock can be adopted, if the time difference is less than the third threshold value, it indicates that the playback progress of the video data is faster than the playback progress of the audio data, and a strategy of waiting or slowing down (such as playing at 0.6 times the speed) can be adopted to make the playback progress of the audio data catch up with the playback progress of the video data, even if the video data is paused for a certain period of time or the video data is played at a speed lower than the normal speed. Among them, the third threshold value is associated with the perception of the human ear. When the threshold value is exceeded, the human ear can perceive that there is a significant interruption in the sound.

[0154] In the scenario of synchronous playback of multiple players, in order to further improve the accuracy of synchronous playback, the above-mentioned second threshold can be adjusted based on the threshold of audio and video synchronization of a single player. Theoretically, the processing accuracy of synchronous playback cannot exceed the accuracy of the synchronization time (i.e., the third threshold) of a single player. In practical applications, the second threshold can be close to the synchronization accuracy of a single player according to the scenario, and it is necessary to pay attention to the impact of single-player synchronization on the multi-player synchronization strategy. That is to say, in practical applications, the second threshold is greater than the third threshold. In an optional example, the above-mentioned second threshold can also be 400ms-50ms=350ms. The embodiment of the present application does not impose any limitation on the specific value of the second threshold.

[0155] According to the above method, the video stream corresponding to each player can be processed with audio and video synchronization, and the playback time information corresponding to the video stream after the audio and video synchronization processing is used as the playback progress of the corresponding player.

[0156] Through the embodiments of the present application, while ensuring the synchronous playback between various players, the audio and video synchronization of each player can also be guaranteed through the player control process, providing higher quality playback effects and visual experience.

[0157] In an optional embodiment, for each video stream, the video stream includes video data and audio data, and the player control plug-in is called to control the at least two players to synchronously play the corresponding video streams on the user interface of the client:

[0158] Call the player control plug-in to control the player corresponding to the target video stream among the at least two video streams to play the audio data and video data of the target video stream, and control other players except the player corresponding to the target video stream to play the video data of their corresponding video streams, wherein the target video stream is one of the at least two video streams.

[0159] Optionally, in actual applications, the user interface of the client can display the video streams of the video streams corresponding to at least two players. For each channel of video stream data, there can be corresponding audio data. If the audio data corresponding to at least two channels of video streams are played simultaneously on the client, the user will hear at least two channels of audio at the same time, which is unreasonable. In order to avoid this situation, only one audio can be provided, and the audio is the reference audio. Optionally, an audio reference clock can be set, and the audio reference clock is used as the reference clock for the audio corresponding to all video streams. The reference audio corresponding to the audio reference clock can contain only a single channel of audio, or it can contain at least two channels of audio. Taking the live broadcast of a football match as an example, the reference audio can contain only the corresponding commentary audio of the commentator, or it can be a mixed audio containing the corresponding commentary audio of the commentator and the audio of the audience in the audience seats, which is not limited here.

[0160] In an optional example, a player control plug-in can be used to control a player corresponding to a target video stream in at least two video streams to play the audio data of the target video stream, where the target video stream can be one of the at least two video streams, and other players other than the player corresponding to the target video stream can be controlled to play the video data of the video streams corresponding to the respective ones. The detailed process of playing the audio data corresponding to the target video stream is described in detail below. The target video stream is the video stream corresponding to the main screen below.

[0161] Each video stream can correspond to a reference audio, that is, each player has its own corresponding reference audio, and the reference audio corresponding to each player is matched with the video stream of the corresponding video stream. For example, taking a football match as an example, assume that there are 3 video streams, video stream 1, video stream 2 and video stream 3. Among them, video stream 1 corresponds to the studio perspective, and the corresponding reference audio is the commentator audio. Video stream 2 is the audience seat perspective, and the corresponding reference audio is the audience seat audio. Video stream 3 is the football field perspective, and the corresponding reference audio is the mixed audio corresponding to the commentator audio + audience seat audio + referee whistle. On the client, the user can freely select the main screen of interest, and control the playback of the corresponding reference audio corresponding to the main screen selected by the user. For example, assuming that the main screen selected by the user is the video stream corresponding to the football field perspective, then the audio played at this time is the mixed audio corresponding to the commentator audio + audience seat audio + referee whistle. When the user switches the main screen perspective to the studio perspective, the reference audio played can be switched to the corresponding commentator audio.

[0162] It is understandable that the above is only an example, and the embodiment of the present application does not limit how to play audio.

[0163] Through the embodiments of the present application, the player control plug-in can be used to control the audio data corresponding to the target video stream. Since the target video stream is related to the main screen perspective selected by the user, playing its corresponding audio data greatly improves the user experience.

[0164] In an optional embodiment, the user interface of the client includes a first display area and at least one second display area, and the controlling the at least two players to synchronously play the respective corresponding video streams on the user interface of the client includes:

[0165] Controlling a first player of the at least two players to play a video stream corresponding to the first player in a first display area of ​​a user interface of the client, and controlling each second player except the first player to play a video stream corresponding to the first player in each second display area;

[0166] Wherein, the target video stream is the video stream corresponding to the first display area;

[0167] Optionally, the method may further include:

[0168] In response to the user's video stream switching operation, the player control plug-in controls the second player corresponding to the video stream switching operation to play the corresponding video stream in the first display area, and controls the first player to play the corresponding video stream in the second display area.

[0169] Optionally, the user interface of the client includes a first display area and at least one second display area, such as Figure 5 As shown, the display area corresponding to the main screen is the first display area, and the display areas corresponding to the auxiliary screens 1, 2, and 3 are all the second display areas. In other words, Figure 5 The display area 51 shown in the figure is the first display area, and the display area 52, the display area 53 and the display area 54 are all the second display areas.

[0170] Controlling at least two players to synchronously play the video streams of their respective corresponding video streams on the user interface of the client through the player control plug-in can be implemented in the following manner:

[0171] The first player of the at least two players can be controlled to play its corresponding video stream in the first display area of ​​the client. The video stream played by the first player is the target video stream. That is to say, the target video stream can be played in the first display area, and each second player except the first player can be controlled to play the video stream of each corresponding video stream in each corresponding second display area.

[0172] For example, in one example, Figure 5 The display area 51 (ie, the first display area) shown displays the video stream corresponding to the first player, while the display areas 52, 53, and 54 display the video streams corresponding to other players except the first player.

[0173] Optionally, the user can initiate a target video stream switching operation, through which the user can arbitrarily switch the screen of interest to the main screen. At this time, the second player corresponding to the target video stream switching operation is controlled to play the video stream of the corresponding video stream in the first display area, and the video stream of the first player originally played in the main screen is switched to a second display area for playback.

[0174] For example, in one example, suppose that Figure 5The display area 51 (i.e., the first display area) shown displays the video stream of the video stream corresponding to player 1. The video stream of the video stream corresponding to player 2 is displayed in display area 52, the video stream of the video stream corresponding to player 3 is displayed in display area 53, and the video stream of the video stream corresponding to player 4 is displayed in display area 54. When the user's target video stream switching operation is received, for example, the video stream of the video stream corresponding to player 4 is displayed on the main screen, at this time, player 1 is controlled to display the video stream of the video stream corresponding to player 1 in display area 54, and player 4 is controlled to display the video stream of the video stream corresponding to player 4 in display area 51, and players 2 and 3 are kept in the original display area to continue to display the video streams of their respective corresponding video streams.

[0175] Through the embodiments of the present application, the video streams of multiple players can be displayed through multiple display areas, and the main screen can be switched according to the switching operation of the user's target video stream, allowing the user to arbitrarily switch the emotional screen to the main screen, thereby improving the user's viewing experience of watching videos.

[0176] See also Figure 8 , Figure 8 1 is a schematic diagram of the structure of a multi-channel video playback control device provided in an embodiment of the present application. The multi-channel video playback control device 1 provided in an embodiment of the present application includes:

[0177] The video stream acquisition module 11 is used to obtain at least two video streams corresponding to the received video playback request in response to the video playback request, wherein the at least two video streams are video streams corresponding to the same scene;

[0178] A creation module 12, used for calling a player control plug-in to create at least two players, wherein the at least two players correspond to the at least two video streams one by one;

[0179] The video stream playing control module 13 is used to control the at least two players to synchronously play their corresponding video streams on the user interface of the client through the player control plug-in.

[0180] In an optional embodiment, when calling the player control plug-in, the video stream playback control module is used to:

[0181] A thread corresponding to the player control plug-in is created, and the player control plug-in is called through the thread corresponding to the player control plug-in, wherein the thread corresponding to the player control plug-in and the thread corresponding to the player belong to the same process.

[0182] In an optional embodiment, when the video stream playback control module controls the at least two players to synchronously play their respective corresponding video streams on the user interface of the client through the player control plug-in, it is used to:

[0183] Repeat the following operations through the above player control plug-in until the playback end condition is met:

[0184] Obtain the playback progress of the video stream played by each of the above players;

[0185] The slowest playback progress among the above-mentioned playback progress is taken as the benchmark playback progress, and the playback progress of the video stream played by the above-mentioned other players is controlled according to the playback progress difference between the corresponding playback progress of each other player and the benchmark playback progress, wherein the above-mentioned other players refer to each player among the above-mentioned at least two players except the benchmark player, and the above-mentioned benchmark player is the player corresponding to the above-mentioned benchmark playback progress.

[0186] In an optional embodiment, the video stream playback control module is used to control the playback progress of the video streams played by the other players:

[0187] For each of the other players, if the playback progress difference corresponding to the player is greater than or equal to a first threshold, or if the playback progress difference corresponding to the player is less than or equal to a second threshold, the video stream corresponding to the player is played according to the playback progress of the player, wherein the first threshold is greater than the second threshold;

[0188] For each of the other players, if the playback progress difference corresponding to the player is smaller than the first threshold and larger than the second threshold, the playback progress of the player is controlled according to the playback progress difference corresponding to the player.

[0189] In an optional embodiment, for each of the other players, the video stream playback control module, when controlling the playback progress of the player according to the playback progress difference corresponding to the player, is used to:

[0190] If the player is in the first state, the video stream of the player is controlled to be in the second state, wherein the maximum duration of the player in the second state is equal to the play progress difference, the first state is the play state, and the second state is the pause state;

[0191] If the player is in the second state, the player is kept in the second state.

[0192] In an optional embodiment, for each video stream, the video stream includes video data and audio data, the audio data and the video data have respective corresponding timestamps, and the video stream playback control module, when acquiring the playback progress of the video stream played by each of the players, is used to:

[0193] For each player, the player control plug-in performs the following operations to obtain the playback progress of the video stream played by the player:

[0194] Get the first timestamp of the audio data and the second timestamp of the video data of the video stream currently played by the player;

[0195] If the time difference between the first timestamp and the second timestamp is greater than or equal to a third threshold, controlling the synchronization of the audio data and the video data played by the player according to the time difference;

[0196] The playing time information of the synchronized audio data and video data is used as the playing progress of the player.

[0197] In an optional embodiment, for each video stream, the video stream includes video data and audio data, and the video stream playback control module, when calling the player control plug-in to control the at least two players to synchronously play the corresponding video streams on the user interface of the client, is used to:

[0198] Call the above-mentioned player control plug-in to control the player corresponding to the target video stream among the above-mentioned at least two video streams to play the audio data and video data of the target video stream, and control other players except the player corresponding to the above-mentioned target video stream to play the video data of their corresponding video streams, wherein the above-mentioned target video stream is one of the above-mentioned at least two video streams.

[0199] In an optional embodiment, the user interface of the client includes a first display area and at least one second display area, and the video stream playback control module, when controlling the at least two players to synchronously play their respective corresponding video streams on the user interface of the client, is used to:

[0200] Controlling a first player of the at least two players to play a video stream corresponding to the first player in a first display area of ​​a user interface of the client, and controlling each second player except the first player to play a video stream corresponding to the first player in each second display area;

[0201] Wherein, the target video stream is the video stream corresponding to the first display area;

[0202] The above device also includes a video stream switching module, which is used to:

[0203] In response to the user's video stream switching operation, the player control plug-in controls the second player corresponding to the video stream switching operation to play the corresponding video stream in the first display area, and controls the first player to play the corresponding video stream in the second display area.

[0204] Through the embodiments of the present application, by calling the player control plug-in, the synchronous playback of each video stream on the client's user interface is controlled. The client only needs to receive the result of the synchronous playback of multiple video streams by the player control plug-in, which reduces the complexity of the client's business logic for implementing synchronous playback of multiple video streams and reduces the difficulty of client development.

[0205] In a specific implementation, the above-mentioned multi-channel video playback control device 1 can execute the above-mentioned various function modules built in it. Figure 2 For the implementation methods provided in each step, please refer to the implementation methods provided in the above steps, which will not be repeated here.

[0206] The above mainly introduces that the execution subject is hardware to implement the multi-channel video playback control method in the present application, but the execution subject of the multi-channel video playback control method in the present application is not limited to hardware. The execution subject of the multi-channel video playback control method in the present application can also be software. The above-mentioned multi-channel video playback control device can be a computer program (including program code) running in a computer device. For example, the multi-channel video playback control device is an application software; the device can be used to execute the corresponding steps in the method provided in the embodiment of the present application.

[0207] In some embodiments, the playback control device for multiple videos provided in the embodiments of the present application can be implemented by a combination of software and hardware. As an example, the playback control device for multiple videos provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the playback control method for multiple videos provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs) or other electronic components.

[0208] In other embodiments, the multi-channel video playback control device provided in the embodiments of the present application can be implemented in a software manner. Figure 8 The multi-channel video playback control device 1 shown can be software in the form of programs and plug-ins, and includes a series of modules, including a video stream acquisition module 11, a creation module 12 and a video stream playback control module, which are used to implement the multi-channel video playback control method provided in the embodiment of the present application.

[0209] See also Fig. 9 , Fig. 9 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the electronic device 1000 in this embodiment may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned electronic device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1004 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As Fig. 9 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0210] exist Fig. 9 In the electronic device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call a computer program stored in the memory 1005.

[0211] It should be understood that in some feasible implementations, the processor 1001 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0212] In a specific implementation, the electronic device 1000 can execute the above-mentioned functions through its built-in functional modules. Figure 2 For the implementation methods provided in each step, please refer to the implementation methods provided in the above steps for details, which will not be repeated here.

[0213] The present application also provides a computer-readable storage medium, which stores a computer program and is executed by a processor to implement Figure 2 For the methods provided in each step, please refer to the implementation methods provided in the above steps for details, which will not be repeated here.

[0214] The computer-readable storage medium may be an internal storage unit of the task processing device provided in any of the aforementioned embodiments, such as a hard disk or memory of an electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device. The computer-readable storage medium may also include a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. Further, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0215] The embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above Figure 2 The method provided by any possible implementation manner.

[0216] The terms "first", "second", etc. in the claims, specification and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. Displaying the phrase at various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and / or" used in the present application specification and the attached claims refers to any combination of one or more of the items listed in association and all possible combinations, and includes these combinations.

[0217] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0218] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for controlling the playback of multi-channel videos, characterized in that: include: In response to the received video playback request, obtaining at least two video streams corresponding to the video playback request, wherein the at least two video streams are video streams corresponding to the same scene; Calling a player control plug-in to create at least two players, wherein the at least two players correspond one to one to the at least two video streams; The player control plug-in repeatedly performs the following operations until a playback end condition is met, so as to control the at least two players to synchronously play their respective corresponding video streams on the user interface of the client: Obtaining the playback progress of the video stream played by each of the players; The slowest playback progress among the playback progresses is used as a benchmark playback progress, and a playback progress difference between the playback progress corresponding to each other player and the benchmark playback progress is determined; wherein the other players refer to each player among the at least two players except the benchmark player, and the benchmark player is a player corresponding to the benchmark playback progress; For each of the other players, if the playback progress difference corresponding to the player is greater than or equal to a first threshold, or if the playback progress difference corresponding to the player is less than or equal to a second threshold, the video stream corresponding to the player is played according to the playback progress of the player, wherein the first threshold is greater than the second threshold; For each of the other players, if the play progress difference corresponding to the player is smaller than the first threshold and larger than the second threshold, the play progress of the player is controlled according to the play progress difference corresponding to the player.

2. The method according to claim 1, characterized in that The calling of the player control plug-in comprises: A thread corresponding to the player control plug-in is created, and the player control plug-in is called through the thread corresponding to the player control plug-in, wherein the thread corresponding to the player control plug-in and the thread corresponding to the player belong to the same process.

3. The method according to claim 1, characterized in that For each of the other players, controlling the playback progress of the player according to the playback progress difference corresponding to the player includes: If the player is in the first state, the video stream of the player is controlled to be in the second state, wherein the maximum duration of the player in the second state is equal to the play progress difference, the first state is the play state, and the second state is the pause state; If the player is in the second state, the player is kept in the second state.

4. The method according to claim 1, characterized in that: For each video stream, the video stream includes video data and audio data, and the audio data and the video data have respective corresponding timestamps. The obtaining of the playback progress of the video stream played by each player includes: For each player, the player control plug-in performs the following operations to obtain the playback progress of the video stream played by the player: Get the first timestamp of the audio data and the second timestamp of the video data of the video stream currently played by the player; If the time difference between the first timestamp and the second timestamp is greater than or equal to a third threshold, controlling the synchronization of the audio data and the video data played by the player according to the time difference; The playing time information of the synchronized audio data and video data is used as the playing progress of the player.

5. The method according to claim 1, characterized in that For each video stream, the video stream includes video data and audio data, and the controlling the at least two players to synchronously play the corresponding video streams on the user interface of the client includes: Call the player control plug-in to control the player corresponding to the target video stream among the at least two video streams to play the audio data and video data of the target video stream, and control other players except the player corresponding to the target video stream to play the video data of their respective corresponding video streams, wherein the target video stream is one of the at least two video streams.

6. The method according to claim 5, characterized in that The user interface of the client includes a first display area and at least one second display area, and the controlling the at least two players to synchronously play the respective corresponding video streams on the user interface of the client includes: Controlling a first player of the at least two players to play a video stream corresponding to the first player in a first display area of ​​a user interface of the client, and controlling each second player except the first player to play a video stream corresponding to each second display area; Wherein, the target video stream is the video stream corresponding to the first display area; The method further comprises: In response to the user's video stream switching operation, the player control plug-in controls the second player corresponding to the video stream switching operation to play the corresponding video stream in the first display area, and controls the first player to play the corresponding video stream in the second display area.

7. A multi-channel video playback control device, characterized in that: include: A video stream acquisition module, configured to acquire, in response to a received video playback request, at least two video streams corresponding to the video playback request, wherein the at least two video streams are video streams corresponding to the same scene; A creation module, used for calling a player control plug-in to create at least two players, wherein the at least two players correspond one to one to the at least two video streams; The video stream playback control module is used to repeatedly perform the following operations through the player control plug-in until the playback end condition is met, so as to control the at least two players to synchronously play their respective corresponding video streams on the user interface of the client: Obtaining the playback progress of the video stream played by each of the players; The slowest playback progress among the playback progresses is used as a benchmark playback progress, and a playback progress difference between the playback progress corresponding to each other player and the benchmark playback progress is determined; wherein the other players refer to each player among the at least two players except the benchmark player, and the benchmark player is a player corresponding to the benchmark playback progress; For each of the other players, if the playback progress difference corresponding to the player is greater than or equal to a first threshold, or if the playback progress difference corresponding to the player is less than or equal to a second threshold, the video stream corresponding to the player is played according to the playback progress of the player, wherein the first threshold is greater than the second threshold; For each of the other players, if the play progress difference corresponding to the player is smaller than the first threshold and larger than the second threshold, the play progress of the player is controlled according to the play progress difference corresponding to the player.

8. The device according to claim 7, characterized in that When the creation module is used to call the player control plug-in, it is specifically used to: A thread corresponding to the player control plug-in is created, and the player control plug-in is called through the thread corresponding to the player control plug-in, wherein the thread corresponding to the player control plug-in and the thread corresponding to the player belong to the same process.

9. The device according to claim 7, characterized in that For each of the other players, when the video stream playback control module is used to control the playback progress of the player according to the playback progress difference corresponding to the player, it is specifically used to: If the player is in the first state, the video stream of the player is controlled to be in the second state, wherein the maximum duration of the player in the second state is equal to the play progress difference, the first state is the play state, and the second state is the pause state; If the player is in the second state, the player is kept in the second state.

10. The device according to claim 7, characterized in that For each video stream, the video stream includes video data and audio data, and the audio data and the video data have respective corresponding timestamps. When the video stream playback control module is used to obtain the playback progress of the video stream played by each player, it is specifically used to: For each player, the player control plug-in performs the following operations to obtain the playback progress of the video stream played by the player: Get the first timestamp of the audio data and the second timestamp of the video data of the video stream currently played by the player; If the time difference between the first timestamp and the second timestamp is greater than or equal to a third threshold, controlling the synchronization of the audio data and the video data played by the player according to the time difference; The playing time information of the synchronized audio data and video data is used as the playing progress of the player.

11. The device according to claim 7, characterized in that For each video stream, the video stream includes video data and audio data. When the video stream playback control module is used to control the at least two players to synchronously play the corresponding video streams on the user interface of the client, it is specifically used to: Call the player control plug-in to control the player corresponding to the target video stream among the at least two video streams to play the audio data and video data of the target video stream, and control other players except the player corresponding to the target video stream to play the video data of their respective corresponding video streams, wherein the target video stream is one of the at least two video streams.

12. The device according to claim 11, characterized in that The user interface of the client includes a first display area and at least one second display area, and the video stream playback control module is used to control the at least two players to synchronously play the corresponding video streams on the user interface of the client, specifically for: Controlling a first player of the at least two players to play a video stream corresponding to the first player in a first display area of ​​a user interface of the client, and controlling each second player except the first player to play a video stream corresponding to each second display area; Wherein, the target video stream is the video stream corresponding to the first display area; The device also includes: The video stream switching module is used to respond to the user's video stream switching operation, control the second player corresponding to the video stream switching operation to play the corresponding video stream in the first display area through the player control plug-in, and control the first player to play the corresponding video stream in the second display area.

13. An electronic device, characterized in that: comprising a processor and a memory, wherein the processor and the memory are connected to each other; The memory is used to store computer programs; The processor is configured to execute the method as claimed in any one of claims 1 to 6 when calling the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method described in any one of claims 1 to 6.

Images