Switching method, switching device and storage medium for multiple video streams

Abstract

The application discloses a switching method and device for multiple video streams, and the method comprises the following steps: firstly, acquiring multiple video streams, and determining key frames of the multiple video streams; embedding time synchronization supplementary enhancement information which is set in advance into the key frames of the multiple video streams; in response to receiving a switching request of the multiple video streams sent by a terminal, acquiring a key frame before switching and a key frame after switching in the multiple video streams; and based on the key frame before switching and the key frame after switching, realizing the switching of the multiple video streams. By obtaining the key frame for switching and embedding the calibrated time synchronization supplementary enhancement information, the switching problem of the multiple video streams can be solved, and the user experience is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of video switching, and particularly relates to a switching method, a switching device and a storage medium for multiple video streams. BACKGROUND

[0002] In the prior art, after multiple different video data are obtained, a key frame that is time-synchronized is found through key frame registration to perform video fusion. This method is simple to implement, but when the video stream is generated at a high speed or the video stream data is large, obvious delay exists. Meanwhile, this method assumes that the multiple video streams are time-synchronized, but in actual application, different methods are used to obtain multiple video streams, and it is actually impossible to keep complete synchronization. The purpose of synchronization is to switch the video streams, such as switching data of one lens to data of another lens due to the angle switching of a multi-lens camera. In order to improve the fluency of video playing during switching, the switched video needs to be synchronized to improve the effect of video playing after switching.

[0003] Therefore, how to solve the problem of asynchronization caused by the prior video switching technology, especially in the switching process of multiple different videos, has become an urgent problem to be solved. SUMMARY

[0004] The application aims to provide a switching method and device for multiple video streams to solve the problem in the prior art. The switching method and device for multiple video streams can solve the problem of switching multiple video streams and improve user experience by obtaining key frames for switching and embedding calibrated time-synchronized supplemental enhancement information.

[0005] One embodiment of the application provides a switching method for multiple video streams, which comprises the following steps.

[0006] Obtaining multiple video streams and determining key frames of the multiple video streams, wherein the multiple video streams comprise key frames and non-key frames;

[0007] Embedding pre-set time-synchronized supplemental enhancement information into the key frames of the multiple video streams, respectively;

[0008] In response to receiving a switching request of the multiple video streams sent by a terminal, obtaining a pre-switching key frame and a post-switching key frame in the multiple video streams;

[0009] Switching the multiple video streams based on the pre-switching key frame and the post-switching key frame.

[0010] Optionally, the step of embedding the pre-set time-synchronized supplemental enhancement information into the key frames of the multiple video streams, respectively, comprises the following steps.

[0011] The calibrated time synchronization supplementary enhancement information is embedded in the key frames of the multiple video streams in a preset manner, and the interval time of the time synchronization supplementary enhancement information of adjacent key frames of the same video stream is not greater than a preset time value.

[0012] Optionally, the step of embedding the calibrated time synchronization supplementary enhancement information in the key frames of the multiple video streams in a preset manner comprises:

[0013] The time synchronization supplementary enhancement information is embedded in the key frames of the multiple video streams in a preset json text format, wherein the content of the preset json text format comprises the time synchronization supplementary enhancement information, index information and time zone information.

[0014] Optionally, the step of obtaining the key frame before switching and the key frame after switching in the multiple video streams in response to receiving the switching request of the terminal comprises:

[0015] The video stream before switching and the video stream after switching are obtained based on the switching request.

[0016] The key frame before switching of the video stream before switching is obtained.

[0017] The key frame after switching of the video stream after switching is obtained based on the time synchronization supplementary enhancement information of the key frame before switching, wherein the time synchronization supplementary enhancement information of the key frame before switching and the time synchronization supplementary enhancement information of the key frame after switching are similar.

[0018] Optionally, the step of obtaining the key frame after switching of the video stream after switching based on the time synchronization supplementary enhancement information of the key frame before switching comprises:

[0019] The first closest key frame in the video stream after switching is searched in a time sequence based on the time synchronization supplementary enhancement information of the key frame before switching, and the first closest key frame is set as the key frame after switching.

[0020] Optionally, after the step of searching the first closest key frame in the video stream after switching in a time sequence based on the time synchronization supplementary enhancement information of the key frame before switching and setting the first closest key frame as the key frame after switching, the method further comprises:

[0021] The second closest key frame in the video stream after switching is searched in a time sequence.

[0022] if a difference between the time information of the time-synchronous supplemental enhancement information of the first closest key frame and the time information of the time-synchronous supplemental enhancement information of the pre-switch key frame is greater than a first set value, and a difference between the time information of the time-synchronous supplemental enhancement information of the pre-switch key frame and the time information of the time-synchronous supplemental enhancement information of the second closest key frame is less than a second set value, the second closest key frame is set as a post-switch key frame.

[0023] Optionally, the method further comprises:

[0024] if the difference between the time information of the time-synchronous supplemental enhancement information of the first closest key frame and the time information of the time-synchronous supplemental enhancement information of the pre-switch key frame is greater than the first set value, and the difference between the time information of the time-synchronous supplemental enhancement information of the pre-switch key frame and the time information of the time-synchronous supplemental enhancement information of the second closest key frame is greater than the second set value, a non-key frame between the first closest key frame and the second closest key frame is set as a post-switch key frame.

[0025] Optionally, the implementation of the switching of the multiple video streams based on the pre-switch key frame and the post-switch key frame comprises:

[0026] the switching of the pre-switch video stream and the post-switch video stream is performed using the pre-switch key frame in the pre-switch video stream and the post-switch key frame in the post-switch video stream, so as to realize the switching of the multiple video streams.

[0027] Yet another embodiment of the present application provides a device for switching of multiple video streams, the device comprising:

[0028] an acquisition module configured to acquire the multiple video streams and determine key frames of the multiple video streams, the multiple video streams comprising key frames and non-key frames;

[0029] an embedding module configured to embed pre-set time-synchronous supplemental enhancement information into the key frames of the multiple video streams respectively;

[0030] a response module configured to, in response to receiving a switching request of the multiple video streams sent by a terminal, acquire a pre-switch key frame and a post-switch key frame in the multiple video streams;

[0031] an implementation module configured to implement the switching of the multiple video streams based on the pre-switch key frame and the post-switch key frame.

[0032] Optionally, the embedding module comprises:

[0033] The embedding unit is configured to embed the calibrated time synchronization supplementary enhancement information into key frames of the multiple video streams in a preset manner, and interval time of time synchronization supplementary enhancement information of adjacent key frames of the same video stream is not greater than a preset time value.

[0034] Optionally, the embedding unit comprises:

[0035] The embedding sub-unit is configured to embed the time synchronization supplementary enhancement information into the key frames of the multiple video streams in a preset json text format, wherein the preset json text format content comprises the time synchronization supplementary enhancement information, index information and time zone information.

[0036] Optionally, the response module comprises:

[0037] The first acquisition unit is configured to acquire the pre-switch video stream and the post-switch video stream based on a switching request.

[0038] The second acquisition unit is configured to acquire a pre-switch key frame of the pre-switch video stream.

[0039] The third acquisition unit is configured to acquire a post-switch key frame of the post-switch video stream based on time synchronization supplementary enhancement information of the pre-switch key frame, wherein the time synchronization supplementary enhancement information of the pre-switch key frame is similar to time synchronization supplementary enhancement information of the post-switch key frame.

[0040] Optionally, the third acquisition unit comprises:

[0041] The backward searching sub-unit is configured to search a first closest key frame in the post-switch video stream in a time sequence based on the time synchronization supplementary enhancement information of the pre-switch key frame, and set the first closest key frame as the post-switch key frame.

[0042] Optionally, after the backward searching sub-unit, the device further comprises:

[0043] The forward searching sub-unit is configured to search a second closest key frame in the post-switch video stream in a time sequence.

[0044] The first setting sub-unit is configured to set the second closest key frame as the post-switch key frame if a difference between time information of the time synchronization supplementary enhancement information of the first closest key frame and time information of the time synchronization supplementary enhancement information of the pre-switch key frame is greater than a first setting value, and a difference between the time information of the time synchronization supplementary enhancement information of the pre-switch key frame and time information of the time synchronization supplementary enhancement information of the second closest key frame is less than a second setting value.

[0045] Optionally, the device further comprises:

[0046] The second setting sub-unit is configured to set a non-key frame between the first closest key frame and the second closest key frame as a post-switch key frame if a difference between time information of the time synchronization supplemental enhancement information of the first closest key frame and time information of the time synchronization supplemental enhancement information of the pre-switch key frame is greater than a first setting value, and a difference between the time information of the time synchronization supplemental enhancement information of the pre-switch key frame and time information of the time synchronization supplemental enhancement information of the second closest key frame is greater than a second setting value.

[0047] Optionally, the implementation module comprises:

[0048] The implementation unit is configured to perform switching of the pre-switch video stream and the post-switch video stream by using the pre-switch key frame in the pre-switch video stream and the post-switch key frame in the post-switch video stream, so as to realize switching of the multiple video streams.

[0049] Another embodiment of the present application provides a storage medium having a computer program stored therein, wherein the computer program is configured to execute the method described in any one of the above embodiments when running.

[0050] Another embodiment of the present application provides an electronic device comprising a memory and a processor, wherein the memory has a computer program stored therein, and the processor is configured to execute the computer program to execute the method described in any one of the above embodiments.

[0051] Compared with the prior art, the present application first acquires multiple video streams, determines key frames of the multiple video streams, embeds pre-set time synchronization supplemental enhancement information into the key frames of the multiple video streams, acquires a pre-switch key frame and a post-switch key frame in the multiple video streams in response to receiving a switching request of the multiple video streams sent by a terminal, and realizes switching of the multiple video streams based on the pre-switch key frame and the post-switch key frame. By obtaining the key frames for switching and embedding the calibrated time synchronization supplemental enhancement information, the present application can solve the problem of switching of the multiple video streams and improve user experience. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 A hardware structure block diagram of a computer terminal for a switching method of multiple video streams is provided for the embodiments of the present application;

[0053] Figure 2 A flowchart of a switching method of multiple video streams is provided for the embodiments of the present application;

[0054] Figure 3 A structure diagram of a switching device for multiple video streams is provided for the embodiments of the present application. Detailed Implementation

[0055] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0056] This invention first provides a method for switching multiple video streams, which can be applied to electronic devices, such as computer terminals, specifically ordinary computers, quantum computers, etc.

[0057] The following detailed explanation uses a computer terminal as an example. Figure 1 This is a hardware structure block diagram of a computer terminal for a multi-video stream switching method provided in an embodiment of the present invention. Figure 1 As shown, a computer terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the computer terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer terminal described above. For example, the computer terminal may also include components that are more complex than those described above. Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0058] The memory 104 can be used to store software programs and modules for application software, such as the program instructions / modules corresponding to the method for switching multiple video streams in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0059] The transmission device 106 is configured to receive or send data via a network. The network can include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module configured to communicate with the Internet wirelessly.

[0060] Referring to Figure 2 , Figure 2 A flowchart of a switching method for multiple video streams according to an embodiment of the present application can include the following steps:

[0061] S201: Obtain multiple video streams and determine key frames of the multiple video streams, wherein the multiple video streams include key frames and non-key frames.

[0062] Specifically, the method of obtaining multiple video streams and determining key frames of the multiple video streams can be by directly playing the video streams. During the playing process, a certain image or images in the video streams are designated as key frames, i.e., a certain picture or pictures are designated as key frames. In a specific implementation, whether the key frame information is input can be detected in real time during the playing process. If the key frame information is input, the current frame is recorded as a key frame. Otherwise, the playing process continues.

[0063] It should be noted that the time of a device for obtaining multiple video streams, such as a camera, needs to be synchronized and calibrated by a network time protocol (NTP) reference server. The camera needs to perform NTP synchronization and calibration at least once when it is turned on. The calibration frequency can be fixed at one minute. The time of the reference server is used to calibrate the standard time of the camera to ensure the accuracy of time synchronization and supplementary enhancement information. The setting frequency of the time synchronization and supplementary enhancement information should be higher than the synchronization and calibration frequency of the camera.

[0064] For example, a terminal receives multiple video streams of a first video and multiple video streams of a second video, wherein the multiple video streams of the first video include different image frames of video frames of the first video, and the multiple video streams of the second video also include different image frames of video frames of the second video.

[0065] S202: Embed time synchronization and supplementary enhancement information pre-set in the key frames of the multiple video streams, respectively.

[0066] Specifically, the time synchronization supplementary enhancement information can be a timestamp corresponding to a key frame of the multiple video streams, can be serial number information corresponding to the key frame, or can be information corresponding to the key frame that can represent the order of the key frame. The present exemplary embodiment does not make special limitations on this, for example, when the target key frame is located in the current video stream, the time synchronization supplementary enhancement information refers to the timestamp information corresponding to the target key frame. It can be customized according to user needs, for example, it can be set to 900ms or 1000ms, etc., which is not limited herein. The pre-set time synchronization supplementary enhancement information is embedded into the key frames of the multiple video streams, which can include:

[0067] The calibrated time synchronization supplementary enhancement information is embedded into the key frames of the multiple video streams in a preset manner, and the interval time of the time synchronization supplementary enhancement information of adjacent key frames of the same video stream is not greater than a preset time value.

[0068] The calibrated time synchronization supplementary enhancement information is embedded into the key frames of the multiple video streams in a preset manner, and the interval time of the time synchronization supplementary enhancement information of adjacent key frames of the same video stream is not greater than a preset time value.

[0069] It should be noted that the key frame position parameter can be a frame sequence number of the key frame, and the preset time value can be 1 to 3 seconds, or a time greater than 3 seconds can be set. However, if the preset time value is too long, it may affect the smoothness of switching.

[0070] For example, the content of the time synchronization supplementary enhancement information is in the json text format, mainly in the sync_info paragraph, which can include:

[0071] The time ts of the synchronization network time protocol (in milliseconds)

[0072] In an optional embodiment, the method for obtaining ts can be as follows:

[0073] int64_t getTs(){

[0074] struct timeval stTv;

[0075] gettimeofday(&stTv, NULL); (get current time)

[0076] return(int64_t)stTv.tv_sec*1000+stTv.tv_usec / 1000;

[0077] }

[0078] It should be noted that if the unit is second, it is multiplied by 1000; if the unit is microsecond, it is divided by 1000, so that the unit is unified, and the time zone of ts is unified UTC time.

[0079] Among them, the index information index is a 64-bit monotonically increasing integer type data starting from 0, which is increased by 1 each time a time synchronization supplementary enhancement information is output, and is a unique guide. If the receiving end receives the time synchronization supplementary enhancement information with the same index, it can be discarded.

[0080] The time zone information can be set to "UTC" or "UTC+8" time zone format, and can be set through the following pseudo code:

[0081] {

[0082] "sync_info":{

[0083] "ts":1655946642426,

[0084] "index":101,

[0085] "tz":"UTC",

[0086] }

[0087] }

[0088] S203: In response to receiving the switching request of the terminal for the multi-channel video stream, acquiring the key frame before switching and the key frame after switching in the multi-channel video stream.

[0089] Specifically, in response to receiving the switching request of the terminal for the multi-channel video stream, acquiring the key frame before switching and the key frame after switching in the multi-channel video stream can include:

[0090] Step 1: Based on the switching request, acquiring the video stream before switching and the video stream after switching.

[0091] Specifically, the switching request is an instruction carrying switching information, so the switching information can be determined based on the switching request, and the video stream before switching and the video stream after switching can be acquired. The switching request can be an instruction received from the terminal, can be an instruction acquired according to the sending rate of the current network, or can be an instruction received in other ways. For example, when the switching request instruction sent from the terminal is received, the switching information can be determined according to the switching instruction. The switching information can be time information, can be identification information of the switching position, can be pause or lock information of the video stream, or can be other information, which is not specially limited in the present exemplary embodiment.

[0092] Step 2: obtaining a pre-switch key frame of the pre-switch video stream.

[0093] Specifically, first, a pre-switch key frame in the pre-switch video stream is determined, i.e., an image frame that needs to be switched.

[0094] Step 3: obtaining a post-switch key frame of the post-switch video stream based on time synchronization supplemental enhancement information of the pre-switch key frame, wherein the time synchronization supplemental enhancement information of the pre-switch key frame and the time synchronization supplemental enhancement information of the post-switch key frame are similar.

[0095] The obtaining of the post-switch key frame of the post-switch video stream based on the time synchronization supplemental enhancement information of the pre-switch key frame can include: based on the time synchronization supplemental enhancement information of the pre-switch key frame, finding a first closest key frame in the post-switch video stream in a time sequence and setting the first closest key frame as the post-switch key frame.

[0096] The first closest key frame is the first key frame found in the post-switch video stream in a time sequence.

[0097] In an optional embodiment, after the first closest key frame in the post-switch video stream is found in a time sequence based on the time synchronization supplemental enhancement information of the pre-switch key frame and the first closest key frame is set as the post-switch key frame, the method can further include:

[0098] 1. finding a second closest key frame in the post-switch video stream in a time sequence;

[0099] 2. if a difference between time information of the time synchronization supplemental enhancement information of the first closest key frame and time information of the time synchronization supplemental enhancement information of the pre-switch key frame is greater than a first set value and a difference between the time information of the time synchronization supplemental enhancement information of the pre-switch key frame and time information of the time synchronization supplemental enhancement information of the second closest key frame is less than a second set value, setting the second closest key frame as the post-switch key frame.

[0100] If the difference between the time information of the time synchronization supplemental enhancement information of the first closest key frame and the time information of the time synchronization supplemental enhancement information of the pre-switch key frame is greater than the first set value and the difference between the time information of the time synchronization supplemental enhancement information of the pre-switch key frame and the time information of the time synchronization supplemental enhancement information of the second closest key frame is greater than the second set value, non-key frames between the first closest key frame and the second closest key frame are set as the post-switch key frame.

[0101] Specifically, based on the time synchronization supplemental enhancement information of the pre-switch key frame, a first closest key frame in the post-switch video stream is searched in time sequence backward, and a second closest key frame in the post-switch video stream is searched in time sequence forward; if a difference between the time information of the time synchronization supplemental enhancement information of the first closest key frame and the time information of the time synchronization supplemental enhancement information of the pre-switch key frame is greater than a first preset time value (for example, greater than 3 seconds), and a difference between the time information of the time synchronization supplemental enhancement information of the pre-switch key frame and the time information of the time synchronization supplemental enhancement information of the second closest key frame is less than a second preset time value (for example, less than 1 second), the second closest key frame is set as the post-switch key frame.

[0102] For example, the time synchronization supplemental enhancement information of the pre-switch key frame is 800 ms, and a first closest key frame in the post-switch video stream is searched in time sequence backward. The time synchronization supplemental enhancement information corresponding to the pre-switch key frame is determined among a plurality of post-switch video stream key frames, the time synchronization supplemental enhancement information corresponding to the pre-switch key frame and the time synchronization supplemental enhancement information in the plurality of post-switch video stream key frames are compared, and a first closest key frame is determined among the plurality of post-switch video stream key frames according to the comparison result, for example, the time synchronization supplemental enhancement information of the determined first closest key frame is 805 ms.

[0103] It should be noted that the difference between the first closest key frame and the pre-switch key frame is too large, and direct switching will affect the fluency, the difference between the second closest key frame and the pre-switch key frame is small, although the time is switched forward, but the time is short, the user does not perceive, and the user perception can also be improved.

[0104] Similarly, if the difference between the time information of the time synchronization supplemental enhancement information of the first closest key frame and the time information of the time synchronization supplemental enhancement information of the pre-switch key frame is greater than a first preset time value (for example, greater than 3 seconds), and the difference between the time information of the time synchronization supplemental enhancement information of the pre-switch key frame and the time information of the time synchronization supplemental enhancement information of the second closest key frame is greater than a second preset time value (for example, greater than 1 second), a non-key frame between the first closest key frame and the second closest key frame is selected as the post-switch key frame. That is, if the frame loss problem occurs, the time information of the pre-switch key frame and the post-switch key frame is different, at this time, the non-key frame between the two key frames of the post-switch video stream can be directly used as the post-switch key frame for video stream switching, so as to improve the fluency of video switching.

[0105] For example, time synchronization supplemental enhancement information is set for key frames of the first video and the second video. When video stream switching from the first video to the second video is needed, the last key frame before switching of the first video is set as a key frame before switching, and then a key frame after switching within a set time interval is searched for in the second video based on the time synchronization supplemental enhancement information of the key frame after switching, i.e. the time synchronization supplemental enhancement information of the key frame after switching and the time synchronization supplemental enhancement information of the key frame before switching should be less than a preset time value, and preferably the time synchronization supplemental enhancement information of the key frame after switching should be greater than the time synchronization supplemental enhancement information of the key frame before switching.

[0106] Preferably, an image frame with time synchronization supplemental enhancement information greater than the time synchronization supplemental enhancement information of the key frame after switching is selected as the key frame after switching. For example, if the first video includes time synchronization supplemental enhancement information of the key frame before switching of 1 second, and the time synchronization supplemental enhancement information of the first closest key frame backward is 1005 ms and the time synchronization supplemental enhancement information of the second closest key frame forward is 998 ms, then the image frame with the time synchronization supplemental enhancement information of the first closest key frame backward of 1005 ms is selected as the key frame after switching.

[0107] For example, if the first video includes time synchronization supplemental enhancement information of the key frame before switching of 1 second, and the time synchronization supplemental enhancement information of the first closest key frame and the second closest key frame of the closest key frame after switching of the second video are 1010 ms and 998 ms respectively, if the difference between the time information 1010 ms of the time synchronization supplemental enhancement information of the first closest key frame and the time information 1 second of the time synchronization supplemental enhancement information of the key frame before switching is greater than a first preset time value, e.g. the first preset time value is 6 ms, and the difference between the time information 1 second of the time synchronization supplemental enhancement information of the key frame before switching and the time information 998 ms of the time synchronization supplemental enhancement information of the second closest key frame is less than a second preset time value, e.g. the second preset time value is 3 ms, then the second closest key frame is set as the key frame after switching.

[0108] In an alternative embodiment, if the difference between the time information 1 second of the time synchronization supplemental enhancement information of the first closest key frame and the time information 1008 ms of the time synchronization supplemental enhancement information of the key frame before switching is greater than a first preset time value, and the difference between the time information 1 second of the time synchronization supplemental enhancement information of the key frame before switching and the time information of the time synchronization supplemental enhancement information of the second closest key frame is 1004 ms which is greater than a second preset time value, then a non-key frame between the first closest key frame and the second closest key frame is set as the key frame after switching.

[0109] For example, the first video includes time synchronization supplemental enhancement information of 1 second for the pre-switch key frame, the first closest key frame of the second video includes time synchronization supplemental enhancement information of 1005 ms, and the second closest key frame of the second video includes time synchronization supplemental enhancement information of 998 ms. In this case, the picture frame of the second video with the time synchronization supplemental enhancement information of 1005 ms is selected as the post-switch key frame. If the first closest key frame of the second video includes time synchronization supplemental enhancement information of 1010 ms, and the second closest key frame of the second video includes time synchronization supplemental enhancement information of 998 ms, the picture frame of the second video with the time synchronization supplemental enhancement information of 998 ms is selected as the post-switch key frame. If the first closest key frame of the second video includes time synchronization supplemental enhancement information of 1010 ms, and the second closest key frame of the second video includes time synchronization supplemental enhancement information of 995 ms, a non-key frame between the picture frame of the second video with the time synchronization supplemental enhancement information of 995 ms and the picture frame of the second video with the time synchronization supplemental enhancement information of 1010 ms is selected as the post-switch key frame.

[0110] S204: Switching of the multiple video streams is implemented based on the pre-switch key frame and the post-switch key frame.

[0111] Specifically, the pre-switch key frame in the pre-switch video stream and the post-switch key frame in the post-switch video stream are used to switch the pre-switch video stream and the post-switch video stream, so as to implement the switching of the multiple video streams.

[0112] In an optional embodiment, since the time information of the post-switch key frame is usually equal to or later than the time information of the pre-switch key frame, the multiple video streams of the first video and the multiple video streams of the second video are parsed, the last key frame before the switching of the multiple video streams of the first video is selected as the pre-switch key frame, the post-switch key frame in the multiple video streams of the second video within a set time interval is determined based on the time synchronization supplemental enhancement information of the pre-switch key frame, and finally the pre-switch key frame and the post-switch key frame are processed in a preset manner, so as to implement the switching of the multiple video streams.

[0113] It can be seen that the application is used for frame-level time alignment and synchronization of multiple video streams. The technical principle is to embed calibrated time synchronization supplemental enhancement information in video frame data. Subsequently, a video switching or playing device performs corresponding time alignment processing on the frame video time of multiple video streams or multiple machines, so as to achieve the purpose of synchronized playing and switching of multiple video contents.

[0114] Compared with the prior art, the application firstly acquires multiple video streams, determines key frames of the multiple video streams, respectively embeds preset time synchronization supplementary enhancement information into the key frames of the multiple video streams, in response to receiving a switching request of the multiple video streams sent by a terminal, acquires a pre-switching key frame and a post-switching key frame in the multiple video streams, and realizes switching of the multiple video streams based on the pre-switching key frame and the post-switching key frame. By obtaining the key frame for switching and embedding the calibrated time synchronization supplementary enhancement information, the application can solve the switching problem of the multiple video streams and improve user experience.

[0115] Another embodiment of the application provides a switching device for multiple video streams, which is applied to any one of the switching methods for multiple video streams. Figure 3 As shown in a structural schematic diagram of a switching device for multiple video streams, the device comprises:

[0116] An acquisition module 301 is configured to acquire multiple video streams and determine key frames of the multiple video streams, wherein the multiple video streams comprise key frames and non-key frames.

[0117] An embedding module 302 is configured to respectively embed preset time synchronization supplementary enhancement information into the key frames of the multiple video streams.

[0118] A response module 303 is configured to, in response to receiving a switching request of the multiple video streams sent by a terminal, acquire a pre-switching key frame and a post-switching key frame in the multiple video streams.

[0119] An implementation module 304 is configured to realize switching of the multiple video streams based on the pre-switching key frame and the post-switching key frame.

[0120] Specifically, the embedding module comprises:

[0121] An embedding unit is configured to respectively embed calibrated time synchronization supplementary enhancement information into the key frames of the multiple video streams in a preset manner, and the interval time of the time synchronization supplementary enhancement information of adjacent key frames of a same video stream is not greater than a preset time value.

[0122] Specifically, the embedding unit comprises:

[0123] An embedding subunit is configured to embed the time synchronization supplementary enhancement information into the key frames of the multiple video streams in a preset json text format, wherein the content of the preset json text format comprises the time synchronization supplementary enhancement information, index information, and time zone information.

[0124] Specifically, the response module comprises:

[0125] The first obtaining unit is configured to obtain a pre-switch video stream and a post-switch video stream based on the switching request.

[0126] The second obtaining unit is configured to obtain a pre-switch key frame of the pre-switch video stream.

[0127] The third obtaining unit is configured to obtain a post-switch key frame of the post-switch video stream based on time synchronization supplemental enhancement information of the pre-switch key frame, wherein the time synchronization supplemental enhancement information of the pre-switch key frame is similar to time synchronization supplemental enhancement information of the post-switch key frame.

[0128] Specifically, the third obtaining unit comprises:

[0129] The backward searching subunit is configured to search a first closest key frame in the post-switch video stream in a time sequence based on the time synchronization supplemental enhancement information of the pre-switch key frame, and set the first closest key frame as the post-switch key frame.

[0130] Specifically, after the backward searching subunit, the device further comprises:

[0131] The forward searching subunit is configured to search a second closest key frame in the post-switch video stream in a time sequence.

[0132] The first setting subunit is configured to set the second closest key frame as the post-switch key frame if a difference between time information of the time synchronization supplemental enhancement information of the first closest key frame and time information of the time synchronization supplemental enhancement information of the pre-switch key frame is greater than a first setting value, and a difference between the time information of the time synchronization supplemental enhancement information of the pre-switch key frame and time information of the time synchronization supplemental enhancement information of the second closest key frame is less than a second setting value.

[0133] Specifically, the device further comprises:

[0134] The second setting subunit is configured to set a non-key frame between the first closest key frame and the second closest key frame as the post-switch key frame if the difference between the time information of the time synchronization supplemental enhancement information of the first closest key frame and the time information of the time synchronization supplemental enhancement information of the pre-switch key frame is greater than the first setting value, and the difference between the time information of the time synchronization supplemental enhancement information of the pre-switch key frame and the time information of the time synchronization supplemental enhancement information of the second closest key frame is greater than the second setting value.

[0135] Specifically, the implementation module comprises:

[0136] The implementation unit is configured to utilize the pre-switch key frame in the pre-switch video stream and the post-switch key frame in the post-switch video stream to switch the pre-switch video stream and the post-switch video stream, so as to realize the switching of the multi-path video stream.

[0137] Compared with the prior art, the application first acquires the multi-path video stream, determines the key frame of the multi-path video stream, embeds the preset time synchronization supplementary enhancement information into the key frame of the multi-path video stream respectively, acquires the pre-switch key frame and the post-switch key frame in the multi-path video stream in response to receiving the switching request of the multi-path video stream sent by the terminal, and realizes the switching of the multi-path video stream based on the pre-switch key frame and the post-switch key frame. By obtaining the key frame for switching and embedding the calibrated time synchronization supplementary enhancement information, the application can solve the switching problem of the multi-path video stream and improve the user experience.

[0138] The application further provides a storage medium in which a computer program is stored, and the computer program is configured to execute the steps in any of the method embodiments when running.

[0139] Specifically, in the embodiment, the storage medium can be configured to store the computer program for executing the following steps:

[0140] S201: Acquire a multi-path video stream, and determine a key frame of the multi-path video stream, wherein the multi-path video stream comprises the key frame and a non-key frame;

[0141] S202: Embed preset time synchronization supplementary enhancement information into the key frame of the multi-path video stream respectively;

[0142] S203: In response to receiving a switching request of the multi-path video stream sent by a terminal, acquire a pre-switch key frame and a post-switch key frame in the multi-path video stream;

[0143] S204: Realize the switching of the multi-path video stream based on the pre-switch key frame and the post-switch key frame.

[0144] Specifically, in the embodiment, the storage medium can include but is not limited to a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various storage media that can store computer programs.

[0145] Compared with the prior art, the application firstly acquires multiple video streams, determines key frames of the multiple video streams, respectively embeds preset time synchronization supplementary enhancement information into the key frames of the multiple video streams, acquires a key frame before switching and a key frame after switching in the multiple video streams in response to receiving a switching request of the multiple video streams sent by a terminal, and realizes switching of the multiple video streams based on the key frame before switching and the key frame after switching. Through obtaining the key frames for switching and embedding the calibrated time synchronization supplementary enhancement information, the application can solve the switching problem of the multiple video streams and improve user experience.

[0146] The embodiment of the application further provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the method embodiments.

[0147] Specifically, the electronic device can further include a transmission device and an input / output device, wherein the transmission device is connected with the processor, and the input / output device is connected with the processor.

[0148] Specifically, in the embodiment, the processor can be configured to execute the following steps through the computer program:

[0149] S201: acquiring multiple video streams and determining key frames of the multiple video streams, wherein the multiple video streams include key frames and non-key frames;

[0150] S202: respectively embedding preset time synchronization supplementary enhancement information into the key frames of the multiple video streams;

[0151] S203: acquiring a key frame before switching and a key frame after switching in the multiple video streams in response to receiving a switching request of the multiple video streams sent by a terminal;

[0152] S204: realizing switching of the multiple video streams based on the key frame before switching and the key frame after switching.

[0153] Compared with the prior art, the application firstly acquires multiple video streams, determines key frames of the multiple video streams, respectively embeds preset time synchronization supplementary enhancement information into the key frames of the multiple video streams, acquires a key frame before switching and a key frame after switching in the multiple video streams in response to receiving a switching request of the multiple video streams sent by a terminal, and realizes switching of the multiple video streams based on the key frame before switching and the key frame after switching. Through obtaining the key frames for switching and embedding the calibrated time synchronization supplementary enhancement information, the application can solve the switching problem of the multiple video streams and improve user experience.

[0154] It should be noted that, for the foregoing method embodiments, the sequences of the described actions are not necessarily required to achieve the objects of the application, and certain steps can be performed in other sequences or even concurrently. Additionally, the described embodiments are merely provided as examples, and not all of the actions described are necessarily required to achieve desirable structures or results.

[0155] In the above embodiments, the description of each embodiment is focused on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0156] In several embodiments provided by the present application, it should be understood that the disclosed apparatus can be implemented in other manners. For example, the embodiments of the apparatus described above are merely schematic; for example, the division of the units is only a logical function division; there can be another division manner for the actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different parts can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrically, mechanical or in other forms.

[0157] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0158] In addition, each functional unit in the embodiments of the present application can be integrated in a processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0159] The integrated unit described above, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the above-mentioned method of each embodiment of the present application. The aforementioned memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0160] The above describes the embodiments of the present application in detail, and the principles and implementation manners of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description of the present application should not be understood as a limitation of the present application.

Claims

1. A method for switching of multiple video streams, characterized by, The method comprises: acquiring a plurality of video streams and determining key frames of the plurality of video streams, the plurality of video streams comprising key frames and non-key frames; embedding preset time synchronization supplemental enhancement information into the key frames of the plurality of video streams respectively; in response to receiving a switching request of the plurality of video streams sent by a terminal, acquiring a pre-switching key frame and a post-switching key frame in the plurality of video streams; implementing switching of the plurality of video streams based on the pre-switching key frame and the post-switching key frame; wherein the acquiring of the pre-switching key frame and the post-switching key frame in the plurality of video streams in response to receiving the switching request of the plurality of video streams sent by the terminal comprises: acquiring a pre-switching video stream and a post-switching video stream based on the switching request; acquiring a pre-switching key frame of the pre-switching video stream; acquiring a post-switching key frame of the post-switching video stream based on time synchronization supplemental enhancement information of the pre-switching key frame, wherein the time synchronization supplemental enhancement information of the pre-switching key frame and time synchronization supplemental enhancement information of the post-switching key frame are similar; the acquiring of the post-switching key frame of the post-switching video stream based on the time synchronization supplemental enhancement information of the pre-switching key frame comprises: based on the time synchronization supplemental enhancement information of the pre-switching key frame, searching for a first closest key frame in the post-switching video stream in a time sequence and setting the first closest key frame as the post-switching key frame; searching for a second closest key frame in the post-switching video stream in a time sequence; if a difference between time information of the time synchronization supplemental enhancement information of the first closest key frame and time information of the time synchronization supplemental enhancement information of the pre-switching key frame is greater than a first set value and a difference between the time information of the time synchronization supplemental enhancement information of the pre-switching key frame and time information of the time synchronization supplemental enhancement information of the second closest key frame is less than a second set value, setting the second closest key frame as the post-switching key frame, wherein the second set value is less than the first set value; if the difference between the time information of the time synchronization supplemental enhancement information of the first closest key frame and the time information of the time synchronization supplemental enhancement information of the pre-switching key frame is greater than the first set value and the difference between the time information of the time synchronization supplemental enhancement information of the pre-switching key frame and the time information of the time synchronization supplemental enhancement information of the second closest key frame is greater than the second set value, setting a non-key frame between the first closest key frame and the second closest key frame as the post-switching key frame to improve the fluency of video switching.

2. The method of claim 1, wherein, the embedding of the preset time synchronization supplemental enhancement information into the key frames of the plurality of video streams respectively comprises: embedding calibrated time synchronization supplemental enhancement information into the key frames of the plurality of video streams in a preset manner, and an interval time of the time synchronization supplemental enhancement information of adjacent key frames of a same video stream is not greater than a preset time value.

3. The method of claim 2, wherein, the embedding of the calibrated time synchronization supplemental enhancement information into the key frames of the plurality of video streams in the preset manner comprises: The time synchronization supplementary enhancement information is embedded into the key frame of the multi-path video stream in a preset json text format, wherein the content of the preset json text format includes the time synchronization supplementary enhancement information, index information and time zone information.

4. The method according to any one of claims 1 to 3, characterized in that, The switching of the multi-path video stream is realized based on the pre-switching key frame and the post-switching key frame. The switching of the pre-switching video stream and the post-switching video stream is realized by using the pre-switching key frame in the pre-switching video stream and the post-switching key frame in the post-switching video stream, so as to realize the switching of the multi-path video stream.

5. A switching device for multiple video streams, characterized in that The device comprises: The acquisition module is configured to acquire a multi-path video stream and determine a key frame of the multi-path video stream, wherein the multi-path video stream comprises a key frame and a non-key frame. The embedding module is configured to embed preset time synchronization supplementary enhancement information into the key frame of the multi-path video stream. The response module is configured to acquire a pre-switching key frame and a post-switching key frame in the multi-path video stream in response to receiving a switching request of the multi-path video stream sent by a terminal. The realization module is configured to realize the switching of the multi-path video stream based on the pre-switching key frame and the post-switching key frame. The response to the switching request of the multi-path video stream sent by the terminal comprises: Based on the switching request, a pre-switching video stream and a post-switching video stream are acquired. The pre-switching key frame of the pre-switching video stream is acquired. The post-switching key frame of the post-switching video stream is acquired based on the time synchronization supplementary enhancement information of the pre-switching key frame, wherein the time synchronization supplementary enhancement information of the pre-switching key frame is similar to the time synchronization supplementary enhancement information of the post-switching key frame. The post-switching key frame of the post-switching video stream is acquired based on the time synchronization supplementary enhancement information of the pre-switching key frame, comprising: The first closest key frame in the post-switching video stream is found in a time sequence backward based on the time synchronization supplementary enhancement information of the pre-switching key frame, and the first closest key frame is set as the post-switching key frame. The second closest key frame in the post-switching video stream is found in a time sequence forward. If the difference between the time information of the time synchronization supplementary enhancement information of the first closest key frame and the time information of the time synchronization supplementary enhancement information of the pre-switching key frame is greater than a first set value, and the difference between the time information of the time synchronization supplementary enhancement information of the pre-switching key frame and the time information of the time synchronization supplementary enhancement information of the second closest key frame is less than a second set value, the second closest key frame is set as the post-switching key frame, wherein the second set value is less than the first set value. If a difference between time information of the time-synchronous supplemental enhancement information of the first closest key frame and time information of the time-synchronous supplemental enhancement information of the key frame before switching is greater than a first set value, and a difference between time information of the time-synchronous supplemental enhancement information of the key frame before switching and time information of the time-synchronous supplemental enhancement information of the second closest key frame is greater than a second set value, non-key frames between the first closest key frame and the second closest key frame are set as key frames after switching, so as to improve the fluency of video switching.

6. A storage medium, characterized by The storage medium has a computer program stored therein, wherein the computer program is configured to implement the method in any one of claims 1 to 4 when executed.

Images