A video processing method, apparatus, electronic device, and storage medium

Abstract

Embodiments of the present disclosure provide a video processing method and device, electronic equipment and storage medium. The method comprises: obtaining stream switching indication information, the stream switching indication information comprising a target parameter; in the case that a first switching condition is met, transmitting a video frame in a first video stream corresponding to the target parameter; in the case that a second switching condition is met, transmitting a video frame in a second video stream corresponding to the target parameter. With the method, when stream switching is performed, a video frame is selected from the first video stream and the second video stream for transmission, frame-level stream switching is achieved, the amount of data transmitted is reduced, and the fluency of video stream switching is improved.

Description

Technical Field

[0001] This disclosure relates to the field of video processing technology, and more particularly to a video processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid popularization of video services, various types of videos have emerged, such as panoramic videos, free-viewpoint videos, and multi-resolution videos.

[0003] To meet viewing requirements, the client needs to acquire the necessary video stream for display. This acquisition process causes video stream switching. Currently, switching video streams requires downloading the entire video. Therefore, the amount of data the client needs to download is large, resulting in poor smoothness during video stream switching. Summary of the Invention

[0004] This disclosure provides a video processing method, apparatus, electronic device, and storage medium to achieve frame-level streaming, reduce the amount of data transmitted, and improve the smoothness of video stream switching.

[0005] In a first aspect, embodiments of this disclosure provide a video processing method, the method comprising:

[0006] Obtain flow cut indication information, which includes target parameters;

[0007] If the first switching condition is met, the video frames in the first video stream corresponding to the target parameters are transmitted.

[0008] If the second switching condition is met, transmit the video frames in the second video stream corresponding to the target parameters;

[0009] The first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream.

[0010] Secondly, embodiments of this disclosure also provide a video processing method, the method comprising:

[0011] Transmit flow-cutting indication information, which includes target parameters;

[0012] Receive video frames, the video frames including video frames in the first video stream corresponding to the target parameter received when the first switching condition is met, and video frames in the second video stream corresponding to the target parameter received when the second switching condition is met.

[0013] The first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream.

[0014] Thirdly, embodiments of this disclosure also provide a video processing apparatus, the apparatus comprising:

[0015] The acquisition module is used to acquire flow cut indication information, which includes target parameters;

[0016] The first transmission module is used to transmit video frames in the first video stream corresponding to the target parameters when the first switching condition is met.

[0017] The second transmission module is used to transmit video frames in the second video stream corresponding to the target parameters when the second switching condition is met.

[0018] The first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream.

[0019] Fourthly, embodiments of this disclosure also provide a video processing apparatus, the apparatus comprising:

[0020] The transmission module is used to transmit flow-cutting indication information, which includes target parameters;

[0021] A receiving module is used to receive video frames, the video frames including video frames in a first video stream corresponding to the target parameter received when a first switching condition is met, and video frames in a second video stream corresponding to the target parameter received when a second switching condition is met.

[0022] The first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream.

[0023] Fifthly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0024] One or more processors;

[0025] Storage device for storing one or more programs.

[0026] When the one or more programs are executed by the one or more processors, the one or more processors implement the video processing method provided in the embodiments of this disclosure.

[0027] In a sixth aspect, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the video processing method provided in embodiments of this disclosure.

[0028] In this embodiment, when a first switching condition is met, video frames from the first video stream corresponding to the target parameter are transmitted; when a second switching condition is met, video frames from the second video stream corresponding to the target parameter are transmitted; the target parameter is included in the stream switching indication information. Transmitting the selected video frame after selection solves the problem of large interactive data volume caused by transmitting the entire video during video stream switching, achieving frame-level stream switching, reducing the amount of transmitted data, and improving the smoothness of video stream switching. Attached Figure Description

[0029] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0030] Figure 1a A schematic diagram of a VR video perspective provided by existing technology;

[0031] Figure 1b A schematic diagram of the perspective of a free-viewpoint video provided by existing technology;

[0032] Figure 1c This is a schematic flowchart of a video processing method provided in an embodiment of the present disclosure;

[0033] Figure 2 This is a schematic flowchart of another video processing method provided in the embodiments of this disclosure;

[0034] Figure 3 This is a schematic flowchart of another video processing method provided in an embodiment of this disclosure;

[0035] Figure 4a This is a schematic flowchart of another video processing method provided in the embodiments of this disclosure;

[0036] Figure 4b This is a schematic diagram of a flow cutting provided in an embodiment of this disclosure;

[0037] Figure 4c This is yet another schematic diagram of flow cutting provided in the embodiments of this disclosure;

[0038] Figure 4d yes Figure 4c A schematic diagram of video frame encoding in a streaming scenario;

[0039] Figure 4e Another schematic diagram of flow cutting provided in this disclosure embodiment;

[0040] Figure 4f yes Figure 4e A schematic diagram of video frame encoding in a streaming scenario;

[0041] Figure 5 This is a schematic diagram of the structure of a video processing apparatus provided in an embodiment of this disclosure;

[0042] Figure 6 This is a schematic diagram of the structure of another video processing apparatus provided in the embodiments of this disclosure;

[0043] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0044] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0045] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0046] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0047] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0048] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0049] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0050] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0051] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0052] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0053] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0054] It is understood that the data involved in this technical solution (including but not limited to the data itself, its acquisition, or its use) shall comply with the requirements of relevant laws, regulations, and provisions. Data may include information, parameters, and messages, such as flow control instructions.

[0055] This disclosure is applicable to scenarios where video streams under any target parameter are switched, such as switching from a video stream under one target parameter to a video stream under another target parameter. Different scenarios correspond to different target parameters, and the target parameter is not limited here. The target parameter can be a parameter of the video. The following explanation uses the target viewpoint as the target parameter. The target viewpoint can be considered as the viewpoint to which the switch is to be made.

[0056] Multi-view interactive video applications include 360° Virtual Reality (VR) panoramic videos and free-viewpoint videos. Generally speaking, the viewpoint refers to the area of ​​the screen covered by the user's field of vision, but the specific definition varies slightly from application to application.

[0057] Figure 1a A schematic diagram of a VR video perspective provided for existing technology. Figure 1b A schematic diagram of the perspective of a free-viewpoint video provided for existing technology, see [link / reference]. Figure 1a and Figure 1b 360VR panoramic video has a perspective direction from the inside out, and all the video views can be stitched together to form a complete picture; free-view video has a perspective direction from the outside in, and the video views cannot be stitched together.

[0058] Multi-view video applications need to switch the video view displayed on the client in real time according to changes in the user's perspective. Transmitting such video applications places high demands on streaming technology. For example, VR scenarios require motion-to-photonlatency to be no more than 20ms. At the same time, video stuttering or degraded image quality will also have a significant impact on the multi-view user experience.

[0059] One approach to stream switching in related technologies is to download the entire video to the local machine before switching. This means the client first downloads the entire video locally, and then displays the portion of the video visible from the user's viewing angle. Typically, a 4K panoramic 360 video at 30 frames per second requires approximately 15Mbps after encoding compression, while an 8K panoramic 360 video requires 60Mbps.

[0060] Taking a free-view video of a solo ballet dancer (1080p) as an example, the IP stream has an intrapicture (I-frame) size of 60-80KB and a predictive-frame (P-frame) size of 5-12KB, with a bitrate estimated to be in the range of 2.6Mbps (gop=30). The I stream has an I-frame size of 60-65KB.

[0061] Taking a free-view video of a multi-person selfie (1080p) as an example, the IP stream has an I-frame size of 130-160KB and a P-frame size of 20-40KB. The I stream has an I-frame size of 65-75KB and the I2 stream has an I-frame size of 100KB.

[0062] A 1080p, 30 frames per second, free-view video containing 48 cameras would require a bitrate of approximately 100-400 Mbps after encoding. This demonstrates that this approach incurs very high user-side transmission bandwidth overhead, resulting in significant data waste. It also places high demands on user-side bandwidth, hindering its widespread adoption. The user side can be considered the client-side interface for viewing the video.

[0063] Another approach to stream switching in related technologies is based on video tile transmission, a technique commonly found in 360VR panoramic video applications. The server first re-encodes the original video into multiple video tiles. The client only needs to download the video tile covered by the user's current viewpoint, rather than the entire video. Typically, a video tile contains 60-120 frames (4 seconds in length, 15-30 fps). When the user's viewpoint changes, the client needs to download the entire tile / GoP locally to decode the current frame, placing significant bandwidth pressure and wasting data. Therefore, this method usually uses pre-downloading of video tiles to ensure smooth streaming for the user, but this exacerbates the data waste problem.

[0064] Another approach to stream switching is using a multi-view encoder, a technique commonly found in free-view applications. After video capture, it is encoded using a dedicated multi-view encoder. Multi-view encoding fully utilizes temporal and spatial redundancy, improving encoding efficiency. Although this approach results in a lower bitrate compared to single-view encoding, the client still needs to download the entire video locally for streaming. Furthermore, this approach introduces significant decoding complexity to the client, noticeably impacting decoding latency and power consumption.

[0065] To address the aforementioned technical problems, this disclosure provides a video processing method. Figure 1c This is a flowchart illustrating a video processing method provided in an embodiment of this disclosure. This embodiment is applicable to situations where electronic devices process video in streaming scenarios. The method can be executed by a video processing device, which can be implemented in software and / or hardware. Optionally, it can be implemented by an electronic device, such as a mobile terminal, PC, or server that holds the video. The video in this embodiment can be from a live streaming scenario or a video-on-demand scenario.

[0066] like Figure 1c As shown, the method includes:

[0067] S110. Obtain flow cutting indication information, wherein the flow cutting indication information includes target parameters.

[0068] Stream switching indication information can be information transmitted from an electronic device browsing video to this electronic device. Stream switching indication information can be used to indicate information related to stream switching, such as information required to implement stream switching. Stream switching can refer to the switching of video streams under different target parameters. Stream switching indication information may include target parameters, which can be a specific parameter of the video stream. Users can browse videos with different target parameters. Videos with different target parameters can be encoded separately into a single video stream, and switching between videos with different target parameters can be done by switching between the video streams corresponding to the different target parameters. The content included in the target parameters is not limited and can be determined based on the actual scenario. It is applicable to any scenario that requires real-time switching between video streams with different target parameters.

[0069] In one embodiment, the target parameters include the target viewing angle and / or the target resolution.

[0070] The target resolution can be considered as the resolution to which you will switch.

[0071] This disclosure can encode videos from different perspectives to obtain corresponding video streams; this disclosure can also encode videos at different resolutions to obtain corresponding video streams.

[0072] When the target parameter is the target viewpoint, if the user's viewpoint changes, the corresponding image will be displayed based on the changed viewpoint (i.e., the target viewpoint). The displayed image can be determined based on the video stream corresponding to the changed viewpoint.

[0073] When the target parameter is the target resolution, if the network speed of the user's electronic device changes, the corresponding resolution (the target resolution) can be selected to display the corresponding image based on the changed network speed. The displayed image can be determined based on the video stream corresponding to the changed resolution (the target resolution). For example, the first resolution corresponds to two video streams, and the second resolution corresponds to two video streams. Network speed and resolution can be directly proportional. The first resolution and the second resolution are different resolutions. Each resolution corresponds to two video streams, which include the first video stream and the second video stream.

[0074] Each target parameter can correspond to two video streams: one video stream (i.e., the second video stream, also known as the normal playback video) is used for normal user viewing, and the other video stream (i.e., the first video stream, also known as the stream-cutting auxiliary video) is used when the target parameter changes. The two video streams are encoded using different encoding parameters.

[0075] This step can obtain stream switching instruction information from the electronic device browsing the video. When the user uses the electronic device to browse the video, if the target parameters change, it can trigger the sending of stream switching instruction information to instruct the electronic device to select and transmit the video frame corresponding to the stream switching instruction information based on the stream switching instruction information, so that the user can browse the video frame under the changed target parameters.

[0076] If the stream switching indication information is obtained in this step, it can be assumed that the target parameters of the viewed video have changed. This electronic device needs to select video frames based on the stream switching indication information and then transmit the selected video frames to the electronic device viewing the video.

[0077] S120. If the first switching condition is met, transmit the video frame in the first video stream corresponding to the target parameter.

[0078] Different target parameters can correspond to different video streams. When the target parameters change, it is necessary to transmit the video frames in the video stream with the corresponding target parameters.

[0079] The first switching condition can be understood as the condition for switching the video stream corresponding to the target parameter when the target parameter changes. For example, if the current parameter of the electronic device is target parameter A, and the target parameter included in the stream switching indication information is target parameter B, the first switching condition can be considered as the condition for switching from the video stream corresponding to target parameter A to the video stream corresponding to target parameter B. Target parameter A corresponds to a first video stream and a second video stream under target parameter A, and target parameter B corresponds to a first video stream and a second video stream under target parameter B. The current parameter (also known as the previous target parameter) is the target parameter included in the last time the stream switching indication information was obtained.

[0080] This embodiment does not limit the first switching condition. The first switching condition can be determined based on the content included in the stream switching indication information and the first video stream of the target parameters.

[0081] The first and second video streams are video streams encoded with different encoding parameters under the target parameters. The Group of Pictures (GOP) length of the first video stream is shorter than that of the second video stream. The second video stream can be encoded normally according to the actual situation, while the first video stream can be encoded into a video stream with a GOP length shorter than that of the second video stream. The first and second video streams may not include bidirectional prediction frames (B-frames). If B-frames are included, they can be converted into P-frames.

[0082] The first video stream is used when the target parameters change, while the second video stream is used when the video plays normally. That is, the second video stream is used when the target parameters do not change. Since the group length of the first video stream is smaller than that of the second video stream, the stream switching can be completed based on the smaller group length when the target parameters change. Because the GOP length is small, the amount of data to be transmitted will also be reduced synchronously after selecting video frames from the first video stream with a small GOP length, thus realizing frame-level stream switching.

[0083] Because the GOP length of the first video stream is small, the number of video frames selected from the first video stream corresponding to the target parameters can be relatively small. In this step, the selection of video frames from the first video stream can be based on the stream switching indication information.

[0084] In one embodiment, the video frames selected from the first video stream in this step can be multiple frames, including P-frames and / or I-frames. The time at which the video frames are selected from the first video stream can be the switching time indicated by the switching indication information, or it can be later than the switching time indicated by the switching indication information. The switching time can be the time indicating the switching target parameter. This electronic device can determine whether the first switching condition is met based on the switching time. If the current time is the switching time or the current time is later than the switching time, the first switching condition can be considered met. The current time can be the time of the electronic device at the current moment, such as the time when determining whether it is equal to the switching time. The switching time and the current time can be absolute timestamps.

[0085] The first switching condition can also be determined based on the video frame corresponding to the switching time of the first video stream. For example, if the video frame corresponding to the switching time is an I-frame and the switching time has been reached, the first switching condition can be considered to be met; otherwise, the first switching condition can be considered not yet met.

[0086] When the video frame selected from the first video frame at the cut-off time is an I-frame, the selected video frame may also include a P-frame following the I-frame. When the video frame selected from the first video frame at the cut-off time is a P-frame, the selected video frame can be the P-frame at the cut-off time, the I-frame closest to the P-frame before the P-frame, and the video frame between the I-frame and the P-frame. When the video frame selected from the first video frame at the cut-off time is a P-frame, the selected video frame can be the P-frame at the cut-off time, and the I-frame closest to the P-frame before the P-frame.

[0087] In one embodiment, the flow switching indication information includes one or more of the following: target parameters, flow switching time, and parameter switching speed.

[0088] Parameter switching speed can be understood as the switching speed of target parameters, also known as the switching speed between videos with different target parameters. Different parameter switching speeds can correspond to different video frame selection methods. When the parameter switching speed is less than a set threshold, it can be considered a scenario where target parameter switching is infrequent, such as a VR video scenario; otherwise, it can be considered a scenario where target parameter switching is frequent, such as a free-viewpoint video scenario. The value of the set threshold is not limited here and can be determined according to the actual situation.

[0089] This disclosure does not limit how the parameter switching speed is determined. The parameter switching speed can be determined by the electronic device holding the video or by the electronic device browsing the video.

[0090] In one embodiment, the parameter switching speed can be determined based on the number of parameter switching times within a preset time period.

[0091] Taking the target parameter as an example, when determining the parameter switching speed, it can be based on information such as the user's head movement speed, head movement angle, and the angles corresponding to different viewpoints within a preset time. The number of parameter switching times can be determined based on the head movement angle and the angles corresponding to different viewpoints.

[0092] The number of P-frames selected from the first video stream can be determined based on the number of reference frames. The number of reference frames can be considered as the number of frames required for P-frame encoding and decoding. When the number of reference frames is 1, selecting one P-frame from the first video stream allows switching to the second video stream with the target parameters to select video frames.

[0093] S130. If the second switching condition is met, transmit the video frame in the second video stream corresponding to the target parameter.

[0094] The second switching condition can be a condition for switching between different video streams under a target parameter. Before the second switching condition is met, video frames can be continuously selected from the first video stream. The second switching condition can be determined based on the first and / or second video streams, such as by the types of video frames included in the first and / or second video streams.

[0095] This step does not impose any restrictions on the second switching condition, as long as it ensures that the selected video frame can be decoded by the electronic device viewing the video.

[0096] The second switching condition includes one of the following:

[0097] Forward predictive coded frames appear in the video frames of the first video stream being transmitted;

[0098] After the cut-off time, an intra-frame image appears in the second video stream.

[0099] In one embodiment, the second switching condition may be that the video frames selected from the first video stream include forward predictive coded frames, i.e., P-frames. The inclusion of P-frames in the selected video frames from the first video stream enables the first frame subsequently selected from the second video stream to be decoded based on the P-frames selected from the first video stream, such as decoding based on the motion vectors of the P-frames selected from the first video stream. The first and second video streams may be encoded using the same encoder, but the encoding parameters of the first and second encoders differ, such as the GOP length in the encoding parameters.

[0100] In one embodiment, the second switching condition can be that the second video stream first produces an I-frame after the stream switching time. Since the second video stream produces an intra-frame image after the stream switching time, it can be directly decoded, thus satisfying the second switching condition.

[0101] In one embodiment, the second switching condition can be that an I-frame appears in the second video stream after the current time.

[0102] In this step, if the second switching condition is met, the video frame selected from the second video stream can be either an I-frame or a P-frame. After the second switching condition is met, video frames can be continuously selected from the second video stream until a stream switching indication is obtained again. If the second switching condition is not met, video frames can be continuously selected from the first video stream until the second switching condition is met.

[0103] In one embodiment, the first video stream and the second video stream do not include bidirectional prediction frames, i.e., B-frames, and the minimum keyframe interval of the second video stream is equal to the frame group length of the second video stream.

[0104] The minimum keyframe interval can be considered as the minimum interval between keyframes. The frame group length can be used to set the upper limit of the actual number of frames within a frame group, while the minimum keyframe interval can set the lower limit. Setting the minimum keyframe interval of the second video stream to be equal to the frame group length ensures that the encoder does not add extra I-frames to the second video stream, guaranteeing that the actual number of frames in the frame group equals the set value. This allows the encoder to directly select video frames from the second video stream after the first video stream has transmitted one P-frame, without receiving a next stream cutoff instruction. Setting the number of reference frames in the second video stream to be less than or equal to the set number of frames allows video frames to be decoded with reference to the set number of frames. The set number of frames can be 1 frame; no specific value is specified here.

[0105] The first video stream can have a GOP length of 2, which allows the second video stream to be set with I-frames and P-frames at intervals. This results in fewer video frames being selected from the first video stream, thus enabling a quicker switch to the second video stream to select video frames from it.

[0106] In one embodiment, assuming no next-stream indication information is received, after transmitting a video frame in the first video stream corresponding to the target parameter, if the transmitted video frame is a P-frame, a video frame can be directly selected from the second video stream and transmitted. This embodiment is predicated on the minimum keyframe interval being equal to the frame group length of the second video stream. When the minimum keyframe interval is not equal to the frame group length of the second video stream, there may be a situation where the selected video frame is the first P-frame after an I-frame. If the first P-frame is selected, decoding cannot be performed using the P-frames transmitted in the first video stream. By setting the minimum keyframe interval to be equal to the frame group length of the second video stream, it is ensured that no redundant I-frames are introduced into the second video stream, and the situation where the first P-frame of the second video stream is selected when switching from the first video stream to the second video stream is avoided.

[0107] In one embodiment, assuming no next-stream indication information is received, if an I-frame appears in the second video stream after a video frame in the first video stream corresponding to the transmission target parameter, then a video frame can be selected from the second video stream.

[0108] In one embodiment, assuming that no next stream indication information is received, after a video frame in the first video stream corresponding to the transmission target parameter appears, the second video stream will have the first P frame after the I frame. The first P frame and the I frame before the P frame can be transmitted.

[0109] After selecting video frames, they can be transmitted. There is no restriction on the timing of transmission; they can be transmitted directly after selection or under certain conditions, such as after selecting a certain number of video frames. The specific transmission method is not limited here.

[0110] The technical solution of this disclosure, by transmitting the selected video frame after selection, solves the problem of large interactive data volume caused by transmitting the entire video during video stream switching. It achieves frame-level stream switching, reduces the amount of transmitted data, and improves the smoothness of video stream switching. When the target parameter is the target viewpoint, real-time switching of multiple viewpoints at the video frame level can be achieved. When the target parameter is the target resolution, real-time switching of multiple resolutions at the video frame level can be achieved.

[0111] In one embodiment, the flow switching indication information includes a flow switching time, which indicates the time to switch to the target parameter.

[0112] The strategy for selecting video frames can be determined by the cut-off time and the current time.

[0113] In one embodiment, the first switching condition includes: the current time is the switching time or the current time is later than the switching time.

[0114] In a free-view video scene, if the current time is the cutoff time or the current time is later than the cutoff time, the first switching condition can be considered to be met, and then a video frame can be selected from the first video stream.

[0115] The current time can be considered the time at which the relationship between the current time and the cutoff time is determined. The cutoff time can be used to indicate whether the cutoff is immediate or delayed. The cutoff time can be an absolute time, such as a timestamp. In delayed cutoff, the cutoff can wait until the cutoff time arrives before proceeding.

[0116] When the cutoff time arrives or the cutoff time indicates immediate cutoff, the selection of video frames can be determined based on the frame type of the first video stream at the cutoff time.

[0117] The first video stream corresponding to the target parameter is selected as frame I in the video frame at the cut-off time.

[0118] If the video frame of the first video stream corresponding to the target parameter is not an I-frame at the cut-off time, the video frame of the first video stream at the cut-off time and the video frame before the cut-off time can be selected and transmitted until the I-frame before the video frame at the cut-off time is selected.

[0119] If the video frame of the first video stream corresponding to the target parameter is not an I-frame at the time of the stream cut, a video frame can be selected from the video stream of the previous target parameter until an I-frame appears in the first video stream corresponding to the target parameter.

[0120] In one embodiment, the first switching condition further includes: the video frame of the first video stream at the switching time is an intra-frame image.

[0121] In a VR video scenario, if the current time is the switching time or the current time is later than the switching time, and the video frame of the first video stream at the switching time is an I-frame, it can be considered that the first switching condition is met, and then a video frame is selected from the first video.

[0122] In one embodiment, before transmitting the video frames in the first video stream corresponding to the target parameters, the method further includes:

[0123] Select video frames from the video stream of the previous target parameter, wherein the video stream of the previous target parameter is either the first video stream of the previous target parameter or the second video stream of the previous target parameter, and the previous target parameter is the target parameter included in the previously obtained stream cutting indication information.

[0124] In this embodiment, when the first switching condition is not met, a video frame is selected from the video stream of the current parameter, i.e., the previous target parameter, such as from the first video stream or the second video stream corresponding to the current parameter. That is, when switching the target parameter, it is possible to switch from the first video stream of the previous target parameter to the video stream corresponding to the target parameter, or from the second video stream of the current parameter to the video stream corresponding to the target parameter.

[0125] In one embodiment, if the second switching condition of the previous target parameter is not met, and if a stream switching indication is obtained, and the first switching condition of the target parameter included in the stream switching indication is met, the stream can be switched from the first video stream of the previous target parameter to the first video stream of the target parameter.

[0126] In one embodiment, when selecting a video frame from the second video stream of the previous target parameter, if a stream switching instruction is received and the first switching condition of the target parameter included in the stream switching instruction is met, the video stream can be switched from the second video stream of the previous target parameter to the first video stream of the target parameter.

[0127] Figure 2 This is a flowchart illustrating another video processing method provided in this disclosure embodiment. This embodiment adds an operation where the first switching condition is not met, such as... Figure 2 As shown, the method includes:

[0128] S210. Select video frames from the video stream of the previous target parameter.

[0129] When the parameter switching speed is less than the set threshold, the first switching condition includes the current time being less than or later than the stream switching time, and the appearance of an I-frame in the first video stream.

[0130] When the parameter switching speed is greater than or equal to the set threshold, the first switching condition includes the current time being less than or later than the flow switching time.

[0131] The execution order of this step and S220 is not limited.

[0132] In one embodiment, after acquiring the flow-cutting indication information, if the first switching condition is not met, a video frame can be selected from the video stream of the previous target parameter. The video stream of the previous target parameter is either the first video stream or the second video stream of the previous target parameter. The previous target parameter is the target parameter included in the previously acquired flow-cutting indication information.

[0133] In one embodiment, after selecting a video frame from the video stream of the previous target data, the stream switching indication information can be obtained.

[0134] S220, Obtain flow cut indication information.

[0135] After obtaining the stream switching indication information, it can be determined whether the first switching condition is met. If not, a video stream can be selected from the video stream of the previous target parameter, or a video frame can be selected from the first video stream corresponding to the target parameter and transmitted.

[0136] S230. If the first switching condition is met, transmit the video frame in the first video stream corresponding to the target parameter.

[0137] This embodiment can determine the corresponding first video stream based on the target parameters included in the stream cutting indication information, and determine when to select video frames from the first video stream based on the stream cutting time and parameter switching speed included in the stream cutting indication information.

[0138] In one embodiment, when the parameter switching speed is less than a set threshold, i.e., the scenario where the target parameter switching is infrequent is determined based on the parameter switching speed, and the first switching condition is currently met, this embodiment can select video frames from the first video stream. Before transmitting the video frames in the first video stream corresponding to the target parameter, video frames can be selected from the video stream of the previous target parameter until the first switching condition is met, at which point video frames are selected from the first video stream.

[0139] If the video frame corresponding to the switching time in the first video stream is an I-frame, after selecting the I-frame corresponding to the switching time from the first video stream, it can be determined whether the second switching condition is met. If not, continue selecting video frames from the first video stream; otherwise, select video frames from the second video stream. If the next switching indication information is received, execute the operations after obtaining the switching indication information.

[0140] S240. If the second switching condition is met, transmit the video frame in the second video stream corresponding to the target parameter. The technical solution provided by this embodiment can effectively select video frames based on the stream switching indication information and the first video stream, realizing frame-level stream switching and improving the efficiency of video frame selection.

[0141] In one embodiment, transmitting video frames in the first video stream corresponding to the target parameter when the first switching condition is met includes:

[0142] When the parameter switching speed is less than a set threshold, the video frames in the video stream of the previous target parameter are transmitted until the first video stream shows an in-frame image and the in-frame image of the first video stream is transmitted.

[0143] When the first switching condition is met, transmitting video frames from the first video stream corresponding to the target parameters includes:

[0144] When the parameter switching speed is less than a set threshold, the video frame of the video stream of the previous target parameter is transmitted until the current time is the stream switching time and the first video stream shows an in-frame image and the in-frame image of the first video stream is transmitted; or until the current time exceeds the stream switching time and the first video stream shows an in-frame image and the in-frame image of the first video stream is transmitted.

[0145] In one embodiment, when the parameter switching speed is less than a set threshold, video frames from the video stream of the previous target parameter are transmitted until the current time is the switching time and the first video stream displays an intra-frame image. The intra-frame image of the first video stream is then transmitted.

[0146] In one embodiment, when the parameter switching speed is less than a set threshold, video frames from the video stream of the previous target parameter are transmitted until the current time exceeds the switching time and the first video stream displays an intra-frame image. The intra-frame image of the first video stream is then transmitted.

[0147] The first video stream can display an in-frame image during the stream switching time or after the stream switching time. If the parameter switching speed is less than a set threshold, video frames can be selected from the video stream with the current parameters until an in-frame image appears in the first video stream, at which point the in-frame image of the first video stream is transmitted. When transmitting the in-frame image of the first video stream, it is also necessary to ensure that the current time is either the stream switching time or has exceeded the stream switching time.

[0148] The video frames included in the first and second video streams can be sorted according to their frame numbers. The content of the frame number is not limited and can be an identifier that indicates the order of the frames.

[0149] The first video stream containing intra-frame images can be considered as following a video frame in the video stream transmitting the previous target parameter, where subsequent adjacent frames in the first video stream are intra-frame images. These subsequent adjacent frames can be video frames whose frame numbers are adjacent to the frame numbers of the transmitted video frames, following the transmitted video frames.

[0150] For example, if the video frame in the transmitted video stream is the 3rd frame, then if the 4th frame in the first video stream is an intra-frame image, then the 4th frame of the first video stream is transmitted.

[0151] In one embodiment, transmitting video frames from the first video stream corresponding to the target parameters further includes:

[0152] Transmit video frames following the intra-frame image in the first video stream until the second switching condition is met.

[0153] If the second switching condition is not met, video frames can be continuously selected and transmitted from the first video stream.

[0154] In one embodiment, when a first switching condition is met, transmitting video frames from the first video stream corresponding to the target parameter includes:

[0155] When the parameter switching speed is greater than or equal to a set threshold, multiple video frames of the first video stream are transmitted.

[0156] When the parameter switching speed is greater than or equal to a set threshold, and the current time is the stream switching time, multiple video frames of the first video stream are transmitted; or,

[0157] When the parameter switching speed is greater than or equal to a set threshold, and the current time exceeds the stream switching time, multiple video frames of the first video stream are transmitted.

[0158] The plurality of video frames include video frames from the reference time to the current time, wherein the reference time is the time corresponding to the intra-frame image referenced by the video frame at the current time.

[0159] When the parameter switching speed is greater than or equal to a set threshold, and the current time is or exceeds the switching time, multiple video frames of the first video stream are transmitted.

[0160] When the parameter switching speed is greater than or equal to a set threshold, multiple video frames can be selected from the first video stream. These multiple video frames include the reference frames required for the video frame corresponding to the current time. Selecting multiple video frames allows the electronic device that displays the video frames to decode the video frame corresponding to the stream switching time based on the multiple video frames.

[0161] The video frames corresponding to the reference time and the cut-off time include the video frames corresponding to the reference time and the video frames corresponding to the cut-off time, as well as the video frames between these two. The reference time can be considered as the time corresponding to the nearest I-frame before the video frame corresponding to the cut-off time.

[0162] In one embodiment, when the parameter switching speed is greater than or equal to a set threshold, i.e., when it is determined based on the parameter switching speed that the target parameter switching is frequent, and the first switching condition is currently met, this embodiment can select video frames from the first video stream. The selected video frame can be one frame or multiple frames. When the video frame at the stream switching time is an I-frame, one frame is selected; when the video frame at the stream switching time is a P-frame, multiple video frames are selected, such as video frames from the reference time to the current time.

[0163] If the first switching condition is not met, select the video frame at the switching time and at least one video frame before the switching time from the first video stream. The selected video frames include I-frames.

[0164] In one embodiment, transmitting video frames from the first video stream corresponding to the target parameters further includes:

[0165] Transmit video frames following the current time frame in the first video stream until the second switching condition is met.

[0166] If the second switching condition is not met, transmit the video frame following the video frame corresponding to the current time of the first video stream.

[0167] Figure 3 This is a schematic flowchart of another video processing method provided in this embodiment. This embodiment is applicable to situations where electronic devices process video in streaming scenarios. The method can be executed by a video processing device, which can be a mobile terminal or PC for video viewing and / or display. Details not covered in this embodiment are provided in the above embodiments and will not be repeated here.

[0168] like Figure 3 As shown, the method includes:

[0169] S310. Transmit flow cutting indication information, wherein the flow cutting indication information includes target parameters.

[0170] If the target parameters of the video to be viewed by this electronic device change, a stream switching instruction can be transmitted.

[0171] S320, receives video frames.

[0172] The received video frames are selected based on the flow cut indication information.

[0173] The video frames include video frames in the first video stream corresponding to the target parameters received when the first switching condition is met, and video frames in the second video stream corresponding to the target parameters received when the second switching condition is met.

[0174] The first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream.

[0175] This step involves decoding the received video frame and then displaying the decoded video frame.

[0176] The video frames received in this embodiment are based on this embodiment. Figure 1c and Figure 2 The method provided in the corresponding embodiment is determined.

[0177] In this embodiment of the disclosure, after transmitting the stream switching instruction information, video frames are received, thereby achieving frame-level stream switching, reducing the amount of data transmitted, and improving the smoothness of video stream switching.

[0178] Figure 4a This is a flowchart illustrating another video processing method provided in this disclosure embodiment. This embodiment further includes decoding the video frame; displaying the decoded desired video frame, wherein the desired video frame is the video frame after removing the target video frame from the decoded video frame, and the target video frame is the video frame decoded from the video frames corresponding to the period from the reference time to the current time. Figure 4a As shown, the method includes:

[0179] S410. Transmit flow cutting indication information, the flow cutting indication information including target parameters.

[0180] S420, receives video frames.

[0181] S430, Decode the video frame.

[0182] When decoding video frames, the existing decoding method can be used. When switching from the P-frame of the first video stream to the P-frame of the second video stream, the P-frame of the second video stream can be decoded based on the P-frame of the first video stream.

[0183] S440: Displays the required video frames after decoding.

[0184] The required video frame is the video frame after removing the target video frame from the decoded video frame. The target video frame is the video frame after decoding the video frames corresponding to the reference time up to the current time.

[0185] In free-view video scenes, to facilitate decoding of video frames at the cut-off time, multiple video frames need to be selected. These multiple video frames include those corresponding to the reference time up to the current time. During this step, only the decoded video frame corresponding to the current time needs to be displayed.

[0186] The video frames corresponding to the time between the reference time and the cutoff time include the video frames corresponding to the reference time, as well as the video frames after the reference time and before the cutoff time.

[0187] The following is an exemplary description of this disclosure, the purpose of which is to propose a video frame-level streaming technique to provide users with a smooth experience during changes in viewing angle. Compared with existing technologies, this technique has significant advantages in terms of bandwidth, cost, and latency.

[0188] This disclosure describes a video frame-level switching technique for multi-view applications. This technique is not limited to 360-degree video and free-view scenarios; it can also be applied to switching between normal video streams, such as switching between video streams of different resolutions in ABR (Automatic Broadcast Logic), where the target parameter is resolution. The following explanation uses viewpoint as the target parameter. The switching methods for other target parameters are the same as those for viewpoint switching, the difference being the target parameter itself. For example, the target parameter could be viewpoint or resolution.

[0189] The frame-level real-time streaming switching technology solution is as follows:

[0190] For each viewpoint, the server (i.e., the electronic device holding the video) encodes and generates two video streams separately; that is, each viewpoint is encoded into two separate streams.

[0191] Normal playback video (i.e., the second video stream): The video stream generated using normal encoding parameters. This stream is the video used when the user watches it normally.

[0192] Stream cut-through auxiliary video (i.e., first video stream): also known as IP frame video, is a video generated by alternating I-frames and P-frames when encoding a viewpoint. This stream is used to ensure smooth video playback when the user's viewpoint changes.

[0193] Based on the user's rotation viewpoint command (i.e., stream switching instruction information), the server selects the video frames required for the user's viewpoint display in real time from the stream switching auxiliary video stream and the normal playback video stream, encapsulates them into a single video stream, and sends it to the client (i.e., the electronic device viewing the video) for real-time decoding and display. With the assistance of the stream switching auxiliary video, when the user switches viewpoints, they can directly refer to the frames of the stream switching auxiliary video for decoding, rather than a video block or GoP, thus achieving frame-level viewpoint switching.

[0194] Compared to existing streaming technologies, this solution effectively reduces bandwidth pressure on the user side. When the user is watching stably, only the content covered by the user's current viewpoint is transmitted. When the user's viewpoint changes, only frames of the streaming auxiliary video are transmitted instead of a video block or a GoP, ensuring smooth viewpoint switching. At the same time, frame-level streaming technology allows users to download video data on demand, avoiding data waste.

[0195] Frame-level streaming technology can be applied to a variety of different video scenarios, such as 360VR panoramic video and free-viewpoint interactive video.

[0196] When encoding two video streams, the server encodes and outputs two separate video streams for each viewpoint. Encoding requires configuring certain encoder parameters to ensure successful decoding. The following are encoding commands for ffmpeg; other encoding commands for different encoders can also achieve similar results:

[0197] When encoding the slicing auxiliary video, set the GOP parameter to a size of 2 to ensure that the output slicing auxiliary video stream is a video stream with I-frames and P-frames interspersed.

[0198] When the encoded video plays normally, no B-frames are introduced. ffmpeg is used to control the minimum keyframe interval parameter `keyint_min` to the GOP size, ensuring the encoder does not add extra I-frames to the video. Simultaneously, the reference frame count is set to 1, ensuring that video frames are decoded with reference to the previous frame.

[0199] In a normal playback video encoding, the GOP size is 30, the minimum keyframe interval is 30, the number of reference frames is 1, and there are no B-frames.

[0200] Figure 4b This is a schematic diagram of a stream switching method provided in this embodiment. When the user is watching normally, the client retrieves video frames from the normal bitstream, i.e., the second video stream (Si-N), to the local machine for decoding and playback. When the user's viewing angle changes and stream switching is required, an auxiliary stream, i.e., the first video stream (Si-R), is used as an aid. Each time, the auxiliary stream from the next viewing angle must be entered first, and after one P-frame, the connection is switched to the normal bitstream. The basic strategy for frame-level stream switching is as follows:

[0201] (a) When switching to the target viewpoint, it is necessary to wait for an I-frame to appear in the R-stream of that viewpoint before switching, which will be delayed by up to 1 frame (because the R-stream GOP is 2). That is, the video frame of the first video stream at the time of the stream switching is an I-frame or an I-frame appears in the first video stream. When switching streams, the I-frame of the auxiliary stream is used first to help switch the viewpoint, and then the normal stream, i.e., the second video stream, is switched back after one P-frame.

[0202] (b) The R stream needs to transmit P frames before switching to the P frames of the N stream, meaning that forward-predictive coded frames appear in the video frames of the first video stream being transmitted. If switching to the I frames of the N stream, the cut can be made directly, meaning that an intra-frame image appears in the second video stream after the cut-off time. This achieves the ability to directly switch to the I and P frames of the normal stream after transmitting P frames in the auxiliary stream;

[0203] (c) When switching perspectives continuously, the R stream of one perspective can be directly switched to the R stream of the next perspective without switching to the N stream; that is, video frames are selected from the video stream of the previous target parameter, wherein the video stream of the previous target parameter is the first video stream of the previous target parameter or the second video stream of the previous target parameter, and the previous target parameter is the target parameter included in the previously obtained stream switching indication information.

[0204] When the video is a free-viewpoint video, the stream-cutting strategy (a) does not need to wait for the R stream to produce an I-frame, that is, it does not need to wait for the first video stream of the target parameter to produce an I-frame, and directly transmits multiple video frames of the first video stream. The multiple video frames include video frames from the reference time to the current time.

[0205] The following is based on Figure 4b Let's take an example to illustrate the specific implementation of stream switching. Assume there are n viewpoints, denoted as S1, S2, ..., Sn, and the video frames of each viewpoint are arranged according to their frame numbers as F1, F2, F3, ... Figure 4b The diagram shows a scenario with 5 viewpoints and 10 video frames. The normal playback video from viewpoint Si is called Si-N, and the stream-switching auxiliary video is called Si-R. Both video streams only have I-frames and P-frames, without B-frames (if B-frames are present, they can be converted to P-frames for post-processing). Consider that after the user's gaze moves from viewpoint 1 to viewpoint 4, they remain at viewpoint 4 for an extended period.

[0206] When a user switches from view 1 to view 2 at F2, S1F1-N-(I), S1F2-N-(P), and S2F3-R-(I) need to be decoded, resulting in a delay of 1 frame; when a user switches from view 2 to view 3 at F5, S2F4-R-(P), S3F5-R-(I), and S3F6-R-(P) need to be decoded, without any delay (as described in strategy (a)).

[0207] When the user stays at viewpoint 4, at time F9 the user can switch from R stream to N stream (as described in strategy (b)).

[0208] During the user's continuous switching from viewpoint 1 to viewpoint 4, the switching is performed directly between the auxiliary streams. The client needs to download and decode the video frames S1F1-N, S1F2-N, S2F3-R, S2F4-R, S3F5-R, S3F6-R, S4F7-R, S4F8-R, S4F9-N, and S4F10-N sequentially.

[0209] There is a one-to-one correspondence between video frames and time, and the frame number can be identified by time. The time increments from F1 to F10. In this disclosure, the image quality is improved when the next I-frame appears during stream switching.

[0210] The technical means that support smooth streaming at the video frame level include:

[0211] 1. The I-frames of the auxiliary stream R stream can be decoded independently, and there is one I-frame every two frames, which ensures that the user can switch to the auxiliary stream smoothly when the viewpoint changes, such as S1F2-N->S2F3-R.

[0212] 2. After the user's perspective stops changing, the P frames of the normal bitstream N stream can be directly decoded by referring to the P frames of the R stream. This allows the client to switch at a lower bitrate without having to re-fetch the entire GoP or stay on the auxiliary stream for a long time. For example, S4F8-R->S4F9-N.

[0213] The server provides the following flow-switching commands, or flow-switching indication information, to the client. The client detects the user's rotation behavior and sends flow-switching commands to the server as needed to trigger flow switching:

[0214] The flow cut indication information includes target parameters (such as target viewpoint), flow cut time, and parameter switching speed.

[0215] Commonly used flow-cutting commands include:

[0216] Immediate stream switching sets the stream switching time to the current time. Delayed stream switching sets the stream switching time to a timestamp; if the server determines that the current time has exceeded the switching time, it immediately switches the stream; otherwise, the server waits until the switching time is reached before switching. Pre-deployment mode is enabled if the parameter switching speed is high (greater than a set threshold). (Pre-deployment is disabled by default; refer to the example in the free-view example for enabling pre-deployment.) Pre-deployment mode selects multiple video frames directly from the first video stream without waiting for an I-frame to appear. These multiple video frames include those corresponding to the time range from the reference time to the current time. In other words, video frames from the first video stream can be deployed without waiting for an I-frame to appear.

[0217] Figure 4c This is yet another schematic diagram of flow cutting provided in the embodiments of this disclosure; Figure 4d yes Figure 4c A diagram illustrating video frame encoding in a streaming scenario is provided. Figure 4c and Figure 4d In a 360VR panoramic video scenario, assuming the 360 ​​panoramic video is divided into 20 viewpoints at a frame rate of 30fps, and the user only moves along one coordinate axis (e.g., the yaw axis) within the 360 ​​video coordinate system, with the yaw axis rotating no more than 90 degrees within 500ms during viewing (covering 5 viewpoints), the image decoded and displayed by the client would be as follows: Figure 4c The client downloads all video frames in the buffer shown.

[0218] Depend on Figure 4c As can be seen, the playback resumes in a single-view mode starting at frame S6F18, after previously being in a state of view rotation, i.e., view switching.

[0219] Figure 4e Another schematic diagram of flow cutting provided in this disclosure embodiment; Figure 4f yes Figure 4eA schematic diagram of video frame encoding in a streaming scenario; see [link / reference] Figure 4e and Figure 4f Assuming the free-view video has 48 viewpoints covering 360 degrees and a frame rate of 15fps, meaning all viewpoints refresh a frame every 66ms. During viewing, the user can rotate the viewing angle by no more than 120 degrees within 500ms, covering 16 viewpoints.

[0220] In this case, the client decodes and displays the image. Figure 4e The client download buffer shown includes S3RF1, S4RF1, S5RF1, S8RF3, S9RF3, S12RF5, S13RF5, S16RF7, and S17RF7 (i.e., Figure 4f The video frames outside the selected video frames. S3RF1, S4RF1, S5RF1, S8RF3, S9RF3, S12RF5, S13RF5, S16RF7, and S17RF7 are only decoded and not displayed. S3RF1, S4RF1, S5RF1, S8RF3, S9RF3, S12RF5, S13RF5, S16RF7, and S17RF7 are the target video frames.

[0221] Figure 5 This is a schematic diagram of the structure of a video processing device provided in an embodiment of the present disclosure. The device includes: an acquisition module 510, a first transmission module 520, and a second transmission module 530.

[0222] The acquisition module 510 is used to acquire flow cutting indication information, the flow cutting indication information including target parameters;

[0223] The first transmission module 520 is used to transmit video frames in the first video stream corresponding to the target parameters when the first switching condition is met.

[0224] The second transmission module 530 is used to transmit video frames in the second video stream corresponding to the target parameters when the second switching condition is met.

[0225] The first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream.

[0226] The technical solution provided in this embodiment first acquires stream switching indication information, including target parameters, through acquisition module 510; then, under the condition of satisfying a first switching condition, the first transmission module 520 transmits video frames from the first video stream corresponding to the target parameters; secondly, under the condition of satisfying a second switching condition, the second transmission module 530 transmits video frames from the second video stream corresponding to the target parameters. By transmitting selected video frames, the solution addresses the problem of large interactive data volume caused by transmitting the entire video stream during video stream switching. It achieves frame-level stream switching, reduces the amount of transmitted data, and improves the smoothness of video stream switching.

[0227] In one embodiment, the flow switching indication information includes a flow switching time, which indicates the time to switch to the target parameter.

[0228] In one embodiment, the first switching condition includes: the current time is the switching time or the current time is later than the switching time.

[0229] In one embodiment, the first switching condition further includes: the video frame of the first video stream at the switching time is an I-frame.

[0230] In one embodiment, the flow switching indication information further includes parameter switching speed, and the first transmission module 520 is specifically used for:

[0231] When the parameter switching speed is less than a set threshold, the video frame of the video stream of the previous target parameter is transmitted until the first video stream has an in-frame image and the in-frame image of the first video stream is transmitted.

[0232] In one embodiment, the first transmission module 520 is specifically used to transmit video frames following the intra-frame images in the first video stream until the second switching condition is met.

[0233] In one embodiment, the flow switching indication information further includes parameter switching speed, and the first transmission module 520 is specifically used for:

[0234] When the parameter switching speed is less than a set threshold, the video frame of the video stream of the previous target parameter is transmitted until the current time is the stream switching time and the first video stream shows an in-frame image and the in-frame image of the first video stream is transmitted; or until the current time exceeds the stream switching time and the first video stream shows an in-frame image and the in-frame image of the first video stream is transmitted.

[0235] In one embodiment, the first transmission module 520 is specifically used for:

[0236] When the parameter switching speed is greater than or equal to a set threshold, multiple video frames of the first video stream are transmitted; or,

[0237] When the parameter switching speed is greater than or equal to a set threshold, and the current time is the stream switching time, multiple video frames of the first video stream are transmitted; or,

[0238] When the parameter switching speed is greater than or equal to a set threshold, and the current time exceeds the stream switching time, multiple video frames of the first video stream are transmitted.

[0239] The plurality of video frames include video frames from the reference time to the current time, wherein the reference time is the time corresponding to the intra-frame image referenced by the video frame at the current time.

[0240] In one embodiment, the first transmission module 520 is specifically used to transmit video frames after the current time video frame in the first video stream until the second switching condition is met.

[0241] In one embodiment, the second switching condition includes one of the following:

[0242] Forward predictive coded frames appear in the video frames of the first video stream being transmitted;

[0243] After the cut-off time, an intra-frame image appears in the second video stream.

[0244] In one embodiment, the apparatus further includes: a third selection module, configured to select a video frame from the video stream of the previous target parameter before transmitting a video frame in the first video stream corresponding to the target parameter, wherein the video stream of the previous target parameter is either the first video stream or the second video stream of the previous target parameter, and the previous target parameter is the target parameter included in the previously obtained stream cutting indication information.

[0245] In one embodiment, the target parameters include the target viewing angle and / or the target resolution.

[0246] In one embodiment, the first video stream and the second video stream do not include bidirectional prediction frames (B-frames), and the minimum keyframe interval of the second video stream is equal to the frame group length of the second video stream.

[0247] The video processing apparatus provided in this disclosure embodiment can execute this disclosure. Figure 1c and Figure 2 The video processing method provided in the corresponding embodiment has the corresponding functional modules and beneficial effects of the method.

[0248] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit and functional module are only for the purpose of distinguishing them from each other, and are not used to limit the protection scope of the embodiments of this disclosure.

[0249] Figure 6 This is a schematic diagram of another video processing device provided in an embodiment of the present disclosure. The device includes: a transmission module 610 and a receiving module 620.

[0250] Transmission module 610 is used to transmit flow-cutting indication information, the flow-cutting indication information including target parameters;

[0251] The receiving module 620 is used to receive video frames, the video frames including video frames in the first video stream corresponding to the target parameter received when the first switching condition is met, and video frames in the second video stream corresponding to the target parameter received when the second switching condition is met.

[0252] The first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream.

[0253] The technical solution provided in this embodiment first transmits flow-cutting indication information through the transmission module 610, the flow-cutting indication information including target parameters; then, the receiving module 620 receives video frames.

[0254] In this embodiment of the disclosure, after transmitting the stream switching instruction information, video frames are received, thereby achieving frame-level stream switching, reducing the amount of data transmitted, and improving the smoothness of video stream switching.

[0255] In one embodiment, the device further includes:

[0256] A decoding module is used to decode the video frames;

[0257] The display module is used to display the decoded required video frame. The required video frame is the video frame after removing the target video frame from the decoded video frame. The target video frame is the video frame after decoding the video frames corresponding to the reference time up to the current time.

[0258] The video frames received by this device are based on this disclosure. Figure 1c and Figure 2 The video processing method provided in the corresponding embodiment is determined.

[0259] The video processing apparatus provided in this disclosure embodiment can execute this disclosure. Figure 3 and Figure 4a The video processing method provided in the corresponding embodiment has the corresponding functional modules and beneficial effects of the method.

[0260] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit and functional module are only for the purpose of distinguishing them from each other, and are not used to limit the protection scope of the embodiments of this disclosure.

[0261] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Refer to the following... Figure 7 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 7 The diagram below shows the structure of the terminal device or server 700. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0262] like Figure 7 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An edit / output (I / O) interface 705 is also connected to the bus 704.

[0263] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0264] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0265] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0266] The electronic device provided in this embodiment and the video processing method provided in the above embodiments belong to the same concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0267] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the video processing method provided in the above embodiments. The computer storage medium may be a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the video processing method provided in this disclosure.

[0268] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0269] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0270] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0271] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to:

[0272] Obtain flow cut indication information, which includes target parameters;

[0273] If the first switching condition is met, the video frames in the first video stream corresponding to the target parameters are transmitted.

[0274] If the second switching condition is met, transmit the video frames in the second video stream corresponding to the target parameters;

[0275] The first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream.

[0276] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire flow-cutting instruction information, the flow-cutting instruction information including target parameters;

[0277] If the first switching condition is met, the video frames in the first video stream corresponding to the target parameters are transmitted.

[0278] If the second switching condition is met, transmit the video frames in the second video stream corresponding to the target parameters;

[0279] The first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream.

[0280] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: transmit flow-cutting instruction information, the flow-cutting instruction information including target parameters;

[0281] Receive video frames, the video frames including video frames in the first video stream corresponding to the target parameter received when the first switching condition is met, and video frames in the second video stream corresponding to the target parameter received when the second switching condition is met.

[0282] The first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream.

[0283] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0284] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0285] The units or modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units or modules do not necessarily limit the specific unit or module; for example, the first transmission module can also be described as "the transmission module for video frames in the first video stream".

[0286] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0287] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0288] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0289] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0290] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A video processing method, characterized in that, The method includes: Acquire streaming indication information, which includes one or more of the following: target parameters, streaming time, and parameter switching speed; wherein, the streaming indication information is generated when the target parameters change during the user's video viewing process; If the first switching condition is met, video frames in the first video stream corresponding to the target parameter are transmitted; wherein, the first switching condition is the condition for switching the video stream corresponding to the target parameter when the target parameter changes; If the second switching condition is met, transmit the video frames in the second video stream corresponding to the target parameters; Wherein, the first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream; the first video stream is the stream-cutting auxiliary video used when the target parameter changes; the second video stream is the normal playback video used by the user for normal viewing; The step of transmitting video frames from the first video stream corresponding to the target parameters when the first switching condition is met includes: When the parameter switching speed is less than a set threshold, the video frame of the video stream of the previous target parameter is transmitted until the current time is the stream switching time and the first video stream shows an in-frame image and the in-frame image of the first video stream is transmitted; or until the current time exceeds the stream switching time and the first video stream shows an in-frame image and the in-frame image of the first video stream is transmitted. The second switching condition includes one of the following: Forward predictive coded frames appear in the video frames of the first video stream being transmitted; After the cut-off time, an intra-frame image appears in the second video stream.

2. The method according to claim 1, characterized in that, The cut-off time indicates the time to switch to the target parameter.

3. The method according to claim 2, characterized in that, The first switching condition includes: the current time is the switching time or the current time is later than the switching time.

4. The method according to claim 3, characterized in that, The first switching condition further includes: the video frame of the first video stream at the switching time is an in-frame image.

5. The method according to claim 1, characterized in that, The transmission of video frames in the first video stream corresponding to the target parameters further includes: Transmit video frames following the intra-frame image in the first video stream until the second switching condition is met.

6. The method according to claim 2, characterized in that, The step of transmitting video frames from the first video stream corresponding to the target parameters when the first switching condition is met includes: When the parameter switching speed is greater than or equal to a set threshold, and the current time is the switching time or exceeds the switching time, multiple video frames of the first video stream are transmitted. The plurality of video frames include video frames from the reference time to the current time, wherein the reference time is the time corresponding to the intra-frame image referenced by the video frame at the current time.

7. The method according to claim 6, characterized in that, The transmission of video frames in the first video stream corresponding to the target parameters further includes: Transmit video frames following the current time frame in the first video stream until the second switching condition is met.

8. The method according to claim 1, characterized in that, Before transmitting the video frames in the first video stream corresponding to the target parameters, the method further includes: Select video frames from the video stream of the previous target parameter, wherein the video stream of the previous target parameter is either the first video stream of the previous target parameter or the second video stream of the previous target parameter, and the previous target parameter is the target parameter included in the previously obtained stream cutting indication information.

9. The method according to any one of claims 1-8, characterized in that, The target parameters include the target viewing angle and / or the target resolution.

10. The method according to any one of claims 1-8, characterized in that, The first video stream and the second video stream do not include bidirectional prediction frames, and the minimum keyframe interval of the second video stream is equal to the frame group length of the second video stream.

11. A video processing method, characterized in that, The method includes: Transmitting stream switching indication information, the stream switching indication information including one or more of the following: target parameters, stream switching time, parameter switching speed; wherein, the stream switching indication information is generated when the target parameters change during the user's video browsing process; The system receives video frames, which include video frames in a first video stream corresponding to the target parameter received when a first switching condition is met, and video frames in a second video stream corresponding to the target parameter received when a second switching condition is met; wherein, the first switching condition is a condition for switching the video stream corresponding to the target parameter when the target parameter changes. Wherein, the first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream; the first video stream is the stream-cutting auxiliary video used when the target parameter changes; the second video stream is the normal playback video used by the user for normal viewing; Wherein, when the first switching condition is met, the video frame in the first video stream corresponding to the target parameter received is the video frame of the video stream of the previous target parameter that is transmitted when the parameter switching speed is less than a set threshold, until the current time is the switching time and the first video stream has an in-frame image and the in-frame image of the first video stream is transmitted; or until the current time exceeds the switching time and the first video stream has an in-frame image and the in-frame image of the first video stream is transmitted. The second switching condition includes one of the following: Forward predictive coded frames appear in the video frames of the first video stream being transmitted; After the cut-off time, an intra-frame image appears in the second video stream.

12. A video processing method, characterized in that, The method includes: Transmitting stream switching indication information, the stream switching indication information including one or more of the following: target parameters, stream switching time, parameter switching speed; wherein, the stream switching indication information is generated when the target parameters change during the user's video browsing process; The system receives video frames, which include video frames in a first video stream corresponding to the target parameter received when a first switching condition is met, and video frames in a second video stream corresponding to the target parameter received when a second switching condition is met; wherein, the first switching condition is a condition for switching the video stream corresponding to the target parameter when the target parameter changes. Wherein, the first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream; the first video stream is the stream-cutting auxiliary video used when the target parameter changes; the second video stream is the normal playback video used by the user for normal viewing; Wherein, when the first switching condition is met, the video frame in the first video stream corresponding to the target parameter received is the video frame of the video stream of the previous target parameter that is transmitted when the parameter switching speed is less than a set threshold, until the current time is the switching time and the first video stream has an in-frame image and the in-frame image of the first video stream is transmitted; or until the current time exceeds the switching time and the first video stream has an in-frame image and the in-frame image of the first video stream is transmitted. The second switching condition includes one of the following: Forward predictive coded frames appear in the video frames of the first video stream being transmitted; After the cut-off time, an intra-frame image appears in the second video stream; The received video frames are determined based on the method described in any one of claims 1-10.

13. A video processing apparatus, characterized in that, include: The acquisition module is used to acquire stream switching indication information, which includes one or more of the following: target parameters, stream switching time, and parameter switching speed; wherein, the stream switching indication information is generated when the target parameters change during the user's video browsing process; A first transmission module is configured to transmit video frames in a first video stream corresponding to the target parameter when a first switching condition is met; wherein, the first switching condition is a condition for switching the video stream corresponding to the target parameter when the target parameter changes; The second transmission module is used to transmit video frames in the second video stream corresponding to the target parameters when the second switching condition is met. Wherein, the first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameters, and the frame group length of the first video stream is less than the frame group length of the second video stream; the first video stream is the stream-cutting auxiliary video used when the target parameters change; the second video stream is the normal playback video used by the user for normal viewing; The first transmission module is specifically used for: When the parameter switching speed is less than a set threshold, the video frame of the video stream of the previous target parameter is transmitted until the current time is the stream switching time and the first video stream shows an in-frame image and the in-frame image of the first video stream is transmitted; or until the current time exceeds the stream switching time and the first video stream shows an in-frame image and the in-frame image of the first video stream is transmitted. The second switching condition includes one of the following: Forward predictive coded frames appear in the video frames of the first video stream being transmitted; After the cut-off time, an intra-frame image appears in the second video stream.

14. A video processing apparatus, characterized in that, include The transmission module is used to transmit stream switching indication information, which includes one or more of the following: target parameters, stream switching time, and parameter switching speed; wherein, the stream switching indication information is generated when the target parameters change during the user's video browsing process; A receiving module is configured to receive video frames, the video frames including video frames in a first video stream corresponding to the target parameter transmitted when a first switching condition is met, and video frames in a second video stream corresponding to the target parameter transmitted when a second switching condition is met; wherein, the first switching condition is a condition for switching the video stream corresponding to the target parameter when the target parameter changes. Wherein, the first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream; the first video stream is the stream-cutting auxiliary video used when the target parameter changes; the second video stream is the normal playback video used by the user for normal viewing; Wherein, when the first switching condition is met, the video frame in the first video stream corresponding to the target parameter received is the video frame of the video stream of the previous target parameter that is transmitted when the parameter switching speed is less than a set threshold, until the current time is the switching time and the first video stream has an in-frame image and the in-frame image of the first video stream is transmitted; or until the current time exceeds the switching time and the first video stream has an in-frame image and the in-frame image of the first video stream is transmitted. The second switching condition includes one of the following: Forward predictive coded frames appear in the video frames of the first video stream being transmitted; After the cut-off time, an intra-frame image appears in the second video stream.

15. A video processing apparatus, characterized in that, include The transmission module is used to transmit stream switching indication information, which includes one or more of the following: target parameters, stream switching time, and parameter switching speed; wherein, the stream switching indication information is generated when the target parameters change during the user's video browsing process; A receiving module is configured to receive video frames, the video frames including video frames in a first video stream corresponding to the target parameter transmitted when a first switching condition is met, and video frames in a second video stream corresponding to the target parameter transmitted when a second switching condition is met; wherein, the first switching condition is a condition for switching the video stream corresponding to the target parameter when the target parameter changes. Wherein, the first video stream and the second video stream are video streams encoded with different encoding parameters for the video corresponding to the target parameter, and the frame group length of the first video stream is less than the frame group length of the second video stream; the first video stream is the stream-cutting auxiliary video used when the target parameter changes; the second video stream is the normal playback video used by the user for normal viewing; Wherein, when the first switching condition is met, the video frame in the first video stream corresponding to the target parameter received is the video frame of the video stream of the previous target parameter that is transmitted when the parameter switching speed is less than a set threshold, until the current time is the switching time and the first video stream has an in-frame image and the in-frame image of the first video stream is transmitted; or until the current time exceeds the switching time and the first video stream has an in-frame image and the in-frame image of the first video stream is transmitted. The second switching condition includes one of the following: Forward predictive coded frames appear in the video frames of the first video stream being transmitted; After the cut-off time, an intra-frame image appears in the second video stream; The received video frames are determined based on the method described in any one of claims 1-10.

16. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-12.

17. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the method as described in any one of claims 1-12.

Images