Encoding method, real-time communication method, apparatus, device, and storage medium

Abstract

The application provides an encoding method, a real-time communication method, a device, equipment and a storage medium, comprising: pre-encoding a current video frame according to a first encoding mode to obtain a pre-encoded frame of the current video frame, in the first encoding mode, a first video frame is pre-encoded as an I frame, and a video frame after the first video frame is pre-encoded as a P frame; if there is no I frame in M encoded frames before the current video frame, determining whether the current video frame is a scene switching frame according to the pre-encoded frame and a target pre-encoded frame, the scene switching frame is a video frame that has a scene switching relative to a previous video frame, the target pre-encoded frame is obtained by pre-encoding a previous video frame of the current video frame, one encoded frame is obtained by encoding a video frame before the current video frame, and M is a preset positive integer; if the current video frame is the scene switching frame, encoding the current video frame as an I frame; and if the current video frame is not the scene switching frame, encoding the current video frame as a P frame. Thus, the encoding efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to an encoding method, a real-time communication method, an apparatus, a device, and a storage medium. Background Technology

[0002] Video is a continuous sequence of images, composed of consecutive frames, each frame representing an image. Due to the persistence of vision, when the frame sequence is played at a certain rate, we perceive continuous motion in the video. Because consecutive frames have high similarity, the original video needs to be encoded and compressed to remove spatial and temporal redundancy for easier storage and transmission. Current video coding standards typically include I-frames and P-frames. I-frames are keyframes, obtained by fully encoding the current video frame. I-frames use only intra-frame prediction for encoding; the decoder can independently decode the content of an I-frame without information from other frames. I-frames generally serve as reference frames for subsequent frames and as entry points for bitstream switching. A sequence of encoded video images usually starts with an I-frame. P-frames are forward predictive coded frames, obtained by encoding the difference data between the current and previous video frames.

[0003] Since I-frames encode a complete video frame, their encoding efficiency is lower than that of P-frames. In existing video encoding methods, the first video frame is encoded as an I-frame, subsequent I-frames are encoded at fixed intervals, and video frames after the I-frames are encoded as P-frames. That is, there are multiple P-frames between two I-frames.

[0004] However, when a video frame scene changes, the inter-frame correlation between the current video frame and the previous video frame suddenly weakens, and the difference data between the current video frame and the previous video frame increases, resulting in an increase in the amount of P-frame data. If the video encoding is still performed in the above way, the encoding efficiency will be low. Summary of the Invention

[0005] This application provides an encoding method, a real-time communication method, an apparatus, a device, and a storage medium to improve encoding efficiency.

[0006] Firstly, an encoding method is provided, including:

[0007] The current video frame in the video stream is pre-encoded according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames.

[0008] If there is no I-frame among the M encoded frames preceding the current video frame, determine whether the current video frame is a scene transition frame based on the precoded frame and the target precoded frame. The scene transition frame is a video frame that has undergone a scene change relative to the previous video frame. The target precoded frame is obtained by precoding the video frame preceding the current video frame. The encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer.

[0009] If the current video frame is a scene transition frame, the current video frame is encoded as an I-frame; if the current video frame is not a scene transition frame, the current video frame is encoded as a P-frame.

[0010] Secondly, a real-time communication method is provided, including:

[0011] The real-time generated video is captured to obtain a video stream;

[0012] The current video frame in the video stream is pre-encoded according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames.

[0013] If there is no I-frame among the M encoded frames preceding the current video frame, determine whether the current video frame is a scene transition frame based on the precoded frame and the target precoded frame. The scene transition frame is a video frame that has undergone a scene change relative to the previous video frame. The target precoded frame is obtained by precoding the video frame preceding the current video frame. The encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer.

[0014] If the current video frame is a scene transition frame, the current video frame is encoded as an I-frame; if the current video frame is not a scene transition frame, the current video frame is encoded as a P-frame, thus obtaining the encoded frame of the current video frame.

[0015] The bitstream is obtained based on the encoded frame of the current video frame and multiple encoded frames preceding the current video frame;

[0016] The bitstream is sent to the client so that the client can display a virtual game screen based on the bitstream.

[0017] Thirdly, an encoding device is provided, comprising:

[0018] The first encoding module is used to pre-encode the current video frame in the video stream according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames.

[0019] The determining module is configured to determine whether the current video frame is a scene switching frame based on the precoded frame and the target precoded frame when there is no I-frame in the M encoded frames preceding the current video frame. The scene switching frame is a video frame that has undergone a scene change relative to the previous video frame. The target precoded frame is obtained by precoding the video frame preceding the current video frame. The encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer.

[0020] The second encoding module is used to encode the current video frame as an I-frame when the current video frame is a scene transition frame, and to encode the current video frame as a P-frame when the current video frame is not the scene transition frame.

[0021] Fourthly, a real-time communication device is provided, comprising:

[0022] The acquisition module is used to acquire video images from the real-time generated video to obtain a video stream;

[0023] The first encoding module is used to pre-encode the current video frame in the video stream according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames.

[0024] The determining module is configured to determine whether the current video frame is a scene switching frame if there is no I-frame among the M encoded frames preceding the current video frame, based on the precoded frame and the target precoded frame. The scene switching frame is a video frame that has undergone a scene change relative to the previous video frame. The target precoded frame is obtained by precoding the video frame preceding the current video frame. The encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer.

[0025] The second encoding module is used to encode the current video frame as an I-frame if the current video frame is a scene switching frame, and to encode the current video frame as a P-frame if the current video frame is not a scene switching frame, thereby obtaining the encoded frame of the current video frame.

[0026] The bitstream is obtained based on the encoded frame of the current video frame and multiple encoded frames preceding the current video frame;

[0027] The sending module is used to send the bitstream to the client so that the client can display a virtual game screen based on the bitstream.

[0028] Fifthly, an electronic device is provided, comprising: a processor and a memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform a method as described in the first aspect or its implementations, or in the second aspect or its implementations.

[0029] Sixthly, a computer-readable storage medium is provided for storing a computer program that causes a computer to perform the method as described in the first aspect or its implementations, or in the second aspect or its implementations.

[0030] A seventh aspect provides a computer program product including computer program instructions that cause a computer to perform a method as described in the first aspect or its implementations, or in the second aspect or its implementations.

[0031] In summary, this application first pre-encodes the current video frame in the video stream according to the first encoding method to obtain the pre-encoded frame of the current video frame. If there is no I-frame in the first M encoded frames of the current video frame, then when it is determined that the current video frame is a scene switching frame (i.e., a video frame scene switching has occurred) based on the pre-encoded frame and the target pre-encoded frame, the current video frame is encoded as an I-frame; otherwise, the current video frame is encoded as a P-frame. The target pre-encoded frame is obtained by pre-encoding the previous video frame of the current video frame. Thus, it achieves the insertion of I-frames when a video frame scene switching occurs. When inserting I-frames and then encoding P-frames, the number of bytes occupied by P-frames is reduced, thereby improving encoding efficiency. Since the encoding efficiency of I-frames is lower than that of P-frames, this application only inserts I-frames when it is determined that there are no I-frames in the first M encoded frames of the current video frame and a video frame scene switching has occurred, which avoids inserting too many I-frames and improves encoding efficiency. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram illustrating an application scenario of an encoding method provided in an embodiment of this application;

[0034] Figure 2 A flowchart illustrating an encoding method provided in an embodiment of this application;

[0035] Figure 3 A flowchart illustrating an encoding method provided in an embodiment of this application;

[0036] Figure 4 A flowchart illustrating an encoding method provided in an embodiment of this application;

[0037] Figure 5 A flowchart illustrating a real-time communication method provided in an embodiment of this application;

[0038] Figure 6 A flowchart illustrating a method for obtaining an optimal decoding configuration as provided in an embodiment of this application;

[0039] Figure 7 A flowchart illustrating a method for obtaining an optimal decoding configuration as provided in an embodiment of this application;

[0040] Figure 8 This is a schematic diagram of the structure of an encoding device provided in an embodiment of this application;

[0041] Figure 9 A schematic diagram of the structure of a real-time communication device provided in an embodiment of this application.

[0042] Figure 10 This is a schematic block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0045] Before introducing the technical solution of this application, the following is a brief introduction to relevant knowledge about this application:

[0046] 1. Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to cloud computing business models. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0047] 2. Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing. Cloud gaming technology enables thin clients with relatively limited graphics processing and data processing capabilities to run high-quality games. In a cloud gaming scenario, the game does not reside on the player's terminal but runs on a cloud server. The cloud server renders the game scene as a video and audio stream, which is then transmitted to the player's terminal via the network. The player's terminal does not need powerful graphics processing and data processing capabilities; it only needs basic streaming media playback capabilities and the ability to receive player input commands and send them to the cloud server.

[0048] 3. I-frame (Intra picture), also known as key frame or intra-coded frame, is obtained by fully encoding the current video frame. I-frames are encoded using only intra-prediction, and the decoder can independently decode the content of the I-frame without information from other frames. I-frames are generally used as reference frames for subsequent frames and also as entry points for bitstream switching.

[0049] 4. P-frame (predictive-frame) is a forward predictive coded frame. A P-frame is obtained by encoding the difference data between the current video frame and the previous video frame.

[0050] As mentioned above, existing video encoding methods suffer from low encoding efficiency when scene transitions occur in video frames. To address this issue, this application performs pre-encoding and secondary encoding when encoding the current video frame in the video stream. First, the current video frame is pre-encoded using a first encoding method to obtain a pre-encoded frame. If there are no I-frames in the first M encoded frames of the current video frame, then when it is determined that the current video frame is a scene transition frame (i.e., a scene transition has occurred) based on the pre-encoded frame and the target pre-encoded frame, the current video frame is encoded as an I-frame; otherwise, it is encoded as a P-frame. The target pre-encoded frame is obtained by pre-encoding the previous video frame of the current video frame. This achieves the insertion of I-frames when scene transitions occur. When inserting I-frames and then encoding P-frames, the number of bytes occupied by P-frames is reduced, thereby improving encoding efficiency. Since the encoding efficiency of I-frames is lower than that of P-frames, this application only inserts I-frames when it is determined that there are no I-frames in the first M encoded frames of the current video frame and a scene transition has occurred, thus avoiding excessive I-frame insertion and improving encoding efficiency.

[0051] It should be understood that the technical solution of this application can be applied to the following scenarios, but is not limited to:

[0052] For example, Figure 1 This is a schematic diagram illustrating an application scenario of an encoding method provided in an embodiment of this application, such as... Figure 1 As shown, the terminal device 110 can communicate with the server 120. The terminal device 110 has a streaming media playback function, and the server 120 has a graphics processing function, such as image segmentation and image fusion. The server 120 also has a video and audio stream data transmission function, such as video encoding.

[0053] In some possible ways, Figure 1 The application scenarios shown can also include: base stations, core network side equipment, etc., in addition, Figure 1 An exemplary terminal device and a server are shown, but in practice, other numbers of terminal devices and servers may be included, and this application does not limit this.

[0054] In some possible ways, Figure 1 The server 120 in this application can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. This application does not impose any restrictions on this.

[0055] In the context of cloud gaming, a cloud server refers to a server that runs games in the cloud and has functions such as video enhancement (pre-encoding processing) and video encoding, but is not limited to these.

[0056] Terminal devices refer to a class of devices that have rich human-computer interaction methods, internet access capabilities, typically run various operating systems, and possess strong processing power. Terminal devices can be smartphones, living room TVs, tablets, in-vehicle terminals, game consoles such as handheld game consoles, etc., but are not limited to these.

[0057] Taking cloud gaming as an example, cloud gaming providers primarily rely on hardware encoders from hardware manufacturers to encode and transmit game content. Hardware encoders are characterized by extremely high encoding speeds, but the intermediate encoded data is difficult to extract and utilize. Simultaneously, cloud gaming providers, as users, find it difficult to modify the hardware encoding process and are mostly limited to using the capabilities provided by the hardware manufacturers. The encoding method provided in this application is an adaptive I-frame insertion method, which can insert I-frames when video frame scene transitions occur, such as inserting I-frames at the moment of game scene transition, thereby improving the encoder's encoding efficiency. The encoding method provided in this application is applicable to cloud gaming scenarios and compatible with various hardware encoders.

[0058] The technical solution of this application will be described in detail below:

[0059] Figure 2 A flowchart illustrating an encoding method provided in this application embodiment, the method being, for example, by... Figure 1 The server 120 shown is executed, but is not limited to this, such as Figure 2 As shown, the method includes the following steps:

[0060] S101. Pre-encode the current video frame in the video stream according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames.

[0061] In this embodiment, the encoding mode of the pre-encoding and the encoding mode of the secondary encoding (referring to the encoding performed in S103) adopt an encoding mode that allows the size of the encoded frame data to reflect the matching degree of inter-frame prediction well. If the inter-frame correlation is high, the frame data size after encoding the current video frame is relatively small; conversely, the frame data size after encoding the current video frame is relatively large.

[0062] Optionally, the encoding mode for pre-coding and the encoding mode for secondary encoding (referring to the encoding performed in S103) can be a fixed quantization parameter (CQP) mode. Using CQP mode encoding allows for a smaller frame data size after encoding the current video frame if the inter-frame correlation is high, and vice versa. It should be noted that other encoding modes can also be used in the preset encoding mode in this application embodiment, and this application does not impose any restrictions on them.

[0063] Since precoding is used to obtain the data size of the precoded frame, to save precoding time, optionally, in S101, the current video frame is precoded according to the first encoding method to obtain the precoded frame of the current video frame, which can be:

[0064] The current video frame is downsampled according to the preset sampling width and preset sampling length to obtain a downsampled video frame. The downsampled video frame is then pre-encoded according to the first encoding method to obtain a pre-encoded frame of the current video frame.

[0065] The sampling width can be, for example, half the width of the original video frame, and the sampling length can be, for example, half the length of the original video frame. That is, the width of the downsampled video frame is half the width of the current video frame, and the length of the downsampled video frame is half the length of the current video frame. The preset sampling width and preset sampling length can also be other values, and this embodiment does not limit them. By downsampling the current video frame to obtain a downsampled video frame, and then precoding the downsampled video frame, precoding time can be saved, thereby improving the overall coding efficiency.

[0066] S102. If there is no I-frame among the M encoded frames preceding the current video frame, determine whether the current video frame is a scene switching frame based on the pre-coded frame and the target pre-coded frame. A scene switching frame is a video frame that has undergone a scene change relative to the previous video frame. The target pre-coded frame is obtained by precoding the previous video frame of the current video frame. An encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer.

[0067] One encoded frame is obtained by encoding the video frame preceding the current video frame, and the other encoded frame is obtained by performing secondary encoding on the video frame.

[0068] Where M is a preset positive integer. In order to avoid too many I-frames being inserted, the embodiments of this application use a preset M to avoid too many I-frames being inserted. For example, M is 128.

[0069] In this embodiment, since the first video frame is pre-coded as an I-frame and subsequent video frames are pre-coded as P-frames in the first encoding method, the pre-coded frame of the current video frame is a P-frame. The target pre-coded frame is the pre-coded frame of the previous video frame, which is also a P-frame. It is possible to determine whether the current video frame is a scene transition frame based on two consecutive P-frames. A scene transition frame is a video frame that has undergone a scene change relative to the previous video frame. Scene transition may include changes in the scene, image background, or other information of two consecutive video frames.

[0070] In one feasible approach, it can be determined whether the current video frame is a scene transition frame based on the size of the precoded frame and the size of the target precoded frame.

[0071] Optionally, the current video frame can be determined as a scene transition frame based on the number of bytes in the precoded frame and the number of bytes in the target precoded frame.

[0072] Optionally, if the ratio of the number of bytes in the precoded frame to the number of bytes in the target precoded frame is greater than or equal to a preset threshold, the current video frame is determined to be a scene transition frame; if the ratio of the number of bytes in the precoded frame to the number of bytes in the target precoded frame is less than the preset threshold, the current video frame is determined not to be a scene transition frame. Optionally, the preset threshold may be, for example, equal to 2, or may be other values; this embodiment does not limit the specific values.

[0073] Generally, due to the significantly stronger inter-frame correlation of video textures compared to intra-frame correlation, the data size of an I-frame is significantly larger than that of a P-frame. This pattern remains quite stable in CQP encoding. When a scene change occurs in the current video frame, the inter-frame correlation suddenly weakens, leading to an increase in the P-frame data size. If the scene difference is sufficiently large, the encoder may partially abandon inter-frame correlation and instead use intra-frame correlation to encode the current frame, resulting in the generation of a large number of intra-frame coded blocks. For two consecutive P-frames, if the ratio of the number of bytes in the second P-frame to the number of bytes in the first P-frame is greater than or equal to a preset threshold (e.g., 2, meaning the number of bytes in the second P-frame is greater than or equal to twice the number of bytes in the first P-frame), then the current video frame corresponding to the second P-frame is determined to be a scene-change frame.

[0074] Furthermore, if there is an I-frame among the M encoded frames preceding the current video frame, then the current video frame is encoded as a P-frame, or the current video frame is encoded according to the second encoding method and the preset encoding mode.

[0075] In one feasible embodiment of this application, the second encoding method may be: starting from the first video frame, encoding I-frames according to a preset period, and encoding subsequent video frames encoded as I-frames as P-frames. For example, the preset period is 200 frames, that is, the first video frame is encoded as an I-frame, the 201st video frame is encoded as an I-frame, the 401st video frame is encoded as an I-frame, ..., and subsequent video frames after each I-frame are encoded as P-frames, and the encoded frames between two I-frames are P-frames.

[0076] In this application embodiment, the above-described second encoding method is one implementable method. Optionally, the second encoding method may also be other implementation methods, and this application embodiment does not limit this.

[0077] The preset encoding mode can be CQP mode.

[0078] Specifically, in this embodiment of the application, if there is an I-frame among the M encoded frames preceding the current video frame, then the current video frame is encoded according to the second encoding method and the preset encoding mode. Specifically, before encoding the current video frame according to the second encoding method and the preset encoding mode, the current video frame is pre-encoded to obtain a pre-encoded frame. Then, it is determined whether there is an I-frame among the M encoded frames preceding the current video frame. If there is no I-frame, then an I-frame is inserted (i.e., encoded). If it is determined that there is an I-frame among the M encoded frames preceding the current video frame, then the current video frame is encoded according to the second encoding method and the preset encoding mode. For example, if the second encoding method is one of the aforementioned feasible encoding methods (i.e., encoding I-frames and P-frames according to a fixed period), then the current video frame is encoded as a P-frame.

[0079] Optionally, if there are no I-frames in the M encoded frames preceding the current video frame, the method in this embodiment may further include:

[0080] Count the number of encoded frames before the current video frame. When the number of encoded frames before the current video frame reaches M, check whether there are any I-frames among the M encoded frames before the current video frame.

[0081] S103. If the current video frame is a scene transition frame, encode the current video frame as an I-frame; if the current video frame is not a scene transition frame, encode the current video frame as a P-frame.

[0082] In this embodiment, I-frames are inserted at scene transitions, which can improve the encoder's encoding efficiency. For example, in cloud gaming, this can make cloud gaming services more bandwidth-efficient while maintaining the same image quality.

[0083] The encoding method provided in this application first pre-encodes the current video frame in the video stream according to a first encoding method to obtain a pre-encoded frame for the current video frame. If there are no I-frames in the first M encoded frames of the current video frame, then when it is determined that the current video frame is a scene switching frame (i.e., a video frame scene switching has occurred) based on the pre-encoded frame and the target pre-encoded frame, the current video frame is encoded as an I-frame; otherwise, the current video frame is encoded as a P-frame. The target pre-encoded frame is obtained by pre-encoding the previous video frame of the current video frame. Thus, it achieves the insertion of I-frames when a video frame scene switching occurs. When inserting I-frames and then encoding P-frames, the number of bytes occupied by P-frames is reduced, thereby improving encoding efficiency. Since the encoding efficiency of I-frames is lower than that of P-frames, this application only inserts I-frames when it is determined that there are no I-frames in the first M encoded frames of the current video frame and a video frame scene switching has occurred, which avoids inserting too many I-frames and improves encoding efficiency.

[0084] The technical solution of the encoding method provided in this application will be described in detail below with reference to a specific embodiment.

[0085] Figure 3 This is a flowchart illustrating an encoding method provided in an embodiment of this application, as shown below. Figure 3 As shown, the method in this embodiment may include:

[0086] S201. Pre-encode the current video frame in the video stream according to the first encoding method to obtain the pre-encoded frame of the current video frame.

[0087] In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames.

[0088] Figure 4 A flowchart illustrating an encoding method provided in this application embodiment, in conjunction with... Figure 4 The current video frame is any video frame in the video frame sequence. Specifically, S202 may include:

[0089] S2011. Downsample the current video frame according to the preset sampling width and preset sampling length to obtain the downsampled video frame.

[0090] The sampling width can be, for example, half the width of the original video frame, and the sampling length can be, for example, half the length of the original video frame. That is, the width of the downsampled video frame is half the width of the current video frame, and the length of the downsampled video frame is half the length of the current video frame. The preset sampling width and preset sampling length can also be other values, and this embodiment does not limit them.

[0091] S2012. Pre-encode the downsampled video frame according to the first encoding method to obtain the pre-encoded frame of the current video frame.

[0092] By downsampling the current video frame to obtain a downsampled video frame, and then precoding the downsampled video frame, precoding time can be saved, thereby improving the overall coding efficiency.

[0093] S202. Determine whether there is an I-frame among the M encoded frames preceding the current video frame, where M is a preset positive integer.

[0094] Specifically, count the number of encoded frames before the current video frame. When the number of encoded frames before the current video frame reaches M, determine whether there is an I-frame among the M encoded frames before the current video frame.

[0095] If yes, then execute S205 to encode the current video frame into a P-frame. If no, then execute S203. In this embodiment, for example, M equals 128.

[0096] In this embodiment, the encoding mode of the pre-encoding and the encoding mode of the secondary encoding (referring to the encoding performed by S204 or S205) can be, for example, CQP mode. Using CQP mode encoding can make the frame data volume of the current video frame relatively small if the inter-frame correlation is large, and vice versa.

[0097] S203. Determine the scene switching frame and whether the ratio of the number of bytes in the precoded frame to the number of bytes in the target precoded frame is greater than or equal to a preset threshold.

[0098] The target precoded frame is obtained by precoding the previous video frame of the current video frame. If it is a scene transition frame, then the current video frame is determined to be a scene transition frame, and step S204 is executed; otherwise, the current video frame is determined not to be a scene transition frame, and step S205 is executed. The preset threshold is, for example, 2.

[0099] S204. Encode the current video frame as an I-frame.

[0100] S205. Encode the current video frame into a P-frame.

[0101] After encoding the current video frame using S204 or S205, the encoded bitstream is obtained.

[0102] The encoding method provided in this embodiment performs pre-encoding and secondary encoding. The pre-encoded frames are used to determine whether a scene change has occurred. If there are no I-frames in the first M encoded frames of the current video frame and a scene change has occurred, the current video frame is encoded as an I-frame; otherwise, it is encoded as a P-frame. This achieves the insertion of I-frames when a video frame scene change occurs. When inserting I-frames and then encoding P-frames, the number of bytes occupied by P-frames is reduced, thereby improving encoding efficiency. Since the encoding efficiency of I-frames is lower than that of P-frames, this application only inserts I-frames when there are no I-frames in the first M encoded frames of the current video frame and a video frame scene change has occurred. This avoids inserting too many I-frames and improves encoding efficiency.

[0103] Figure 5 A flowchart of a real-time communication method provided in an embodiment of this application is shown below. Figure 5 As shown, the method in this embodiment may include:

[0104] S301. The server acquires video images from the real-time generated video to obtain a video stream.

[0105] The video stream includes multiple video frames, each of which contains an image composed of virtual game scenes.

[0106] S302. The server pre-encodes the current video frame in the video stream according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames.

[0107] S303. If there are no I-frames in the M encoded frames preceding the current video frame, the server determines whether the current video frame is a scene switching frame based on the pre-coded frame and the target pre-coded frame. A scene switching frame is a video frame that has changed scenes relative to the previous video frame. The target pre-coded frame is obtained by precoding the previous video frame of the current video frame. An encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer.

[0108] S304. If the current video frame is a scene transition frame, the server encodes the current video frame as an I-frame. If the current video frame is not a scene transition frame, the server encodes the current video frame as a P-frame, thus obtaining the encoded frame of the current video frame.

[0109] In this embodiment, the specific implementation method of the server encoding the current video frame can be found in [reference needed]. Figure 2 The descriptions in the illustrated embodiments will not be repeated here.

[0110] S305. The server obtains the bitstream based on the encoded frame of the current video frame and the multiple encoded frames preceding the current video frame.

[0111] S306, The server sends a bitstream to the client.

[0112] S307: The client displays the virtual game screen based on the bitstream.

[0113] The real-time communication method provided in this embodiment pre-encodes the current video frame in the video stream through a server to obtain a pre-encoded frame for the current video frame. If there are no I-frames in the first M encoded frames of the current video frame, then when it is determined that the current video frame is a scene switching frame (i.e., a video frame scene switching has occurred) based on the pre-encoded frame and the target pre-encoded frame, the current video frame is encoded as an I-frame; otherwise, the current video frame is encoded as a P-frame. The target pre-encoded frame is obtained by pre-encoding the previous video frame of the current video frame. Thus, it achieves the insertion of I-frames when a video frame scene switching occurs. When inserting I-frames and then encoding P-frames, the number of bytes occupied by P-frames is reduced, thereby improving encoding efficiency. Since the encoding efficiency of I-frames is lower than that of P-frames, this application only inserts I-frames when it is determined that there are no I-frames in the first M encoded frames of the current video frame and a video frame scene switching has occurred, which avoids inserting too many I-frames and improves encoding efficiency.

[0114] It should be understood that in cloud gaming scenarios, the above image encoding method is only meaningful when the decoding end, i.e., the aforementioned terminal device, has the ability to decode the encoded bitstream. The following will provide a method for obtaining the optimal decoding configuration.

[0115] Figure 6 A flowchart illustrating a method for obtaining an optimal decoding configuration provided in an embodiment of this application is shown below. Figure 6 As shown, the method includes:

[0116] S401, The cloud server sends a decoding capability request to the terminal device.

[0117] S402. The cloud server receives the decoding capability response from the terminal device. The decoding capability response includes the decoding capability of the terminal device.

[0118] S403. The cloud server determines the optimal decoding configuration based on the terminal device's decoding capability, the type of cloud game, and the current network status.

[0119] S404, the cloud server sends the optimal decoding configuration to the terminal device.

[0120] S405. The terminal device decodes the video stream bitstream using the optimal decoding configuration.

[0121] Optionally, Figure 7 A flowchart illustrating a method for obtaining an optimal decoding configuration provided in an embodiment of this application is shown below. Figure 7 As shown, the cloud server can send a decoding capability request to the terminal device through a client installed on the terminal device, and the terminal device can also return a decoding capability response to the cloud server through the same client. In a cloud gaming scenario, this client can be a cloud gaming client.

[0122] Optionally, the decoding capability request is used to request the decoding capability of the terminal device.

[0123] Optionally, the decoding capability request may include at least one of the following, but is not limited to: protocol version number, specific decoding protocol query.

[0124] Optionally, the protocol version number refers to the minimum protocol version supported by the cloud server, which may be a decoding protocol.

[0125] Optionally, the specific decoding protocol query refers to the decoding protocol that the cloud server wants to query, such as the video decoding protocol H264 or H265.

[0126] For example, the code implementation for a decoding capability request can be as follows:

[0127] [codec_ability] ; Encoding and decoding capabilities

[0128] version=1.0; the minimum protocol version supported by the cloud server.

[0129] type=16,17; Query H264 and H265 capabilities

[0130] For an explanation of the various data structures in this code, please refer to Table 1 below; this application will not elaborate further. The data structure for the terminal device's decoding capability is shown in Table 1:

[0131] Table 1 Data Structure of Terminal Device Decoding Capability

[0132]

[0133]

[0134] The definitions of each decoding protocol are shown in Table 2:

[0135] Table 2 Decoding Protocol

[0136] Decoding Protocol Enumeration definition H264 16 H265 17 AV1 48

[0137] The Profile definitions supported by the terminal device for each decoding protocol are shown in Table 3:

[0138] Table 3. Profile definitions supported by the decoding protocol

[0139]

[0140]

[0141] The Level definitions supported by the terminal device for each decoding protocol are shown in Table 4.

[0142] Table 4. Level definitions supported by the decoding protocol

[0143]

[0144]

[0145]

[0146] The Profile and Level supported by the terminal device are listed in the form of a tuple. For example, if device A supports H.264 capability: (Baseline, Level51), (Main, Level51), (High, Level51).

[0147] Optionally, in addition to including the decoding capability of the terminal device, the decoding capability response may also include: an indicator of whether the decoding protocol queried by the cloud server was successfully queried, and the protocol version number supported by the terminal device.

[0148] Optionally, if the query for the decoding protocol to be queried by the cloud server is successful, the indicator of whether the query for the decoding protocol to be queried by the cloud server is successful can be represented by 0. If the query for the decoding protocol to be queried by the cloud server fails, the indicator of whether the query for the decoding protocol to be queried by the cloud server is successful can be represented by an error code, such as 001.

[0149] Optionally, the protocol version number refers to the lowest protocol version supported by the terminal device, which may be a decoding protocol.

[0150] Optionally, the decoding capability of the terminal device includes at least one of the following, but is not limited to: the type of decoding protocol supported by the terminal device, the profile, level, and performance supported by the decoding protocol, etc.

[0151] Example 1: The code implementation for decoding capability response can be as follows:

[0152]

[0153]

[0154] For an explanation of the various data structures in this code, please refer to Table 1 below. This application will not elaborate further on them.

[0155] Example 2: If the terminal device only supports some decoding protocols, then the supported decoding protocol information is returned. The code implementation for the decoding capability response in this case can be as follows:

[0156]

[0157]

[0158] For an explanation of the various data structures in this code, please refer to Table 1 below. This application will not elaborate further on them.

[0159] Example 3: If the terminal device does not support the decoding protocol, it returns codecs=0. The code implementation for the decoding capability response in this case can be as follows:

[0160] [codec_ability] ; Encoding and decoding capabilities

[0161] state = 0; a successful query returns a status code of 0.

[0162] version=1.0; Terminal device protocol version

[0163] codecs=0; Supports 0 hardware codecs.

[0164] For an explanation of the various data structures in this code, please refer to Table 1 below. This application will not elaborate further on them.

[0165] Example 4: If the request for decoding capability from the terminal device fails, a specific error code is returned. The code implementation for the decoding capability response in this case can be as follows:

[0166] [codec_ability] ; Encoding and decoding capabilities

[0167] state = -1; query failure returns status code -1.

[0168] version = 0.9; Terminal device protocol version

[0169] For an explanation of the various data structures in this code, please refer to Table 1 below. This application will not elaborate further on them.

[0170] Optionally, for more complex cloud game types, the cloud server can select higher capabilities within the decoding capabilities of the terminal device. For example, in Example 1 above, profile3 and performance3 are selected. The cloud server can select the optimal decoding configuration according to the mapping relationship between the cloud game type and the decoding capabilities of the terminal device, or it can select the optimal decoding configuration according to other selection rules.

[0171] Optionally, for worse network conditions, the cloud server can select a higher capability within the decoding capability range of the terminal device. For example, in Example 1 above, profile3 and performance3 are selected. The cloud server can select the optimal decoding configuration according to the mapping relationship between network conditions and the decoding capability of the terminal device, or it can select the optimal decoding configuration according to other selection rules.

[0172] Optionally, the cloud server can select the optimal decoding configuration based on the mapping relationship between cloud game type, network status and terminal device decoding capability, or it can select the optimal decoding configuration according to other selection rules.

[0173] In summary, this application does not impose any restrictions on how the optimal decoding configuration is determined.

[0174] In summary, the technical solution provided in this embodiment enables the terminal device to decode the video stream bitstream using the optimal decoding configuration, thereby improving the decoding effect.

[0175] Figure 8 This is a schematic diagram of the structure of an encoding device provided in an embodiment of this application, as shown below. Figure 8 As shown, the encoding device may include: a first encoding module 11, a determining module 12, and a second encoding module.

[0176] The first encoding module 11 pre-encodes the current video frame in the video stream according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames.

[0177] The determining module 12 is used to determine whether the current video frame is a scene switching frame based on the pre-coded frame and the target pre-coded frame when there is no I-frame in the M encoded frames preceding the current video frame. The scene switching frame is a video frame that has changed scenes relative to the previous video frame. The target pre-coded frame is obtained by pre-coding the previous video frame of the current video frame. The encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer.

[0178] The second encoding module 13 is used to encode the current video frame as an I-frame when the current video frame is a scene transition frame, and to encode the current video frame as a P-frame when the current video frame is not a scene transition frame.

[0179] Optionally, the first encoding module 11 is further configured to: if there is an I-frame among the M encoding frames preceding the current video frame, then encode the current video frame as a P-frame, or encode the current video frame according to the second encoding method and the preset encoding mode.

[0180] Optionally, the first encoding module 11 is used to: downsample the current video frame according to a preset sampling width and a preset sampling length to obtain a downsampled video frame;

[0181] The downsampled video frame is pre-encoded according to the first encoding method to obtain the pre-encoded frame of the current video frame.

[0182] Optionally, the determining module 12 is also used to: count the number of encoded frames preceding the current video frame;

[0183] When the number of encoded frames preceding the current video frame reaches M, check whether there are any I-frames among the M encoded frames preceding the current video frame.

[0184] Optional, M is 128.

[0185] Optionally, the determining module 12 is used to: determine whether the current video frame is a scene switching frame based on the size of the precoded frame and the size of the target precoded frame.

[0186] Optionally, the determining module 12 is used to: determine whether the current video frame is a scene switching frame based on the number of bytes in the precoded frame and the number of bytes in the target precoded frame.

[0187] Optionally, module 12 is specifically used for:

[0188] If the ratio of the number of bytes in the precoded frame to the number of bytes in the target precoded frame is greater than or equal to a preset threshold, then the current video frame is determined to be a scene transition frame.

[0189] If the ratio of the number of bytes in the precoded frame to the number of bytes in the target precoded frame is less than a preset threshold, then the current video frame is determined not to be a scene transition frame.

[0190] Optionally, the preset encoding mode is the fixed quantization parameter CQP mode, and the pre-encoded encoding mode is the CQP mode.

[0191] Optionally, the second encoding method includes:

[0192] Starting from the first video frame, I-frames are encoded according to a preset period, and subsequent video frames encoded as I-frames are encoded as P-frames.

[0193] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 8 The encoding device shown can perform Figure 2 The corresponding method embodiments, and the foregoing and other operations and / or functions of each module in the encoding device are respectively for implementing Figure 2 For the sake of brevity, the corresponding processes in the method embodiments will not be described in detail here.

[0194] Figure 9 This is a schematic diagram of the structure of a real-time communication device provided in an embodiment of this application, as shown below. Figure 9 As shown, the real-time communication device may include: an acquisition module 21, a first encoding module 22, a determination module 23, a second encoding module 24, and a transmission module 25.

[0195] The receiving module 21 is used to acquire video images from the real-time generated video to obtain a video stream;

[0196] The first encoding module 22 is used to pre-encode the current video frame in the video stream according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames.

[0197] The determination module 23 is used to determine whether the current video frame is a scene switching frame if there is no I-frame in the M encoded frames preceding the current video frame, based on the pre-coded frame and the target pre-coded frame. A scene switching frame is a video frame that has changed scenes relative to the previous video frame. The target pre-coded frame is obtained by pre-coding the previous video frame of the current video frame. An encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer.

[0198] The second encoding module 24 is used to encode the current video frame as an I-frame if the current video frame is a scene transition frame, and to encode the current video frame as a P-frame if the current video frame is not a scene transition frame, so as to obtain the encoded frame of the current video frame.

[0199] The bitstream is obtained based on the encoded frame of the current video frame and the multiple encoded frames preceding the current video frame;

[0200] The sending module 25 is used to send a bitstream to the client so that the client can display a virtual game screen based on the bitstream.

[0201] Optionally, each of the multiple video frames includes an image composed of virtual game visuals.

[0202] Optionally, the second encoding module 24 is further configured to: if there is an I-frame among the M encoding frames preceding the current video frame, then encode the current video frame as a P-frame, or encode the current video frame according to the second encoding method and the preset encoding mode.

[0203] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 9 The encoding device shown can perform Figure 5 The corresponding method embodiments, and the foregoing and other operations and / or functions of each module in the encoding device are respectively for implementing Figure 5 For the sake of brevity, the corresponding processes in the method embodiments will not be described in detail here.

[0204] The encoding apparatus of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware encoding processor, or execution by a combination of hardware and software modules in the encoding processor. Optionally, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0205] Figure 10 This is a schematic block diagram of an electronic device provided in an embodiment of this application. The electronic device may be a server as described in the above method embodiments.

[0206] like Figure 10As shown, the electronic device may include:

[0207] The system includes a memory 210 and a processor 220. The memory 210 stores computer programs and transfers the program code to the processor 220. In other words, the processor 220 can retrieve and run the computer program from the memory 210 to implement the methods described in the embodiments of this application.

[0208] For example, the processor 220 can be used to execute the above-described method embodiments according to instructions in the computer program.

[0209] In some embodiments of this application, the processor 220 may include, but is not limited to:

[0210] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0211] In some embodiments of this application, the memory 210 includes, but is not limited to:

[0212] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0213] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 210 and executed by the processor 220 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0214] like Figure 10 As shown, the electronic device may also include:

[0215] Transceiver 230, which can be connected to processor 220 or memory 210.

[0216] The processor 220 can control the transceiver 230 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 230 may include a transmitter and a receiver. The transceiver 230 may further include antennas, and the number of antennas may be one or more.

[0217] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0218] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0219] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0220] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0221] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0222] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0223] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An encoding method, characterized in that, include: The current video frame in the video stream is pre-encoded according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames. If there is no I-frame among the M encoded frames preceding the current video frame, determine whether the current video frame is a scene transition frame based on the precoded frame and the target precoded frame. The scene transition frame is a video frame that has undergone a scene change relative to the previous video frame. The target precoded frame is obtained by precoding the video frame preceding the current video frame. The encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer. If the current video frame is a scene transition frame, the current video frame is encoded as an I-frame; if the current video frame is not a scene transition frame, the current video frame is encoded as a P-frame. The step of determining whether the current video frame is a scene transition frame based on the precoded frame and the target precoded frame includes: Based on the size of the precoded frame and the size of the target precoded frame, determine whether the current video frame is a scene transition frame.

2. The method according to claim 1, characterized in that, The method further includes: If there is an I-frame among the M encoded frames preceding the current video frame, then the current video frame is encoded as a P-frame, or the current video frame is encoded according to the second encoding method and the preset encoding mode.

3. The method according to claim 1, characterized in that, The step of precoding the current video frame according to the first encoding method to obtain the precoded frame of the current video frame includes: The current video frame is downsampled according to a preset sampling width and a preset sampling length to obtain a downsampled video frame; The downsampled video frame is pre-encoded according to the first encoding method to obtain the pre-encoded frame of the current video frame.

4. The method according to claim 1, characterized in that, If there are no I-frames in the M encoded frames preceding the current video frame, the method further includes: Count the number of encoded frames preceding the current video frame; When the number of encoded frames preceding the current video frame reaches M, it is confirmed whether there are any I-frames among the M encoded frames preceding the current video frame.

5. The method according to claim 1, characterized in that, M is 128.

6. The method according to claim 1, characterized in that, The step of determining whether the current video frame is a scene transition frame based on the size of the precoded frame and the size of the target precoded frame includes: Based on the number of bytes in the precoded frame and the number of bytes in the target precoded frame, determine whether the current video frame is a scene transition frame.

7. The method according to claim 6, characterized in that, Determining whether the current video frame is a scene transition frame based on the number of bytes in the precoded frame and the number of bytes in the target precoded frame includes: If the ratio of the number of bytes in the precoded frame to the number of bytes in the target precoded frame is greater than or equal to a preset threshold, then the current video frame is determined to be a scene switching frame. If the ratio of the number of bytes in the precoded frame to the number of bytes in the target precoded frame is less than the preset threshold, then the current video frame is determined not to be a scene transition frame.

8. The method according to claim 2, characterized in that, The preset encoding mode is the fixed quantization parameter CQP mode, and the pre-encoded encoding mode is the CQP mode.

9. The method according to claim 2, characterized in that, The second encoding method includes: Starting from the first video frame, I-frames are encoded according to a preset period, and subsequent video frames encoded as I-frames are encoded as P-frames.

10. The method according to claim 1 or 2, characterized in that, The method is applied to cloud gaming scenarios, and the method further includes: Send a decoding capability request to the terminal device; Receive a decoding capability response from the terminal device, the decoding capability response including: the decoding capability of the terminal device; The optimal decoding configuration is determined based on the decoding capability of the terminal device, the type of cloud game, and the current network status. The optimal decoding configuration is sent to the terminal device so that the terminal device can decode the video stream bitstream using the optimal decoding configuration.

11. A real-time communication method, characterized in that, include: The real-time generated video is captured to obtain a video stream; The current video frame in the video stream is pre-encoded according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames. If there is no I-frame among the M encoded frames preceding the current video frame, determine whether the current video frame is a scene transition frame based on the precoded frame and the target precoded frame. The scene transition frame is a video frame that has undergone a scene change relative to the previous video frame. The target precoded frame is obtained by precoding the video frame preceding the current video frame. The encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer. If the current video frame is a scene transition frame, the current video frame is encoded as an I-frame; if the current video frame is not a scene transition frame, the current video frame is encoded as a P-frame, thus obtaining the encoded frame of the current video frame. The bitstream is obtained based on the encoded frame of the current video frame and multiple encoded frames preceding the current video frame; The bitstream is sent to the client so that the client displays a virtual game screen based on the bitstream; The step of determining whether the current video frame is a scene transition frame based on the precoded frame and the target precoded frame includes: Based on the size of the precoded frame and the size of the target precoded frame, determine whether the current video frame is a scene transition frame.

12. The method according to claim 11, characterized in that, The video stream includes multiple video frames, each of which comprises an image composed of virtual game scenes.

13. The method according to claim 11, characterized in that, If there is an I-frame among the M encoded frames preceding the current video frame, then the current video frame is encoded as a P-frame, or the current video frame is encoded according to the second encoding method and the preset encoding mode.

14. An encoding device, characterized in that, include: The first encoding module is used to pre-encode the current video frame in the video stream according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames. The determining module is used to determine whether the current video frame is a scene transition frame based on the size of the precoded frame and the size of the target precoded frame when there is no I-frame in the M encoded frames preceding the current video frame. The scene transition frame is a video frame that has undergone a scene change relative to the previous video frame. The target precoded frame is obtained by precoding the video frame preceding the current video frame. The encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer. The second encoding module is used to encode the current video frame as an I-frame when the current video frame is a scene transition frame, and to encode the current video frame as a P-frame when the current video frame is not the scene transition frame.

15. A real-time communication device, characterized in that, include: The acquisition module is used to acquire video images from the real-time generated video to obtain a video stream; The first encoding module is used to pre-encode the current video frame in the video stream according to the first encoding method to obtain the pre-encoded frame of the current video frame. In the first encoding method, the first video frame in the video stream is pre-encoded as an I-frame, and the video frames after the first video frame are pre-encoded as P-frames. The determining module is configured to determine whether the current video frame is a scene transition frame if there is no I-frame among the M encoded frames preceding the current video frame, based on the size of the precoded frame and the size of the target precoded frame. The scene transition frame is a video frame that has undergone a scene change relative to the previous video frame. The target precoded frame is obtained by precoding the video frame preceding the current video frame. The encoded frame is obtained by encoding the video frame preceding the current video frame. M is a preset positive integer. The second encoding module is used to encode the current video frame as an I-frame if the current video frame is a scene switching frame, and to encode the current video frame as a P-frame if the current video frame is not a scene switching frame, thereby obtaining the encoded frame of the current video frame. The bitstream is obtained based on the encoded frame of the current video frame and multiple encoded frames preceding the current video frame; The sending module is used to send the bitstream to the client so that the client can display a virtual game screen based on the bitstream.

16. An electronic device, characterized in that, include: A processor and a memory, the memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform the method of any one of claims 1 to 10 or 11-13.

17. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 10 or 11-13.

Images