Video encoding and decoding method and related device

Abstract

The invention discloses a video encoding and decoding method and a related device, and belongs to the field of video compression. The method comprises the following steps: determining scene information corresponding to a first GOP, wherein the first GOP is a current GOP to be coded; determining a coding mode of the first GOP from a plurality of GOP coding modes based on the scene information corresponding to the first GOP; and according to the coding mode of the first GOP, coding the video frame included in the first GOP into a code stream. According to the invention, the corresponding GOP coding mode is selected according to the scene information corresponding to the GOP instead of adopting a unified coding mode for all GOPs, so that a suitable coding mode can be selected for each GOP, and the video coding quality and the optimal code stream can be considered.

Description

Technical Field

[0001] This application relates to the field of video compression, and in particular to a video encoding / decoding method and related apparatus. Background Technology

[0002] Video compression is a technique that utilizes redundant information in video to represent the original video with less data (video bitstream). Video compression can reduce the pressure on network bandwidth occupied by video storage and transmission. Video compression technology includes video encoding and decoding. Video encoding and decoding performance (reflecting video quality) and video encoding and decoding efficiency (reflecting time consumption) are factors that need to be considered in video compression technology. However, the encoding and decoding performance of some current video compression schemes is not good. Summary of the Invention

[0003] This application provides a video encoding / decoding method and related apparatus, which can select a suitable encoding mode for each GOP, thereby achieving a balance between video encoding quality and optimal bitrate. The technical solution is as follows:

[0004] In a first aspect, a video encoding method is provided, the method comprising: determining scene information corresponding to a first GOP, wherein the first GOP is a GOP to be encoded; determining an encoding mode for the first GOP from multiple GOP encoding modes based on the scene information; and encoding the video frames included in the first GOP into a bitstream according to the encoding mode of the first GOP.

[0005] In this application, for any GOP in the video to be encoded, the corresponding GOP encoding mode is selected based on the scene information corresponding to that GOP, instead of using a uniform encoding mode for all GOPs. In this way, a suitable encoding mode can be selected for each GOP, thereby balancing video encoding quality and optimal bitrate.

[0006] In one possible implementation, determining the scene information corresponding to the first GOP includes: determining the area ratio of the Region of Interest (ROI) of the first video frame included in the first GOP, wherein the ROI area ratio is the ratio of the area of ​​the ROI in the first video frame to the area of ​​the first video frame; determining the encoding mode of the first GOP from multiple GOP encoding modes based on the scene information includes: determining the encoding mode of the first GOP from the multiple GOP encoding modes based on the ROI area ratio.

[0007] The scene information corresponding to the first GOP can indicate whether the first GOP is in a moving or static scene. That is, whether the first GOP is in a moving scene, a static scene, or both. Moving and static scenes can be characterized by the area ratio of the region of interest (ROI). Therefore, determining the scene information corresponding to the first GOP includes determining the area ratio of the ROI of the first video frames included in the first GOP.

[0008] In one possible implementation, multiple GOP encoding patterns correspond to multiple area percentage ranges, and different GOP encoding patterns correspond to different area percentage ranges; the step of determining the encoding pattern of the first GOP from the multiple GOP encoding patterns based on the ROI area percentage includes: determining the GOP encoding pattern corresponding to the first area percentage range among the multiple area percentage ranges as the encoding pattern of the first GOP, where the first area percentage range is the area percentage range in which the ROI area percentage is located.

[0009] As an example, the multiple GOP encoding modes include DualP mode, SmartP mode, and NormalP mode. The area ratio range corresponding to DualP mode is [1 / 2, 1], the area ratio range corresponding to SmartP mode is [1 / 3, 1 / 2), and the area ratio range corresponding to NormalP mode is [0, 1 / 3]. If the ROI area ratio of the first video frame is within the area ratio range [1 / 2, 1], then the encoding mode of the first GOP is determined to be DualP mode. If the ROI area ratio of the first video frame is within the area ratio range [1 / 3, 1 / 2), then the encoding mode of the first GOP is determined to be SmartP mode. If the ROI area ratio of the first video frame is within the area ratio range [0, 1 / 3), then the encoding mode of the first GOP is determined to be NormalP mode.

[0010] DualP mode is suitable for motion scenes, SmartP mode is suitable for scenes with both static and dynamic elements, and NormalP mode is suitable for any scene. Alternatively, DualP mode can be understood as suitable for scenes with high bitrate requirements, SmartP mode for scenes with moderate bitrate requirements, and NormalP mode for scenes with any bitrate requirements. If the ROI area ratio of the first video frame is within the range [1 / 2, 1], it indicates that the ROI area in the first video frame is large, and the bitrate requirement for encoding the video frames in the first GOP is high. Therefore, the first GOP can be considered to be in a motion scene, and DualP mode is determined as the encoding mode for the first GOP. If the ROI area ratio of the first video frame is within the range [1 / 3, 1 / 2), it indicates that the ROI area in the first video frame is moderate, and the bitrate requirement for encoding the video frames in the first GOP is moderate. Therefore, the first GOP can be considered to be in a scene with both static and dynamic elements, and SmartP mode is determined as the encoding mode for the first GOP. If the ROI area ratio of the first video frame is within the area ratio range [0, 1 / 3), it indicates that the ROI area in the first video frame is small, and the bit rate requirement when encoding the video frames in the first GOP is low. Therefore, it can be considered that the first GOP is in a static scene. Moreover, for this scene, encoding can be achieved using only one reference frame without the need for multiple reference frames. Therefore, the NormalP mode is determined as the encoding mode of the first GOP.

[0011] In one possible implementation, multiple GOP encoding patterns correspond to multiple pattern selection parameters, with different GOP encoding patterns corresponding to different pattern selection parameters. These multiple pattern selection parameters also correspond to multiple area percentage ranges, with different pattern selection parameters corresponding to different area percentage ranges. The step of determining the encoding pattern of the first GOP from the multiple GOP encoding patterns based on the ROI area percentage includes: determining the pattern selection parameter corresponding to a first area percentage range among the multiple area percentage ranges as a first pattern selection parameter, where the first area percentage range is the area percentage range in which the ROI area percentage is located; and determining the GOP encoding pattern corresponding to the first pattern selection parameter as the encoding pattern of the first GOP.

[0012] As an example, the multiple GOP encoding modes include DualP mode, SmartP mode, and NormalP mode. The mode selection parameter corresponding to DualP mode is 2, the mode selection parameter corresponding to SmartP mode is 1, and the mode selection parameter corresponding to NormalP mode is 0. The area ratio range corresponding to mode selection parameter 2 is [1 / 2, 1], the area ratio range corresponding to mode selection parameter 1 is [1 / 3, 1 / 2), and the area ratio range corresponding to mode selection parameter 0 is [0, 1 / 3]. If the ROI area ratio of the first video frame is within the area ratio range [1 / 2, 1], then the first mode selection parameter is determined to be 2, and the encoding mode of the first GOP is determined to be DualP mode; if the ROI area ratio of the first video frame is within the area ratio range [1 / 3, 1 / 2), then the first mode selection parameter is determined to be 1, and the encoding mode of the first GOP is determined to be SmartP mode; if the ROI area ratio of the first video frame is within the area ratio range [0, 1 / 3), then the first mode selection parameter is determined to be 0, and the encoding mode of the first GOP is determined to be NormalP mode.

[0013] In one possible implementation, determining the region of interest (ROI) area ratio of the first video frame included in the first GOP includes: determining the ROI in the first video frame; determining the area of ​​the first video frame and the area of ​​the ROI; and determining the ratio of the area of ​​the ROI to the area of ​​the first video frame as the ROI area ratio of the first video frame.

[0014] In one possible implementation, determining the ROI in the first video frame includes: using the first video frame as input to a segmentation model to obtain a segmentation image output by the segmentation model, the segmentation image being used to segment the ROI and non-ROI in the first video frame; or, determining the type of each object in the first video frame, and determining the ROI in the first video frame based on the type of each object in the first video frame.

[0015] Determining the ROI in the first video frame using a segmentation model can also be called determining the ROI based on a machine learning model. Determining the ROI based on the type of each object in the first video frame can be called determining the ROI based on a content-aware segmentation algorithm. Determining the ROI using a machine learning model improves the efficiency of ROI determination, while determining the ROI using a content-aware segmentation algorithm avoids labeling the segmented images corresponding to the sample video frames and avoids training the initial network model.

[0016] In one possible implementation, determining the scene information corresponding to the first image group (GOP) further includes: determining the content complexity of the first video frame and / or the transmission bandwidth of the first GOP; determining the encoding mode of the first GOP from multiple GOP encoding modes based on the scene information further includes: determining the encoding mode of the first GOP from the multiple GOP encoding modes based on the ROI area ratio, the content complexity, and / or the transmission bandwidth.

[0017] The complexity of the content of a first video frame can be characterized by a complexity level. For example, if the environment in which each object in the first video frame is relatively complex, then the content complexity of the first video frame is determined to be level one. If the environment in which each object in the first video frame is relatively simple, then the content complexity of the first video frame is determined to be level two. If the environment in which some objects in the first video frame are relatively complex, and the environment in which other objects are relatively simple, then the content complexity of the first video frame is determined to be level three. Level one is higher than level two, and level two is higher than level three.

[0018] Generally, outdoor scenes have more complex environments, while indoor scenes have simpler environments. Therefore, by determining whether an object in the first video frame is in an outdoor or indoor scene, we can determine whether its environment is complex or simple. For example, if an object is in an outdoor scene, its environment is considered more complex; if it is in an indoor scene, its environment is considered more simple.

[0019] In one possible implementation, the first video frame is an I-frame included in the first GOP.

[0020] Since the first video frame of each GOP is an I-frame, and the I-frame is fully encoded, decoding does not require reliance on other video frames. Therefore, the I-frames included in the first GOP are used as the first video frame. The encoding mode of the first GOP is determined by the ROI area ratio in the first video frame. In this way, even if the GOP encoding mode indication information is not encoded in the bitstream, the encoding mode of the first GOP can be quickly determined after the reconstructed first video frame is decoded, thus improving the video decoding rate.

[0021] In one possible implementation, the method further includes: encoding first mode indication information into the bitstream, wherein the first mode indication information is used to indicate the encoding mode of the first GOP. Thus, for the decoding end, the encoding mode of the first GOP can be determined directly by parsing the first mode indication information from the bitstream, facilitating the decoding of the first GOP.

[0022] Secondly, a video decoding method is provided, the method comprising: determining the encoding mode of a first group of images (GOP), wherein the encoding mode of the first GOP is determined from multiple GOP encoding modes based on scene information corresponding to the first GOP, and the first GOP is the current GOP to be decoded; and parsing the bitstream according to the encoding mode of the first GOP to obtain the reconstructed video frames included in the first GOP.

[0023] In one possible implementation, determining the encoding mode of the first group of images (GOP) includes: parsing the bitstream to obtain a reconstructed first video frame, wherein the first video frame is an I-frame included in the first GOP; determining the region of interest (ROI) area ratio of the reconstructed first video frame, wherein the ROI area ratio is the ratio of the area of ​​the ROI in the reconstructed first video frame to the area of ​​the reconstructed first video frame; and determining the encoding mode of the first GOP from the plurality of GOP encoding modes based on the ROI area ratio.

[0024] Since the first video frame is an I-frame included in the first GOP, and the image of an I-frame is completely encoded into the bitstream, the reconstructed first video frame can be directly obtained by parsing the bitstream. Therefore, based on the ROI area ratio of the reconstructed first video frame, the encoding mode of the first GOP can be determined from multiple GOP encoding modes.

[0025] In one possible implementation, determining the encoding mode of the first GOP from the plurality of GOP encoding modes based on the ROI area ratio includes: determining the content complexity of the reconstructed first video frame; and determining the encoding mode of the first GOP from the plurality of GOP encoding modes based on the ROI area ratio and the content complexity.

[0026] In one possible implementation, determining the encoding mode of the first image group (GOP) includes: parsing the bitstream to obtain first mode indication information, the first mode indication information being used to indicate the encoding mode of the first GOP; and determining the encoding mode of the first GOP based on the first mode indication information.

[0027] In other words, the encoding mode indication information for the first GOP can be encoded into the bitstream, or it can be omitted. If the encoding mode indication information is encoded, the encoding mode of the first GOP can be directly determined by parsing the bitstream, without needing to determine the scene information of the first GOP to determine its encoding mode. This operation is simpler and more efficient. If the encoding mode indication information for the first GOP is not encoded into the bitstream, the scene information corresponding to the first GOP can be determined through the reconstructed first video frame. This reduces the amount of encoding mode indication information required in the bitstream.

[0028] Thirdly, a video encoding apparatus is provided, the apparatus having the functionality to implement the method behavior described in the first aspect. The encoding apparatus includes at least one module for implementing the method provided in the first aspect.

[0029] Fourthly, a video decoding apparatus is provided, the apparatus having the functionality to implement the method behavior described in the second aspect above. The decoding apparatus includes at least one module for implementing the method provided in the second aspect above.

[0030] Fifthly, an encoding device is provided, the encoding device comprising: a processor coupled to a memory for storing programs or instructions, wherein when the programs or instructions are executed by the processor, the encoding device performs the method described in the first aspect.

[0031] In a sixth aspect, a decoding device is provided, the decoding device comprising: a processor coupled to a memory for storing programs or instructions, wherein when the programs or instructions are executed by the processor, the decoding device performs the method described in the second aspect above.

[0032] In a seventh aspect, a coding and decoding system is provided, the coding and decoding system comprising the encoding device described in the fifth aspect and / or the decoding device described in the sixth aspect.

[0033] Eighthly, a computer-readable storage medium is provided, including program code that, when executed on a computer, causes the computer to perform the method described in the first aspect.

[0034] A ninth aspect provides a computer-readable storage medium including program code that, when executed on a computer, causes the computer to perform the method described in the second aspect above.

[0035] In a tenth aspect, a computer program product is provided, including instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect.

[0036] Eleventhly, a computer program product is provided, including instructions that, when executed on a computer, cause the computer to perform the method described in the second aspect above.

[0037] In a twelfth aspect, a computer-readable storage medium is provided, on which a bitstream obtained according to the method described in the first aspect is stored.

[0038] In a thirteenth aspect, a device for storing a bitstream is provided, comprising at least one storage medium and a communication interface; the communication interface is used to receive or transmit the bitstream; the at least one storage medium is used to store the bitstream; the bitstream is encoded by an encoder according to the method described in the first aspect above.

[0039] In a fourteenth aspect, a method for storing a bitstream is provided, comprising: receiving the bitstream via a communication interface; and storing the bitstream in one or more storage media, wherein the bitstream is encoded by an encoder according to the method described in the first aspect above.

[0040] In a fifteenth aspect, a system for distributing a bitstream is provided, comprising at least one storage medium and a video streaming device; the at least one storage medium is used to store the bitstream, which is encoded by an encoder according to the method described in the first aspect; the video streaming device is used to send the bitstream in the at least one storage medium to the decoder in response to a request from the decoder.

[0041] In a sixteenth aspect, a method for distributing a bitstream is provided, comprising: receiving a first request; in response to the first request, selecting a bitstream from at least one storage medium; and sending the bitstream to a destination device; wherein the at least one storage medium is used to store the bitstream, the bitstream being encoded by an encoder according to the method described in the first aspect.

[0042] In a seventeenth aspect, a system for processing bitstreams is provided, comprising an image source device, an encoder, one or more storage media, and a destination device; the image source device is used to provide image data; the encoder is used to acquire the image data from the image source device through an interface and encode the image data to obtain one or more bitstreams, the bitstreams being encoded by the encoder according to the method described in the first aspect; the encoder is used to store the one or more bitstreams in one or more storage media; or the encoder is used to encapsulate the one or more bitstreams to obtain a transmission bitstream; the encoder is used to transmit the transmission bitstream to the destination device via a communication link or communication network; the destination device is used to decapsulate the transmission bitstream to obtain the one or more bitstreams; and the destination device is used to decode the one or more bitstreams to obtain decoded data.

[0043] The technical effects achieved by the second to seventeenth aspects mentioned above are similar to those achieved by the corresponding technical means in the first or second aspects, and will not be repeated here. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of a GOP structure in an IPPP mode provided in an embodiment of this application;

[0045] Figure 2 This is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0046] Figure 3 This is a flowchart of a video encoding method provided in an embodiment of this application;

[0047] Figure 4 This is a schematic diagram illustrating the selection process of a GOP encoding mode provided in an embodiment of this application;

[0048] Figure 5 This is a flowchart of a video decoding method provided in an embodiment of this application;

[0049] Figure 6 This is a schematic diagram of the structure of a video encoding device provided in an embodiment of this application;

[0050] Figure 7 This is a schematic diagram of the structure of a video decoding device provided in an embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0052] Before providing a detailed explanation of the video encoding and decoding methods provided in the embodiments of this application, the terminology, application scenarios, and implementation environment involved in the embodiments of this application will be introduced first.

[0053] First, the terminology used in the embodiments of this application will be introduced.

[0054] 1. Group of pictures (GOP)

[0055] A Group of Pictures (GOP) comprises multiple images; a GOP is a group of consecutive frames. For video, a GOP consists of multiple video frames; a GOP is a video segment. In video coding, a long video segment is divided into multiple GOPs of equal length. Typically, a GOP includes I-frames and P-frames; in some cases, it also includes B-frames. A GOP includes one I-frame, and the I-frame is the first video frame of the GOP. That is, the length between two adjacent I-frames is the length of a GOP.

[0056] 2. I-frame, P-frame, B-frame

[0057] An I-frame, also known as an intraframe, is a keyframe that requires intra-frame predictive coding. It can be understood as follows: during encoding, the entire I-frame's image needs to be preserved; during decoding, only the data from that frame is needed. In other words, the entire I-frame's image needs to be encoded into the bitstream, and decoding can be performed without relying on other frames.

[0058] P-frames, also known as predictive frames, require inter-frame predictive coding using a forward reference frame; that is, P-frames are forward-predictively coded frames. In other words, during encoding, the differences between the P-frame and the reference frame are incorporated into the bitstream, and decoding relies on the reference frame and these differences to complete the decoding of the P-frame. For example, the reference frame for a P-frame might be an I-frame or another preceding P-frame.

[0059] B-frames, also known as bi-directional interpolated prediction frames, require inter-frame predictive coding using forward and backward reference frames; that is, B-frames are bi-directional predictive interpolated coded frames. In other words, during encoding, the differences between the B-frame and the forward and backward reference frames are incorporated into the bitstream, and during decoding, the forward and backward reference frames, along with these differences, are relied upon to decode the B-frame. For example, the reference frames for a B-frame are the most recent I-frame or P-frame preceding the B-frame, and the most recent P-frame following the B-frame.

[0060] It should be noted that, for the P-frames and B-frames mentioned above, the forward reference frame is the video frame preceding the current frame. This forward reference frame may or may not be adjacent to the current frame, and there may be one or more forward reference frames. Similarly, the backward reference frame is the video frame following the current frame. This backward video frame may or may not be adjacent to the current frame, and there may be one or more backward reference frames. This application does not limit the scope of this application.

[0061] 3. Region of Interest (ROI)

[0062] In image processing, people may be interested in a specific region of an image; this region is called the Region of Interest (ROI). In other words, the ROI is the area of ​​focus within the image. For example, for a video frame, the ROI could be the foreground element, such as a person, a vehicle, or a non-motorized vehicle.

[0063] 4. GOP encoding mode

[0064] The GOP encoding mode indicates the reference relationships among the video frames in a GOP, excluding I-frames. In other words, it specifies which video frames (excluding I-frames) serve as references to the other video frames in the GOP. These reference relationships indicate the structure of the GOP. Different GOP encoding modes correspond to different GOP structures. For example, GOP encoding modes include NormalP, SmartP, DualP, AdvSingleP, BiPredB, etc., which will not be listed here.

[0065] NormalP mode, also known as SingleP mode, is the most commonly used GOP encoding mode. Unless otherwise specified, the reference relationship in NormalP mode is simple: each P-frame references a forward reference frame. NormalP mode has no requirements for the application scenario and can be used in any scenario.

[0066] In SmartP mode, P-frames reference both the instantaneous decoding refresh (IDR) frame (i.e., the long-term reference frame) and the forward reference frame (i.e., the short-term reference frame), leveraging the temporal correlation between the two reference frames to improve coding compression performance. SmartP mode is primarily used in scenarios with both static and moving elements. For example, SmartP mode is applied in surveillance scenarios, characterized by fixed camera installations and both static and moving people and objects in the scene. In static areas, utilizing the temporal correlation between the long-term reference frame and the current frame can significantly reduce the bitrate and decrease breathing and trailing effects; in moving areas, the short-term reference frame is used for motion estimation.

[0067] In DualP mode, the P-frame references the two nearest forward reference frames, leveraging the temporal correlation of more reference frames to improve coding and compression performance. DualP mode is primarily used in motion scenarios with low latency requirements, and its compression performance is higher than that of NormalP mode. Since both reference frames in DualP mode use forward reference frames, there is no encoding or decoding latency.

[0068] AdvSingleP mode adds virtual I-frames to NormalP mode. A virtual I-frame is essentially a regular P-frame, but its reference frame is not the previous P-frame but an IDR frame. Like I-frames, IDR frames use intra-frame predictive coding; they are essentially the same thing. During encoding and decoding, to easily distinguish the first I-frame from other I-frames, the first I-frame is called the IDR frame. The GOP structure for AdvSingleP mode is basically the same as that for SmartP mode, the difference being that the IDR frame in AdvSingleP mode is generated internally by the encoder and is not displayed by the player.

[0069] In BiPredB mode, 1-3 B-frames can be inserted between two adjacent P-frames. The B-frames are not used as reference frames; instead, one forward reference frame and one backward reference frame are used. For motion scenes, bidirectional prediction can perform better motion estimation and supports weighted prediction, thus improving the encoder's compression performance.

[0070] Secondly, the application scenarios involved in the embodiments of this application will be introduced.

[0071] In the field of video encoding and decoding, GOPs are generally encoded in IPPP mode, i.e., NormalP mode. For example... Figure 1 As shown, I-frames are fully encoded and decoded independently, while P-frames depend on the previous frame for encoding and decoding. In other words, in related technologies, the NormalP mode is uniformly used for encoding a single video stream. However, different GOP structures are suitable for different scenarios. Although the GOP structure corresponding to the NormalP mode is applicable to any scenario, it cannot simultaneously achieve optimal video quality and bitrate. Therefore, this application provides a video encoding / decoding method that uses different GOP encoding modes based on different scenarios, thereby achieving optimal video quality and bitrate.

[0072] Finally, the implementation environment involved in the embodiments of this application will be described.

[0073] Please refer to Figure 2 , Figure 2 This is a schematic diagram of an implementation environment provided in an embodiment of this application. The implementation environment includes a source device 10, a destination device 20, a link 30, and a storage device 40. The source device 10 can generate encoded video. Therefore, the source device 10 can also be referred to as an encoding device or encoding end. The destination device 20 can decode the encoded video generated by the source device 10. Therefore, the destination device 20 can also be referred to as a decoding device or decoding end. The link 30 can receive the encoded video generated by the source device 10 and can transmit the encoded video to the destination device 20. The storage device 40 can receive the encoded video generated by the source device 10 and can store the encoded video. Under such conditions, the destination device 20 can directly obtain the encoded video from the storage device 40. Alternatively, the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded video generated by the source device 10. Under such conditions, the destination device 20 can stream or download the encoded video stored in the storage device 40.

[0074] Both source device 10 and destination device 20 may include one or more processors and memory coupled to the one or more processors. This memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or any other media that can be used to store desired program code in the form of computer-accessible instructions or data structures. For example, both source device 10 and destination device 20 may include mobile phones, smartphones, personal digital assistants (PDAs), wearable devices, pocket PCs (PPCs), tablets, smart car systems, smart TVs, smart speakers, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0075] Link 30 may include one or more media or devices capable of transmitting encoded video from source device 10 to destination device 20. In one possible implementation, link 30 may include one or more communication media enabling source device 10 to directly transmit encoded video to destination device 20 in real time. In this embodiment, source device 10 may modulate the encoded video based on a communication standard, such as a wireless communication protocol, and transmit the modulated video to destination device 20. The one or more communication media may include wireless and / or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, wide area network, or global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices facilitating communication from source device 10 to destination device 20, etc., which are not specifically limited in this embodiment.

[0076] In one possible implementation, storage device 40 can store the received encoded video sent by source device 10, and destination device 20 can directly retrieve the encoded video from storage device 40. Under such conditions, storage device 40 can include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, digital versatile discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video.

[0077] In one possible implementation, storage device 40 may correspond to a file server or another intermediate storage device capable of storing the encoded video generated by source device 10, and destination device 20 may stream or download the video stored on storage device 40. The file server may be any type of server capable of storing and sending the encoded video to destination device 20. In one possible implementation, the file server may include a web server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive, etc. Destination device 20 may acquire the encoded video via any standard data connection (including an Internet connection). Any standard data connection may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both suitable for acquiring the encoded video stored on a file server. The transmission of the encoded video from storage device 40 may be streaming, downloading, or a combination of both.

[0078] Figure 2 The implementation environment shown is only one possible implementation, and the technology of this application embodiment can be applied to more than just this. Figure 2 The source device 10 shown can encode video, and the destination device 20 can decode the encoded video. It can also be applied to other devices that can encode video and decode encoded video. This application embodiment does not specifically limit them.

[0079] exist Figure 2In the illustrated implementation environment, source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, the output interface 140 may include a modem / demodulator and / or a transmitter, wherein the transmitter may also be referred to as a transmitter. The data source 120 may include a video capture device (e.g., a camera, etc.), an archive containing previously captured video, a feed interface for receiving video from a video content provider, and / or a computer graphics system for generating video, or a combination of these sources of video.

[0080] Data source 120 can send video to encoder 100, which can encode the received video from data source 120 to obtain encoded video. The encoder can then send the encoded video to an output interface. In some embodiments, source device 10 directly sends the encoded video to destination device 20 via output interface 140. In other embodiments, the encoded video can also be stored on storage device 40 for later retrieval by destination device 20 for decoding and / or display.

[0081] exist Figure 2 In the illustrated implementation environment, destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive encoded video via link 30 and / or from storage device 40, and then send it to the decoder 200, which may decode the received encoded video to obtain decoded video. The decoder may send the decoded video to the display device 220. The display device 220 may be integrated with destination device 20 or may be external to destination device 20. Generally, the display device 220 displays the decoded video. The display device 220 may be any type of display device, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0082] although Figure 2Not shown, but in some respects, encoder 100 and decoder 200 may be integrated with each other and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data stream or separate data streams. In some embodiments, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as User Datagram Protocol (UDP), if applicable.

[0083] Encoder 100 and decoder 200 may each be any of the following circuits: one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the techniques of the embodiments of this application are implemented in part in software, the apparatus may store instructions for software in a suitable non-volatile computer-readable storage medium, and the instructions may be executed in hardware using one or more processors to implement the techniques of the embodiments of this application. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) may be considered as one or more processors. Each of encoder 100 and decoder 200 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (encoder-decoder) in the respective apparatus.

[0084] In this application embodiment, encoder 100 may be generally referred to as an apparatus that “signals” or “sends” certain information to, for example, decoder 200. The terms “signals” or “sends” may generally refer to the transmission of syntax elements and / or other data for decoding compressed video. This transmission may occur in real-time or near real-time. Alternatively, this communication may occur after a period of time, for example, during encoding when syntax elements are stored in a computer-readable storage medium in a encoded bitstream, and the decoding apparatus may then retrieve the syntax elements at any time after they have been stored in this medium.

[0085] It should be noted that the application scenarios and implementation environments described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, as the application scenarios and implementation environments evolve, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0086] The video encoding and decoding method provided in the embodiments of this application will now be explained in detail.

[0087] Please refer to Figure 3 , Figure 3 This is a flowchart of a video encoding method provided in an embodiment of this application, which is applied at the encoding end. The method includes the following steps.

[0088] Step 301: Determine the scene information corresponding to the first GOP. The first GOP is the GOP to be encoded.

[0089] The first GOP is the GOP currently being encoded in the video to be encoded. The video to be encoded may include multiple GOPs.

[0090] The scene information corresponding to the first GOP can indicate whether the first GOP is in a moving or static scene. That is, whether the first GOP is in a moving scene, a static scene, or both.

[0091] In some embodiments, dynamic and static scenes can be characterized by the area ratio of ROIs. In this case, determining the scene information corresponding to the first GOP includes: determining the area ratio of the ROIs of the first video frames included in the first GOP, where the area ratio of the ROIs in the first video frame is the ratio of the area of ​​the ROIs in the first video frame to the area of ​​the first video frame.

[0092] As an example, the Region of Interest (ROI) in the first video frame is determined, as are the area of ​​the first video frame and the area of ​​the ROI within the first video frame. The ratio of the area of ​​the ROI in the first video frame to the area of ​​the first video frame is defined as the ROI area percentage of the first video frame.

[0093] The Region of Interest (ROI) in the first video frame can be determined in several ways. Two of these methods will be described below.

[0094] The first method uses the first video frame as input to the segmentation model to obtain a segmentation image output by the segmentation model. This segmentation image is used to segment the ROI and non-ROI in the first video frame.

[0095] In other words, the segmentation model can distinguish between ROIs and non-ROIs in the first video frame. Therefore, after inputting the first video frame into the segmentation model, the segmentation model outputs a segmented image to separate the ROIs and non-ROIs in the first video frame. Thus, the ROIs in the first video frame can be determined from this segmented image.

[0096] The segmentation model is pre-trained based on sample video frames and their corresponding segmentation images. For example, multiple sample video frames and their corresponding segmentation images are obtained, with each image labeled with a region of interest (ROI) and a non-ROI. These sample video frames are used as input to an initial network model, and the corresponding segmentation images are used as output. This initial network model is then trained to obtain the segmentation model.

[0097] It should be noted that the segmentation model can be a neural network model or other network models, and this application embodiment does not limit it.

[0098] The second method involves determining the type of each object in the first video frame, and then determining the ROI in the first video frame based on the type of each object.

[0099] The type of each object in the first video frame can be obtained by detecting each object in the first video frame. This application embodiment does not limit the detection method.

[0100] The objects in the first video frame refer to people, objects, etc., that are present in the first video frame. Correspondingly, the types of objects in the first video frame refer to people, motor vehicles, non-motor vehicles, trees, overpasses, etc.

[0101] In some embodiments, after determining the type of each object in the first video frame, for each object in the first video frame, if the target type set includes the object's type, then the region where the object is located is determined to be a Region of Interest (ROI). If the target type set does not include the object's type, then the region where the object is located is determined to be a non-ROI. In this way, the ROI in the first video frame can be determined.

[0102] The target type set includes the types of each object in the ROI, and the target type set is predetermined. Therefore, the ROI in the first video frame can be determined through the target type set.

[0103] The first method described above can also be called determining the ROI in the first video frame based on a machine model, while the second method can also be called determining the ROI in the first video frame based on a content-aware segmentation algorithm. Determining the ROI through a machine model can improve the efficiency of ROI determination, while determining the ROI through a content-aware segmentation algorithm can avoid labeling the segmented images corresponding to the sample video frames and avoid training the initial network model.

[0104] It should be noted that the first video frame mentioned above can be any video frame in the first GOP, or it can be a specific video frame in the first GOP. For example, the first video frame is an I-frame included in the first GOP.

[0105] Since the first video frame of each GOP is an I-frame, and the I-frame is fully encoded, decoding does not require reliance on other video frames. Therefore, the I-frames included in the first GOP are used as the first video frame. The encoding mode of the first GOP is determined by the ROI area ratio in the first video frame. In this way, even if the GOP encoding mode indication information is not encoded in the bitstream, the encoding mode of the first GOP can be quickly determined after the reconstructed first video frame is decoded, thus improving the video decoding rate.

[0106] In other embodiments, the scene information corresponding to the first GOP may also indicate the content complexity of the first GOP, network transmission status, etc. Content complexity can be characterized by content complexity level, and network transmission status can be characterized by transmission bandwidth. Therefore, determining the scene information corresponding to the first image group GOP further includes: determining the content complexity of the first video frame and / or the transmission bandwidth of the first GOP.

[0107] The complexity of the content of a first video frame can be characterized by a complexity level. For example, if the environment in which each object in the first video frame is relatively complex, then the content complexity of the first video frame is determined to be level one. If the environment in which each object in the first video frame is relatively simple, then the content complexity of the first video frame is determined to be level two. If the environment in which some objects in the first video frame are relatively complex, and the environment in which other objects are relatively simple, then the content complexity of the first video frame is determined to be level three. Level one is higher than level two, and level two is higher than level three.

[0108] Generally, outdoor scenes have more complex environments, while indoor scenes have simpler environments. Therefore, by determining whether an object in the first video frame is in an outdoor or indoor scene, we can determine whether its environment is complex or simple. For example, if an object is in an outdoor scene, its environment is considered more complex; if it is in an indoor scene, its environment is considered more simple.

[0109] It should be noted that the above-described method for determining the content complexity of the first video frame is an example. In practical applications, the content complexity of the first video frame can also be determined in other ways, and this application does not limit this method.

[0110] Normally, the video to be encoded is captured by the video capture device and then transmitted to the encoding end. Therefore, the transmission bandwidth of the first GOP can refer to the bandwidth of the video capture device transmitting the first GOP to the encoding end.

[0111] Step 302: Based on the scene information corresponding to the first GOP, determine the encoding mode of the first GOP from multiple GOP encoding modes.

[0112] When the scene information corresponding to the first GOP indicates a dynamic or static scene, the encoding mode of the first GOP is determined from multiple GOP encoding modes based on the ROI area ratio of the first video frame.

[0113] There are several ways to determine the encoding mode of the first GOP from multiple GOP encoding modes based on the ROI area ratio of the first video frame. Two of them will be introduced below.

[0114] In the first approach, multiple GOP encoding modes correspond to multiple area percentage ranges, and different GOP encoding modes correspond to different area percentage ranges. In this case, the GOP encoding mode corresponding to the first area percentage range among the multiple area percentage ranges is determined as the encoding mode of the first GOP, and the first area percentage range is the area percentage range in which the ROI area percentage of the first video frame is located.

[0115] As an example, the multiple GOP encoding modes include DualP mode, SmartP mode, and NormalP mode. The area ratio range corresponding to DualP mode is [1 / 2, 1], the area ratio range corresponding to SmartP mode is [1 / 3, 1 / 2), and the area ratio range corresponding to NormalP mode is [0, 1 / 3]. If the ROI area ratio of the first video frame is within the area ratio range [1 / 2, 1], then the encoding mode of the first GOP is determined to be DualP mode. If the ROI area ratio of the first video frame is within the area ratio range [1 / 3, 1 / 2), then the encoding mode of the first GOP is determined to be SmartP mode. If the ROI area ratio of the first video frame is within the area ratio range [0, 1 / 3), then the encoding mode of the first GOP is determined to be NormalP mode.

[0116] Based on the above description, DualP mode is suitable for motion scenes, SmartP mode is suitable for scenes with both static and dynamic elements, and NormalP mode is suitable for any scene. This can also be understood as DualP mode being suitable for scenes with high bitrate requirements, SmartP mode being suitable for scenes with moderate bitrate requirements, and NormalP mode being suitable for scenes with any bitrate requirements. If the ROI area ratio of the first video frame is within the range [1 / 2, 1], it indicates that the ROI area in the first video frame is large, and the bitrate requirement for encoding the video frames in the first GOP is high. Therefore, the first GOP can be considered to be in a motion scene, and DualP mode is determined as the encoding mode for the first GOP. If the ROI area ratio of the first video frame is within the range [1 / 3, 1 / 2), it indicates that the ROI area in the first video frame is moderate, and the bitrate requirement for encoding the video frames in the first GOP is moderate. Therefore, the first GOP can be considered to be in a scene with both static and dynamic elements, and SmartP mode is determined as the encoding mode for the first GOP. If the ROI area ratio of the first video frame is within the area ratio range [0, 1 / 3), it indicates that the ROI area in the first video frame is small, and the bit rate requirement when encoding the video frames in the first GOP is low. Therefore, it can be considered that the first GOP is in a static scene. Moreover, for this scene, encoding can be achieved using only one reference frame without the need for multiple reference frames. Therefore, the NormalP mode is determined as the encoding mode of the first GOP.

[0117] For example, please refer to Figure 4 For the first video frame, a segmentation image corresponding to the first video frame is determined by the segmentation model. This segmentation image is used to segment the ROI (such as...). Figure 4 The area where the fox is located) and non-ROI (such as Figure 4 (The region excluding the fox). Based on this segmented image, the ROI area ratio of the first video frame is determined. Based on the ROI area ratio of the first video frame, the encoding mode of the first GOP is selected from DualP mode, SmartP mode, and NormalP mode.

[0118] It should be noted that the area percentage ranges corresponding to the DualP, SmartP, and NormalP modes shown above are merely examples. This application does not limit the area percentage ranges corresponding to these three modes, as long as the value within the area percentage range corresponding to the DualP mode is greater than the value within the area percentage range corresponding to the SmartP mode, and the value within the area percentage range corresponding to the SmartP mode is greater than the value within the area percentage range corresponding to the NormalP mode. Furthermore, the above description uses the DualP, SmartP, and NormalP modes as examples; these multiple GOP encoding modes can also include other encoding modes, and this application does not limit them.

[0119] The second approach involves multiple GOP encoding modes corresponding to multiple mode selection parameters. Different GOP encoding modes correspond to different mode selection parameters, and these multiple mode selection parameters correspond to multiple area proportion ranges. In this case, the mode selection parameter corresponding to the first area proportion range among the multiple area proportion ranges is determined as the first mode selection parameter. The first area proportion range is the area proportion range in which the ROI area proportion of the first video frame is located. The GOP encoding mode corresponding to the first mode selection parameter is determined as the encoding mode of the first GOP.

[0120] As an example, the multiple GOP encoding modes include DualP mode, SmartP mode, and NormalP mode. The mode selection parameter corresponding to DualP mode is 2, the mode selection parameter corresponding to SmartP mode is 1, and the mode selection parameter corresponding to NormalP mode is 0. The area ratio range corresponding to mode selection parameter 2 is [1 / 2, 1], the area ratio range corresponding to mode selection parameter 1 is [1 / 3, 1 / 2), and the area ratio range corresponding to mode selection parameter 0 is [0, 1 / 3]. If the ROI area ratio of the first video frame is within the area ratio range [1 / 2, 1], then the first mode selection parameter is determined to be 2, and the encoding mode of the first GOP is determined to be DualP mode; if the ROI area ratio of the first video frame is within the area ratio range [1 / 3, 1 / 2), then the first mode selection parameter is determined to be 1, and the encoding mode of the first GOP is determined to be SmartP mode; if the ROI area ratio of the first video frame is within the area ratio range [0, 1 / 3), then the first mode selection parameter is determined to be 0, and the encoding mode of the first GOP is determined to be NormalP mode.

[0121] Based on the above description, DualP mode is suitable for motion scenes, SmartP mode is suitable for scenes with both static and dynamic elements, and NormalP mode is suitable for any scene. This can also be understood as DualP mode being suitable for scenes with high bitrate requirements, SmartP mode being suitable for scenes with moderate bitrate requirements, and NormalP mode being suitable for scenes with any bitrate requirements. If the ROI area ratio of the first video frame is within the range [1 / 2, 1], it indicates that the ROI area in the first video frame is large, and the bitrate requirement for encoding the video frames in the first GOP is high. Therefore, the first GOP can be considered to be in a motion scene, and the first mode selection parameter is set to 2, thus determining DualP mode as the encoding mode for the first GOP. If the ROI area ratio of the first video frame is within the range [1 / 3, 1 / 2), it indicates that the ROI area in the first video frame is moderate, and the bitrate requirement for encoding the video frames in the first GOP is moderate. Therefore, the first GOP can be considered to be in a scene with both static and dynamic elements, and the first mode selection parameter is set to 1, thus determining SmartP mode as the encoding mode for the first GOP. If the ROI area ratio of the first video frame is within the area ratio range [0, 1 / 3), it indicates that the ROI area in the first video frame is small, and the bitrate requirement when encoding the video frames in the first GOP is low. Therefore, it can be assumed that the first GOP is in a static scene. Moreover, for this scene, encoding can be achieved using only one reference frame without the intervention of multiple reference frames. Therefore, the first mode selection parameter is determined to be 0, and the NormalP mode is determined as the encoding mode of the first GOP.

[0122] It should be noted that the mode selection parameters and area ratio ranges corresponding to the DualP, SmartP, and NormalP modes shown above are merely examples. This application does not limit the mode selection parameters and area ratio ranges corresponding to these three modes, as long as the value within the area ratio range corresponding to the mode selection parameter for the DualP mode is greater than the value within the area ratio range corresponding to the mode selection parameter for the SmartP mode, and the value within the area ratio range corresponding to the mode selection parameter for the SmartP mode is greater than the value within the area ratio range corresponding to the mode selection parameter for the NormalP mode. Furthermore, the above description uses DualP, SmartP, and NormalP modes as examples; these multiple GOP encoding modes can also include other encoding modes, which this application does not limit.

[0123] Given that the scene information corresponding to the first GOP also indicates the content complexity and network transmission status of the first GOP, the encoding mode of the first GOP is determined from multiple GOP encoding modes based on the ROI area ratio of the first video frame, as well as the content complexity and / or transmission bandwidth of the first GOP.

[0124] There are several ways to determine the encoding mode of the first GOP from multiple GOP encoding modes, based on the ROI area ratio of the first video frame, the content complexity of the first GOP, and / or the transmission bandwidth. One of these methods will be introduced below. In this method, the scene information corresponding to the first GOP also indicates the content complexity and transmission bandwidth of the first GOP, and the content complexity of the first GOP is represented by a content complexity level.

[0125] Multiple GOP encoding modes correspond to multiple area percentage ranges, multiple content complexity levels, and multiple transmission bandwidth ranges. Different GOP encoding modes correspond to different area percentage ranges, different content complexity levels, and different transmission bandwidth ranges. Therefore, the GOP encoding mode corresponding to the area percentage range of the ROI of the first video frame within these multiple area percentage ranges is determined; the GOP encoding mode corresponding to the content complexity level of the first GOP within these multiple content complexity levels is determined; and the GOP encoding mode corresponding to the transmission bandwidth range of the first GOP within these multiple transmission bandwidth ranges is determined. Based on these three GOP encoding modes, the encoding mode of the first GOP is determined.

[0126] In some embodiments, the three GOP encoding patterns may be the same or different. In this case, the GOP encoding pattern that appears most frequently among the three encoding patterns can be determined as the encoding pattern of the first GOP. Of course, the encoding pattern of the first GOP can also be determined from the three GOP encoding patterns in other ways.

[0127] As an example, the multiple GOP encoding modes include DualP mode, SmartP mode, and NormalP mode. The area ratio range for DualP mode is [1 / 2, 1], for SmartP mode it is [1 / 3, 1 / 2), and for NormalP mode it is [0, 1 / 3]. The content complexity level for DualP mode is level one, for SmartP mode it is level two, and for NormalP mode it is level three. Level one is higher than level two, and level two is higher than level three. The transmission bandwidth range for DualP mode is the first range, for SmartP mode it is the second range, and for NormalP mode it is the third range. The transmission bandwidth within the first range is greater than the transmission bandwidth within the second range, and the transmission bandwidth within the second range is greater than the transmission bandwidth within the third range. If the ROI area ratio of the first video is within the area ratio range [1 / 2, 1], the content complexity level of the first GOP is the first level, and the transmission bandwidth range of the first GOP is the first range, then the encoding mode of the first GOP is determined to be DualP mode.

[0128] Based on the above description, DualP mode is suitable for motion scenes, SmartP mode is suitable for scenes with both static and dynamic elements, and NormalP mode is suitable for any scene. This can also be understood as DualP mode being suitable for scenes with high bitrate requirements, SmartP mode being suitable for scenes with moderate bitrate requirements, and NormalP mode being suitable for scenes with any bitrate requirements. If the content complexity level of the first GOP is level one (i.e., a high content complexity level), it indicates that the bitrate requirement for encoding the video frames in the first GOP is high; therefore, DualP mode is determined as the encoding mode corresponding to the content complexity level of the first GOP. If the content complexity level of the first GOP is level two (i.e., a moderate content complexity level), it indicates that the bitrate requirement for encoding the video frames in the first GOP is moderate; therefore, SmartP mode is determined as the encoding mode corresponding to the content complexity level of the first GOP. If the content complexity level of the first GOP is level three (i.e., low content complexity), it indicates a low bitrate requirement when encoding the video frames in the first GOP. Furthermore, in this scenario, encoding can be achieved using only one reference frame, eliminating the need for multiple reference frames. Therefore, the NormalP mode is determined as the encoding mode corresponding to the content complexity level of the first GOP. Similarly, if the transmission bandwidth of the first GOP falls within the first range, it indicates that the video frames in the first GOP transmit a large amount of image information, thus requiring a higher bitrate. Therefore, the DualP mode is determined as the GOP encoding mode corresponding to the transmission bandwidth range of the first GOP. If the transmission bandwidth of the first GOP falls within the second range, it indicates that the video frames in the first GOP transmit a moderate amount of image information, requiring a moderate bitrate. Therefore, the SmartP mode is determined as the GOP encoding mode corresponding to the transmission bandwidth range of the first GOP. If the transmission bandwidth of the first GOP falls within the third range, it indicates that the video frames in the first GOP transmit less image information. In this case, the bitrate requirement for encoding is also low. Moreover, for this scenario, encoding can be achieved using only one reference frame without the need for multiple reference frames. Therefore, the NormalP mode is determined as the GOP encoding mode corresponding to the transmission bandwidth range of the first GOP.

[0129] It should be noted that the above description uses DualP, SmartP, and NormalP modes as examples. These multiple GOP encoding modes can also include other encoding modes, and this application embodiment does not limit this. Furthermore, the method of determining the encoding mode of the first GOP based on the ROI area ratio and content complexity of the first video frame is similar to the above, as is the method of determining the encoding mode of the first GOP based on the ROI area ratio and transmission bandwidth of the first video frame, and will not be discussed further here. Moreover, when the scene information corresponding to the first GOP also indicates the content complexity and transmission bandwidth of the first GOP, a mode selection parameter can also be introduced. The detailed implementation process is similar to the second method described above, and will not be repeated here.

[0130] Step 303: Encode the video frames included in the first GOP into the bitstream according to the encoding mode of the first GOP.

[0131] Based on the above description, the GOP encoding mode indicates the reference relationship of each video frame in the GOP, excluding I-frames. Therefore, after determining the encoding mode of the first GOP, for each I-frame in the first GOP, the I-frame is encoded into the bitstream. For each video frame in the first GOP, excluding I-frames, the reference frame of that video frame is determined based on the reference relationship of the video frames indicated by the encoding mode of the first GOP, and the video frame is encoded into the bitstream based on the reference frame.

[0132] In addition to I-frames, the first GOP may include P-frames and B-frames. The reference frames for P-frames and B-frames may differ depending on the GOP coding mode. For P-frames, after determining the reference frame, they can be encoded into the bitstream using inter-frame predictive coding. For B-frames, after determining the reference frame, they can be encoded into the bitstream using bidirectional predictive interpolation coding.

[0133] In some embodiments, first mode indication information can also be encoded into the bitstream. The first mode indication information is used to indicate the encoding mode of the first GOP. In this way, the decoding end can directly parse the first mode indication information from the bitstream to know the encoding mode of the first GOP, which facilitates the decoding of the first GOP.

[0134] It should be noted that the above describes the encoding mode for selecting the first GOP based on the scene information corresponding to the first GOP. In other embodiments, the length of the first GOP (i.e., the number of video frames included in the first GOP) can also be adjusted based on the scene information corresponding to the first GOP. For example, multiple GOP lengths correspond to multiple scene information, and a GOP length that matches the scene information corresponding to the first GOP can be selected, thereby adjusting the length of the first GOP according to the selected GOP length.

[0135] In addition, during video encoding, the average bitrate (ABR) technique can be used to control the bitrate of video encoding to ensure transmission quality under different network conditions. In some embodiments, the quantization parameter (QP) of the first group of frames (GOPs) is adjusted based on the ROI area ratio of the first video frame, and then the encoding of each video frame in the first GOP is implemented according to the QP of the first GOP. For example, the ROI area ratio of the first video frame is inversely proportional to the QP of the first GOP; that is, the larger the ROI area ratio of the first video frame, the smaller the QP of the first GOP, resulting in a higher bitrate allocated to the first GOP; conversely, the smaller the ROI area ratio of the first video frame, the larger the QP of the first GOP, resulting in a lower bitrate allocated to the first GOP.

[0136] Based on the above description, each GOP can select its corresponding GOP encoding mode according to the scene information; that is, different GOPs may have different encoding modes. For example, for the second GOP included in the video to be encoded, the scene information corresponding to the second GOP can be determined. The second GOP is the GOP to be encoded in the video other than the first GOP. Based on the scene information corresponding to the second GOP, the encoding mode of the second GOP is determined from multiple GOP encoding modes, and the video frames included in the second GOP are encoded into the bitstream according to the second GOP's encoding mode. The method for determining the scene information corresponding to the second GOP and the encoding mode of the second GOP is similar to that of the first GOP. For detailed implementation, please refer to the description above; it will not be repeated here.

[0137] In summary, in this embodiment of the application, for any GOP in the video to be encoded, the corresponding GOP encoding mode can be selected based on the scene information corresponding to the GOP, instead of using a uniform encoding mode for all GOPs. In this way, a suitable encoding mode can be selected for each GOP, thereby balancing video encoding quality and optimal bitrate.

[0138] Please refer to Figure 5 , Figure 5 This is a flowchart of a video decoding method provided in an embodiment of this application, which is applied at the decoding end. The method includes the following steps.

[0139] Step 501: Determine the encoding mode of the first GOP. The encoding mode of the first GOP is determined from multiple GOP encoding modes based on the scene information corresponding to the first GOP. The first GOP is the GOP to be decoded.

[0140] In some embodiments, the bitstream is parsed to obtain a reconstructed first video frame, the first video frame being an I-frame included in a first GOP; the ROI area ratio of the reconstructed first video frame is determined, the ROI area ratio being the ratio of the area of ​​the ROI in the reconstructed first video frame to the area of ​​the reconstructed first video frame; and based on the ROI area ratio of the reconstructed first video frame, the encoding mode of the first GOP is determined from multiple GOP encoding modes.

[0141] Since the first video frame is an I-frame included in the first GOP, and the image of an I-frame is completely encoded into the bitstream, the reconstructed first video frame can be directly obtained by parsing the bitstream. Therefore, based on the ROI area ratio of the reconstructed first video frame, the encoding mode of the first GOP can be determined from multiple GOP encoding modes.

[0142] Based on the preceding description, when the scene information corresponding to the first GOP indicates a dynamic or static scene, the encoding mode of the first GOP can be directly determined from multiple GOP encoding modes based on the ROI area ratio of the reconstructed first video frame. The detailed implementation process for determining the encoding mode of the first GOP is similar to that at the encoding end and will not be elaborated here.

[0143] When the scene information corresponding to the first GOP also indicates the content complexity of the first GOP, for the decoding end, determining the encoding mode of the first GOP from multiple GOP encoding modes based on the ROI area ratio of the reconstructed first video frame includes: determining the content complexity of the reconstructed first video frame, and determining the encoding mode of the first GOP from multiple GOP encoding modes based on the ROI area ratio and the content complexity of the first video frame.

[0144] The method for determining the encoding mode of the first GOP from multiple GOP encoding modes based on the ROI area ratio and content complexity of the first video frame is similar to that of the encoding end, and will not be elaborated here.

[0145] In other embodiments, the encoding end also encodes the first mode indication information into the bitstream. In this case, the bitstream can also be parsed to obtain the first mode indication information. The first mode indication information is used to indicate the encoding mode of the first GOP, and the encoding mode of the first GOP is determined based on the first mode indication information.

[0146] In other words, the encoding mode indication information for the first GOP can be encoded into the bitstream, or it can be omitted. If the encoding mode indication information is encoded, the encoding mode of the first GOP can be directly determined by parsing the bitstream, without needing to determine the scene information of the first GOP to determine its encoding mode. This operation is simpler and more efficient. If the encoding mode indication information for the first GOP is not encoded into the bitstream, the scene information corresponding to the first GOP can be determined through the reconstructed first video frame. This reduces the amount of encoding mode indication information required in the bitstream.

[0147] Step 502: Parse the bitstream according to the encoding mode of the first GOP to obtain the reconstructed video frames included in the first GOP.

[0148] Based on the above description, the GOP encoding mode indicates the reference relationship of each video frame in the GOP, excluding I-frames. Therefore, after determining the encoding mode of the first GOP, for each video frame in the first GOP, excluding I-frames, the reference frame of that video frame is determined based on the reference relationship of the video frames indicated by the encoding mode of the first GOP, and the reconstructed video frame is obtained by parsing the bitrate based on the reference frame of that video frame.

[0149] In summary, in this embodiment of the application, for any GOP in the video to be encoded, the corresponding GOP encoding mode can be selected based on the scene information corresponding to the GOP, instead of using a uniform encoding mode for all GOPs. In this way, a suitable encoding mode can be selected for each GOP, thereby balancing video encoding quality and optimal bitrate.

[0150] Figure 6 This is a schematic diagram of the structure of a video encoding device provided in an embodiment of this application. The device can be implemented as part or all of an encoding end device by software, hardware, or a combination of both. This device can be... Figure 2 The source device shown, or the encoder within the source device. See also... Figure 6 The device includes: a scene determination module 601, an encoding mode determination module 602, and a GOP encoding module 603.

[0151] Scene determination module 601 is used to determine the scene information corresponding to the first image group GOP, where the first GOP is the GOP to be encoded.

[0152] The encoding mode determination module 602 is used to determine the encoding mode of the first GOP from multiple GOP encoding modes based on scene information;

[0153] GOP encoding module 603 is used to encode the video frames included in the first GOP into the bitstream according to the encoding mode of the first GOP.

[0154] In one possible implementation, the scene determination module 601 is specifically used for:

[0155] Determine the area percentage of the Region of Interest (ROI) of the first video frame included in the first GOP. The ROI area percentage is the ratio of the area of ​​the ROI in the first video frame to the area of ​​the first video frame.

[0156] The encoding mode determination module is specifically used for:

[0157] Based on the ROI area ratio, the encoding pattern of the first GOP is determined from multiple GOP encoding patterns.

[0158] In one possible implementation, multiple GOP encoding patterns correspond to multiple area percentage ranges, and different GOP encoding patterns correspond to different area percentage ranges.

[0159] The encoding mode determination module 602 is specifically used for:

[0160] The GOP encoding pattern corresponding to the first area percentage range among multiple area percentage ranges is determined as the encoding pattern of the first GOP, where the first area percentage range is the area percentage range in which the ROI area percentage is located.

[0161] In one possible implementation, multiple GOP encoding modes correspond to multiple mode selection parameters. Different GOP encoding modes correspond to different mode selection parameters. Multiple mode selection parameters correspond to multiple area proportion ranges. Different mode selection parameters correspond to different area proportion ranges.

[0162] The encoding mode determination module 602 is specifically used for:

[0163] The mode selection parameter corresponding to the first area proportion range among multiple area proportion ranges is determined as the first mode selection parameter, and the first area proportion range is the area proportion range in which the ROI area proportion is located.

[0164] The GOP encoding mode corresponding to the first mode selection parameter is determined as the encoding mode of the first GOP.

[0165] In one possible implementation, the encoding mode determination module 602 is specifically used for:

[0166] Determine the ROI in the first video frame;

[0167] Determine the area of ​​the first video frame and the area of ​​the Region of Interest (ROI);

[0168] The ratio of the area of ​​the ROI to the area of ​​the first video frame is defined as the ROI area percentage of the first video frame.

[0169] In one possible implementation, the encoding mode determination module 602 is specifically used for:

[0170] The first video frame is used as input to the segmentation model to obtain a segmented image output by the model. This segmented image is used to separate the Regions of Interest (ROIs) and non-ROIs in the first video frame; or

[0171] Determine the type of each object in the first video frame, and based on the type of each object in the first video frame, determine the ROI in the first video frame.

[0172] In one possible implementation, the scene determination module 601 is further used for:

[0173] Determine the content complexity of the first video frame and / or the transmission bandwidth of the first GOP;

[0174] The encoding mode determination module is also used for:

[0175] Based on the ROI area ratio, content complexity, and / or transmission bandwidth, the encoding mode of the first GOP is determined from multiple GOP encoding modes.

[0176] In one possible implementation, the first video frame is an I-frame included in the first GOP.

[0177] In one possible implementation, the device further includes:

[0178] The instruction information encoding module is used to encode the first mode instruction information into the bitstream. The first mode instruction information is used to indicate the encoding mode of the first GOP.

[0179] In summary, in this embodiment of the application, for any GOP in the video to be encoded, the corresponding GOP encoding mode can be selected based on the scene information corresponding to the GOP, instead of using a uniform encoding mode for all GOPs. In this way, a suitable encoding mode can be selected for each GOP, thereby balancing video encoding quality and optimal bitrate.

[0180] It should be noted that the video encoding device provided in the above embodiments is only illustrated by the division of the above functional modules during video encoding. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the video encoding device and the video encoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0181] Figure 7This is a schematic diagram of the structure of a video decoding device provided in an embodiment of this application. This device can be implemented as part or all of a decoding end device by software, hardware, or a combination of both. This device can be... Figure 2 The target device or the decoder within the target device shown. See also Figure 7 The device includes an encoding pattern determination module 701 and a GOP reconstruction module 702.

[0182] The encoding mode determination module 701 is used to determine the encoding mode of the first image group GOP. The encoding mode of the first GOP is determined from multiple GOP encoding modes based on the scene information corresponding to the first GOP. The first GOP is the GOP to be decoded at the moment.

[0183] GOP reconstruction module 702 is used to parse the bitstream according to the encoding mode of the first GOP to obtain the reconstructed video frames included in the first GOP.

[0184] In one possible implementation, the encoding mode determination module 701 is specifically used for:

[0185] The bitstream is parsed to obtain the reconstructed first video frame, which is the I-frame included in the first GOP;

[0186] Determine the area percentage of the Region of Interest (ROI) in the reconstructed first video frame. The ROI area percentage is the ratio of the area of ​​the ROI in the reconstructed first video frame to the total area of ​​the reconstructed first video frame.

[0187] Based on the ROI area ratio, the encoding pattern of the first GOP is determined from multiple GOP encoding patterns.

[0188] In one possible implementation, the encoding mode determination module 701 is specifically used for:

[0189] Determine the content complexity of the first reconstructed video frame;

[0190] Based on the ROI area ratio and content complexity, the encoding pattern of the first GOP is determined from multiple GOP encoding patterns.

[0191] In one possible implementation, the encoding mode determination module 701 is specifically used for:

[0192] The bitstream is parsed to obtain the first mode indication information, which is used to indicate the encoding mode of the first GOP.

[0193] The encoding mode of the first GOP is determined based on the first mode indication information.

[0194] In summary, in this embodiment of the application, for any GOP in the video to be encoded, the corresponding GOP encoding mode can be selected based on the scene information corresponding to the GOP, instead of using a uniform encoding mode for all GOPs. In this way, a suitable encoding mode can be selected for each GOP, thereby balancing video encoding quality and optimal bitrate.

[0195] It should be noted that the video decoding device provided in the above embodiments is only illustrated by the division of the above functional modules during video decoding. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the video decoding device and the video decoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0196] This application embodiment also provides an encoding device, the encoding device including: a processor, the processor being coupled to a memory, the memory being used to store programs or instructions, and when the program or instructions are executed by the processor, the encoding device performs the above-described encoding method.

[0197] This application embodiment also provides a decoding device, the decoding device including: a processor, the processor being coupled to a memory, the memory being used to store programs or instructions, and when the program or instructions are executed by the processor, the decoding device performs the above-described decoding method.

[0198] This application also provides an encoding / decoding system, which includes the encoding device and / or the decoding device.

[0199] This application also provides a computer-readable storage medium including program code, which, when run on a computer, causes the computer to execute the above-described encoding method.

[0200] This application also provides a computer-readable storage medium including program code, which, when run on a computer, causes the computer to perform the above-described decoding method.

[0201] This application also provides a computer program product, including instructions that, when executed on a computer, cause the computer to perform the above-described encoding method.

[0202] This application also provides a computer program product, including instructions that, when executed on a computer, cause the computer to perform the above-described decoding method.

[0203] This application also provides a computer-readable storage medium storing a bitstream obtained according to the above-described encoding method.

[0204] This application also provides a device for storing a bitstream, including at least one storage medium and a communication interface; the communication interface is used to receive or send the bitstream; the at least one storage medium is used to store the bitstream; the bitstream is encoded by an encoder according to the above-described encoding method.

[0205] This application also provides a method for storing a bitstream, comprising: receiving a bitstream through a communication interface; storing the bitstream in one or more storage media, wherein the bitstream is encoded by an encoder according to the above-described encoding method.

[0206] This application also provides a system for distributing bitstreams, including at least one storage medium and a video streaming device; the at least one storage medium is used to store the bitstream, which is encoded by an encoder according to the above-described encoding method; the video streaming device is used to send the bitstream in the at least one storage medium to the decoder in response to a request from the decoder.

[0207] This application embodiment also provides a method for distributing a bitstream, comprising: receiving a first request; in response to the first request, selecting a bitstream from at least one storage medium; sending the bitstream to a destination device; wherein the at least one storage medium is used to store the bitstream, and the bitstream is encoded by an encoder according to the above-described encoding method.

[0208] This application also provides a system for processing bitstreams, including an image source device, an encoder, one or more storage media, and a destination device; the image source device is used to provide image data; the encoder is used to acquire the image data from the image source device through an interface and encode the image data to obtain one or more bitstreams, wherein the bitstreams are encoded by the encoder according to the above-described encoding method; the encoder is used to store the one or more bitstreams in one or more storage media; or the encoder is used to encapsulate the one or more bitstreams to obtain a transmission bitstream; the encoder is used to transmit the transmission bitstream to the destination device through a communication link or communication network; the destination device is used to decapsulate the transmission bitstream to obtain the one or more bitstreams; the destination device is used to decode the one or more bitstreams to obtain decoded data.

[0209] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0210] It should be understood that "multiple" as mentioned herein refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., do not necessarily imply that they are different.

[0211] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0212] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A video encoding method, characterized in that, The method includes: Determine the scene information corresponding to the first image group GOP, where the first GOP is the GOP to be encoded. Based on the scenario information, the encoding mode of the first GOP is determined from multiple GOP encoding modes; According to the encoding mode of the first GOP, the video frames included in the first GOP are encoded into the bitstream.

2. The method as described in claim 1, characterized in that, The determination of the scene information corresponding to the first image group (GOP) includes: Determine the area ratio of the region of interest (ROI) of the first video frame included in the first GOP, wherein the ROI area ratio is the ratio of the area of ​​the ROI in the first video frame to the area of ​​the first video frame. Determining the encoding mode of the first GOP from multiple GOP encoding modes based on the scene information includes: Based on the ROI area ratio, the encoding pattern of the first GOP is determined from the plurality of GOP encoding patterns.

3. The method as described in claim 2, characterized in that, The multiple GOP encoding patterns correspond to multiple area percentage ranges, and different GOP encoding patterns correspond to different area percentage ranges. The step of determining the encoding pattern of the first GOP from the multiple GOP encoding patterns based on the ROI area ratio includes: The GOP encoding mode corresponding to the first area percentage range among the multiple area percentage ranges is determined as the encoding mode of the first GOP, and the first area percentage range is the area percentage range in which the ROI area percentage is located.

4. The method as described in claim 2, characterized in that, The multiple GOP encoding modes correspond to multiple mode selection parameters. Different GOP encoding modes correspond to different mode selection parameters. The multiple mode selection parameters correspond to multiple area ratio ranges. Different mode selection parameters correspond to different area ratio ranges. The step of determining the encoding pattern of the first GOP from the multiple GOP encoding patterns based on the ROI area ratio includes: The mode selection parameter corresponding to the first area proportion range among the multiple area proportion ranges is determined as the first mode selection parameter, and the first area proportion range is the area proportion range in which the ROI area proportion is located. The GOP encoding mode corresponding to the first mode selection parameter is determined as the encoding mode of the first GOP.

5. The method according to any one of claims 2-4, characterized in that, Determining the area ratio of the Region of Interest (ROI) of the first video frame included in the first GOP includes: Determine the ROI in the first video frame; Determine the area of ​​the first video frame and the area of ​​the ROI; The ratio of the area of ​​the ROI to the area of ​​the first video frame is determined as the ROI area ratio of the first video frame.

6. The method as described in claim 5, characterized in that, Determining the ROI in the first video frame includes: The first video frame is used as input to a segmentation model to obtain a segmented image output by the model. This segmented image is used to segment the Region of Interest (ROI) and non-ROI in the first video frame; or Determine the type of each object in the first video frame, and based on the type of each object in the first video frame, determine the ROI in the first video frame.

7. The method according to any one of claims 2-6, characterized in that, The step of determining the scene information corresponding to the first image group (GOP) further includes: Determine the content complexity of the first video frame and / or the transmission bandwidth of the first GOP; The step of determining the encoding mode of the first GOP from multiple GOP encoding modes based on the scene information further includes: Based on the ROI area ratio, the content complexity, and / or the transmission bandwidth, the encoding mode of the first GOP is determined from the plurality of GOP encoding modes.

8. The method according to any one of claims 2-7, characterized in that, The first video frame is an I-frame included in the first GOP.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: The first mode indication information is encoded into the bitstream, and the first mode indication information is used to indicate the encoding mode of the first GOP.

10. A video decoding method, characterized in that, The method includes: The encoding mode of the first image group (GOP) is determined. The encoding mode of the first GOP is determined from multiple GOP encoding modes based on the scene information corresponding to the first GOP. The first GOP is the GOP to be decoded. The bitstream is parsed according to the encoding mode of the first GOP to obtain the reconstructed video frames included in the first GOP.

11. The method as described in claim 10, characterized in that, Determining the encoding mode of the first image group (GOP) includes: The bitstream is parsed to obtain the reconstructed first video frame, which is an I-frame included in the first GOP. The area ratio of the region of interest (ROI) in the reconstructed first video frame is determined, where the ROI area ratio is the ratio of the area of ​​the ROI in the reconstructed first video frame to the area of ​​the reconstructed first video frame. Based on the ROI area ratio, the encoding pattern of the first GOP is determined from the plurality of GOP encoding patterns.

12. The method as described in claim 11, characterized in that, The step of determining the encoding pattern of the first GOP from the multiple GOP encoding patterns based on the ROI area ratio includes: Determine the content complexity of the reconstructed first video frame; Based on the ROI area ratio and the content complexity, the encoding pattern of the first GOP is determined from the multiple GOP encoding patterns.

13. The method as described in claim 10, characterized in that, Determining the encoding mode of the first image group (GOP) includes: The bitstream is parsed to obtain first mode indication information, which is used to indicate the encoding mode of the first GOP. The encoding mode of the first GOP is determined based on the first mode indication information.

14. A video encoding device, characterized in that, The device includes: The scene determination module is used to determine the scene information corresponding to the first image group GOP, where the first GOP is the GOP to be encoded. The encoding mode determination module is used to determine the encoding mode of the first GOP from multiple GOP encoding modes based on the scene information. The GOP encoding module is used to encode the video frames included in the first GOP into the bitstream according to the encoding mode of the first GOP.

15. The apparatus as claimed in claim 14, characterized in that, The scene determination module is specifically used for: Determine the area ratio of the region of interest (ROI) of the first video frame included in the first GOP, wherein the ROI area ratio is the ratio of the area of ​​the ROI in the first video frame to the area of ​​the first video frame. The encoding mode determination module is specifically used for: Based on the ROI area ratio, the encoding pattern of the first GOP is determined from the plurality of GOP encoding patterns.

16. The apparatus as claimed in claim 15, characterized in that, The multiple GOP encoding patterns correspond to multiple area percentage ranges, and different GOP encoding patterns correspond to different area percentage ranges. The encoding mode determination module is specifically used for: The GOP encoding mode corresponding to the first area percentage range among the multiple area percentage ranges is determined as the encoding mode of the first GOP, and the first area percentage range is the area percentage range in which the ROI area percentage is located.

17. The apparatus as claimed in claim 15, characterized in that, The multiple GOP encoding modes correspond to multiple mode selection parameters. Different GOP encoding modes correspond to different mode selection parameters. The multiple mode selection parameters correspond to multiple area ratio ranges. Different mode selection parameters correspond to different area ratio ranges. The encoding mode determination module is specifically used for: The mode selection parameter corresponding to the first area proportion range among the multiple area proportion ranges is determined as the first mode selection parameter, and the first area proportion range is the area proportion range in which the ROI area proportion is located. The GOP encoding mode corresponding to the first mode selection parameter is determined as the encoding mode of the first GOP.

18. The apparatus according to any one of claims 15-17, characterized in that, The encoding mode determination module is specifically used for: Determine the ROI in the first video frame; Determine the area of ​​the first video frame and the area of ​​the ROI; The ratio of the area of ​​the ROI to the area of ​​the first video frame is determined as the ROI area ratio of the first video frame.

19. The apparatus as claimed in claim 18, characterized in that, The encoding mode determination module is specifically used for: The first video frame is used as input to a segmentation model to obtain a segmented image output by the model. This segmented image is used to segment the Region of Interest (ROI) and non-ROI in the first video frame; or Determine the type of each object in the first video frame, and based on the type of each object in the first video frame, determine the ROI in the first video frame.

20. The apparatus according to any one of claims 15-19, characterized in that, The scene determination module is also used for: Determine the content complexity of the first video frame and / or the transmission bandwidth of the first GOP; The encoding mode determination module is also used for: Based on the ROI area ratio, the content complexity, and / or the transmission bandwidth, the encoding mode of the first GOP is determined from the plurality of GOP encoding modes.

21. The apparatus according to any one of claims 15-20, characterized in that, The first video frame is an I-frame included in the first GOP.

22. The apparatus according to any one of claims 14-21, characterized in that, The device further includes: An instruction information encoding module is used to encode first mode instruction information into the bitstream, wherein the first mode instruction information is used to indicate the encoding mode of the first GOP.

23. A video decoding device, characterized in that, The device includes: The encoding mode determination module is used to determine the encoding mode of the first image group (GOP). The encoding mode of the first GOP is determined from multiple GOP encoding modes based on the scene information corresponding to the first GOP. The first GOP is the GOP to be decoded. The GOP reconstruction module is used to parse the bitstream according to the encoding mode of the first GOP to obtain the reconstructed video frames included in the first GOP.

24. The apparatus as claimed in claim 23, characterized in that, The encoding mode determination module is specifically used for: The bitstream is parsed to obtain the reconstructed first video frame, which is an I-frame included in the first GOP. The area ratio of the region of interest (ROI) in the reconstructed first video frame is determined, where the ROI area ratio is the ratio of the area of ​​the ROI in the reconstructed first video frame to the area of ​​the reconstructed first video frame. Based on the ROI area ratio, the encoding pattern of the first GOP is determined from the plurality of GOP encoding patterns.

25. The apparatus as claimed in claim 24, characterized in that, The encoding mode determination module is specifically used for: Determine the content complexity of the reconstructed first video frame; Based on the ROI area ratio and the content complexity, the encoding pattern of the first GOP is determined from the multiple GOP encoding patterns.

26. The apparatus as claimed in claim 23, characterized in that, The encoding mode determination module is specifically used for: The bitstream is parsed to obtain first mode indication information, which is used to indicate the encoding mode of the first GOP. The encoding mode of the first GOP is determined based on the first mode indication information.

27. An encoding device, characterized in that, The encoding device includes: a processor coupled to a memory for storing programs or instructions, which, when executed by the processor, cause the encoding device to perform the method as described in any one of claims 1-9.

28. A decoding device, characterized in that, The decoding device includes: a processor coupled to a memory for storing programs or instructions, which, when executed by the processor, cause the decoding device to perform the method as described in any one of claims 10-13.

29. A codec system, characterized in that, The encoding / decoding system includes the encoding device as described in claim 27, and / or the decoding device as described in claim 28.

30. A computer-readable storage medium, characterized in that, Includes program code that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-9.

31. A computer-readable storage medium, characterized in that, Includes program code that, when run on a computer, causes the computer to perform the method as described in any one of claims 10-13.

32. A computer program product, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-9.

33. A computer program product, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 10-13.

34. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a bitstream obtained by the method according to any one of claims 1-9.

35. A device for storing bitstreams, characterized in that, Includes at least one storage medium and a communication interface; The communication interface is used to receive or send code streams; The at least one storage medium is used to store the bit stream; The bitstream is from an encoder. Encoded according to any one of the encoding methods in claims 1-9.

36. A method for storing a bitstream, characterized in that, include: Receive the bitstream through the communication interface; The bitstream is stored in one or more storage media, and the bitstream is an encoder. Encoded according to any one of the encoding methods in claims 1-9.

37. A system for distributing bitstreams, characterized in that, Includes at least one storage medium and a video streaming device; The at least one storage medium is used to store a bitstream, which is an encoder. Encoded according to any one of the encoding methods in claims 1-9; The video streaming device is configured to, in response to a request from the decoder, transmit the bitstream in the at least one storage medium to the decoder.

38. A method for distributing a bitstream, characterized in that, include: Receive the first request; In response to the first request, a bitstream is selected from at least one storage medium; Send the bitstream to the destination device; The at least one storage medium is used to store a bitstream, which is an encoder. Encoded according to any one of the encoding methods in claims 1-9.

39. A system for processing bitstreams, characterized in that, Includes an image source device, an encoder, one or more storage media, and a destination device; The image source device is used to provide image data; The encoder is used to acquire the image data of the image source device through an interface, and encode the image data to obtain one or more bitstreams, wherein the bitstreams are encoded by the encoder according to any one of the encoding methods in claims 1-9; The encoder is used to store the one or more bit streams into one or more storage media; or The encoder is used to encapsulate the one or more bit streams to obtain a transmission bit stream; The encoder is used to transmit the transmitted code stream to the destination device via a communication link or communication network; The destination device is used to decapsulate the transmitted code stream to obtain the one or more code streams; The target device is used to decode the one or more bitstreams to obtain decoded data.

Images