Video encoding method, decoding method, device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a video encoding method and device, a video decoding method and device, an electronic device and a storage medium, which can be related to the fields of multimedia, cloud technology, artificial intelligence, and the like, and in particular to the field of coding and decoding. The encoding method comprises: determining an inter-frame prediction effect of a current to-be-encoded block; if the inter-frame prediction effect meets a preset condition, skipping an intra-block copy (IBC) prediction mode, and adopting a prediction encoding mode other than the IBC prediction mode as a target prediction mode of the current to-be-encoded block; and encoding the current to-be-encoded block by using the target prediction mode of the current to-be-encoded block to obtain an encoding result of the current to-be-encoded block. By using the method provided in the present application, the encoding efficiency can be effectively improved.

Description

Technical Field

[0001] This application belongs to the field of encoding and decoding technology and may relate to multimedia, cloud technology, artificial intelligence and other fields. Specifically, this application relates to a video decoding method, encoding method, apparatus and electronic device. Background Technology

[0002] Video is a continuous sequence of images, consisting of consecutive frames, each frame representing an image. In practical applications, video typically requires encoding to reduce the amount of data stored or transmitted.

[0003] Predictive coding is a crucial technique in video coding. It encodes prediction residuals rather than the original pixels, significantly improving coding performance. Broadly speaking, predictive coding can be divided into intra-frame prediction and inter-frame prediction. Intra-frame prediction utilizes the spatial correlation between neighboring pixels within an image, using already encoded pixels to predict the current pixel, thus removing spatial redundancy. Inter-frame prediction utilizes the temporal correlation between images, using neighboring encoded pixels to predict the current pixel, thus removing temporal redundancy.

[0004] With the rapid development of technology and the improvement of people's living standards, the demand for video encoding and decoding is also constantly increasing. How to reduce the complexity of encoding and decoding and improve the encoding effect has always been one of the important issues that relevant technical personnel are studying. Summary of the Invention

[0005] The purpose of this application is to provide a video encoding method, decoding method, apparatus, electronic device, and storage medium that can improve encoding efficiency. To achieve the above objective, the technical solutions provided by this application are as follows:

[0006] On one hand, embodiments of this application provide a video encoding method, the method comprising:

[0007] Determine the inter-frame prediction performance of the current block to be encoded;

[0008] If the inter-frame prediction effect meets the preset conditions, the IBC (Intra Block Copy) prediction mode is skipped, and a prediction coding mode other than the IBC prediction mode is used as the target prediction mode for the current block to be coded.

[0009] The target prediction mode is used to encode the current block to be encoded, and the encoding result of the current block to be encoded is obtained.

[0010] Optionally, the inter-frame prediction effect is characterized by at least one of the coding cost or the optimal inter-frame prediction mode of the current block to be coded; the inter-frame prediction effect satisfies at least one of the following preset conditions:

[0011] The coding cost of inter-frame prediction is less than or equal to the set cost; the optimal inter-frame prediction mode is one of the specified inter-frame prediction modes.

[0012] On the other hand, embodiments of this application provide a video encoding apparatus, which includes:

[0013] The pattern decision information determination module is used to determine the inter-frame prediction effect of the current block to be encoded;

[0014] The mode decision module is used to skip the IBC prediction mode and adopt a prediction coding mode other than the IBC prediction mode as the target prediction mode of the current block to be coded when the inter-frame prediction effect meets the preset conditions.

[0015] The encoding module is used to encode the current block to be encoded using the target prediction mode to obtain the encoding result of the current block to be encoded.

[0016] Optionally, the inter-frame prediction effect is characterized by at least one of the coding cost or the best inter-frame prediction mode of the current block to be coded, and the inter-frame prediction effect satisfies at least one of the following preset conditions:

[0017] The coding cost of inter-frame prediction is less than or equal to the set cost;

[0018] The optimal inter-frame prediction mode is one of the specified inter-frame prediction modes.

[0019] Optionally, the specified inter-frame prediction modes mentioned above include at least the inter-frame skip mode.

[0020] Optionally, the pattern decision information determination module can be used to: determine the content type of the current block to be encoded, wherein the content type is natural content or screen content;

[0021] The mode decision module can be used to: skip the IBC prediction mode and adopt a prediction coding mode other than the IBC prediction mode as the target prediction mode for the current block to be encoded when the content type of the current block to be encoded is natural content; or skip the IBC prediction mode when the inter-frame prediction effect meets the preset conditions and the content type of the current block to be encoded is natural content.

[0022] Optionally, the content type of the current block to be encoded is obtained in any of the following ways:

[0023] Based on the pixel values ​​of each pixel in the image content of the current block to be encoded, the color statistical features of the current block to be encoded are determined, and based on the color statistical features, the content type of the current block to be encoded is determined;

[0024] The content type of each block to be encoded in the video frame to which the current block to be encoded belongs is predicted by the trained classification model, and each block to be encoded includes the current block to be encoded.

[0025] Optionally, the trained classification model is obtained by training an initial classification model based on multiple training samples. Each training sample includes a sample image patch and a label for that sample image patch. The label of a sample image patch indicates whether the content type of the sample image patch is natural content or screen content. The multiple training samples are obtained in the following way:

[0026] Acquire multiple sample image patches;

[0027] For each sample image block, determine the color statistical features of the image content of the sample image block, and obtain the label of the sample image block based on the color statistical features of the sample image block.

[0028] Optionally, the color statistical features include at least one of global statistical features or local statistical features. The global statistical features of the block to be processed (the block to be processed is the current block to be encoded or a sample image block) are determined in the following way:

[0029] Determine the pixel value of each pixel in the image content of the block to be processed;

[0030] For each pixel, determine the color category to which the pixel value belongs among a plurality of first color categories, wherein each first color category corresponds to a pixel value range;

[0031] Based on the color category to which each pixel belongs, the number of the first category of color categories appearing in the image content of the block to be processed is determined, and the global statistical feature includes the number of the first category;

[0032] The content type of the block to be processed is determined in the following way:

[0033] If the number of the first category is not less than the first preset value or the number of the first category is not greater than the second preset value, the content type of the block to be processed is determined to be natural content.

[0034] Optionally, the color statistical features include local statistical features, which are determined in the following ways:

[0035] The block to be processed is divided into multiple sub-blocks;

[0036] For each sub-block, based on the pixel value of each pixel in the sub-block, determine the color category to which the pixel value of each pixel in the sub-block belongs in multiple second image categories, and determine the number of second categories of color categories appearing in the sub-block;

[0037] The local statistical features include the number of second categories corresponding to each sub-block;

[0038] The content type of the block to be processed is determined in the following way:

[0039] The content type of the block to be processed is determined to be natural content if at least one of the following conditions is met:

[0040] The proportion of the corresponding second category number of sub-blocks in all sub-blocks is not less than the set value and is not less than the set proportion.

[0041] The number of sub-blocks in the corresponding second category that is not less than a set value is not less than a set number.

[0042] Optionally, the pattern decision information determination module can be used to: determine the block size of the block to be processed; if the block size is less than or equal to a set size, use the global statistical features as the color statistical features; if the block size is greater than the set size, the color statistical features include local statistical features.

[0043] Optionally, the inter-frame prediction effect meeting the preset conditions includes: the best inter-frame prediction mode of the current block to be encoded is one of the specified inter-frame prediction modes;

[0044] The target prediction mode of the current block to be encoded is determined in the following way:

[0045] If the inter-frame prediction effect meets the preset conditions, the best inter-frame prediction mode is taken as the target prediction mode.

[0046] Without skipping the IBC prediction mode, the IBC prediction mode is used as the target prediction mode, or the target prediction mode is determined from a first candidate mode set, wherein the first candidate mode set includes at least one prediction coding mode other than the specified inter-frame prediction mode and the IBC prediction mode.

[0047] When the step of determining the inter-frame prediction effect is performed when the content type of the current block to be encoded is determined to be screen content, if the content type of the current block to be encoded is natural content, the target prediction mode is determined from the second candidate mode set, wherein the second candidate mode set includes at least one prediction coding mode other than the IBC prediction mode.

[0048] Optionally, the pattern decision pattern can also be used to perform at least one of the following:

[0049] If the current encoding / decoding application scenario is a scenario in the first scenario set, then the process of determining whether to skip the IBC prediction mode is not performed. Instead, the target prediction mode of the current block to be encoded is determined from the candidate prediction coding mode set including the IBC prediction coding mode. The first scenario set includes at least one encoding / decoding application scenario.

[0050] If the current encoding / decoding application scenario is a scenario in the second scenario set, then skip the IBC prediction mode;

[0051] If the current block to be encoded is the background region of a video frame, then skip the IBC prediction mode.

[0052] On the other hand, embodiments of this application provide a video decoding method, the method comprising:

[0053] Obtain the encoding result corresponding to the current block to be decoded, wherein the encoding result is obtained by encoding the current block to be encoded using the encoding method provided in the embodiments of this application;

[0054] By analyzing the encoding results, the target prediction mode corresponding to the current block to be decoded is determined;

[0055] Based on the target prediction pattern, the encoding result is parsed to obtain the decoding result of the current block to be decoded.

[0056] On the other hand, embodiments of this application provide a video decoding apparatus, which includes:

[0057] The encoding result acquisition module is used to acquire the encoding result corresponding to the current block to be decoded, wherein the encoding result is obtained by encoding the current block to be encoded using the encoding method provided in the embodiments of this application;

[0058] The decoding module is used to determine the target prediction mode corresponding to the current block to be decoded by parsing the encoding result, and to obtain the decoding result of the current block to be decoded by parsing the encoding result based on the target prediction mode.

[0059] On the other hand, embodiments of this application also provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method provided in any optional embodiment of this application.

[0060] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in any optional embodiment of this application.

[0061] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the methods provided in any optional embodiment of this application.

[0062] The beneficial effects of the technical solution provided in this application are as follows:

[0063] The video coding method provided in this application, when encoding the current block to be encoded, can evaluate whether the IBC prediction mode is suitable for the current frame to be encoded based on the inter-frame prediction effect of the current block to be encoded. Specifically, when at least one of the encoding cost of the current block to be encoded corresponding to the inter-frame prediction mode or the best inter-frame prediction mode of the current frame to be encoded meets a preset condition, it can be determined to skip the IBC prediction mode and use another prediction mode as the target prediction mode for encoding the current block to be encoded. Using the scheme provided in this application, some situations where the IBC prediction mode can be skipped can be quickly identified, effectively reducing the proportion of the encoding complexity of the IBC prediction mode in the overall encoding complexity of the video. This improves encoding efficiency while maintaining encoding performance, better meeting the needs of practical applications. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0065] Figure 1 A flowchart illustrating a video encoding method provided in an embodiment of this application;

[0066] Figure 2 A flowchart illustrating a video coding method provided in this application;

[0067] Figure 3 This application provides a schematic diagram of a color classification process for image blocks.

[0068] Figure 4 and Figure 5 A flowchart illustrating two video encoding methods provided in the embodiments of this application;

[0069] Figure 6 A flowchart illustrating a video decoding method provided in an embodiment of this application;

[0070] Figure 7 A schematic diagram of a system architecture in an application scenario provided by an embodiment of this application;

[0071] Figure 8 A schematic diagram of a system architecture in another application scenario provided by an embodiment of this application;

[0072] Figure 9 A flowchart illustrating a video encoding method provided in an embodiment of this application;

[0073] Figure 10 This is a schematic diagram of the structure of a video encoding device provided in an embodiment of this application;

[0074] Figure 11 This is a schematic diagram of the structure of a video decoding device provided in an embodiment of this application;

[0075] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0076] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0077] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.” When describing multiple (two or more) items, if the relationship between the multiple items is not explicitly defined, the multiple items can refer to one, several or all of the multiple items. For example, the description of "parameter A includes A1, A2, A3" can be implemented as parameter A includes A1 or A2 or A3, or it can be implemented as parameter A includes at least two of the three items A1, A2 and A3.

[0078] This application provides a video encoding and decoding scheme. Using this scheme, the inter-frame prediction effect and / or the content features of the block to be encoded can be used to determine whether to skip the IBC prediction mode. This scheme can reduce the complexity of encoding and effectively improve encoding efficiency while maintaining encoding performance, thus better meeting the needs of practical applications.

[0079] Optionally, the data processing involved in the methods provided in this application embodiment can be implemented based on cloud technology. For example, the data calculations involved in the video encoding and decoding process can be implemented using cloud computing technology, and the data storage involved can be implemented using cloud storage, such as video storage.

[0080] Cloud computing is a computing model that distributes computing tasks across a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" are infinitely scalable, readily available, on-demand, expandable, and pay-as-you-go. Cloud storage is a new concept that extends and develops from cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to aggregate a large number of various types of storage devices (also called storage nodes) in a network through application software or application interfaces to work collaboratively and provide data storage and business access functions.

[0081] It should be noted that, in the optional embodiments of this application, the data related to object information, when applied to specific products or technologies, requires the permission or consent of the object. Furthermore, the collection, use, and processing of this data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if the embodiments of this application involve data related to an object, it must be obtained with the object's authorization and consent, the authorization and consent of relevant departments, and in accordance with the relevant laws, regulations, and standards of the country and region. If the embodiments involve personal information, the acquisition of all personal information requires the individual's consent. If sensitive information is involved, the separate consent of the information subject is required. The embodiments also need to be implemented with the object's authorization and consent.

[0082] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0083] Figure 1 This illustration shows a flowchart of a video encoding method provided in an embodiment of this application. This method can be executed by any electronic device, such as a user terminal (also referred to as a terminal, terminal device, or user device) or a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The user terminal can be a smartphone, tablet, laptop, desktop computer, smart voice interaction device (e.g., smart speaker), wearable electronic device (e.g., smartwatch), in-vehicle terminal, smart home appliance (e.g., smart TV), AR / VR device, etc., but is not limited to these. The user terminal and the server can be directly or indirectly connected via wired or wireless communication, and this embodiment of the application does not impose any limitations on this connection.

[0084] like Figure 1 As shown, the video encoding method provided in this application embodiment may include the following steps S110 to S130:

[0085] Step S110: Determine the inter-frame prediction effect of the current block to be encoded.

[0086] Step S120: If the inter-frame prediction effect meets the preset conditions, skip the IBC prediction mode and use a prediction coding mode other than the IBC prediction mode as the target prediction mode for the current block to be coded.

[0087] Step S130: Encode the current block to be encoded using the target prediction mode of the current block to be encoded, and obtain the encoding information of the current block to be encoded.

[0088] It is understood that the current block to be encoded refers to the image block currently being encoded in the current frame to be encoded (which can be simply referred to as the current frame to be encoded / current frame) during the encoding process. In some embodiments described in this application, the block being processed during the encoding and decoding processes can be referred to as the current block. The current block in the encoding stage is the current block to be encoded, and the current block in the decoding stage is the current block to be decoded, i.e., the image block to be reconstructed.

[0089] In this embodiment, the specific acquisition / determination method for the current frame and the current block to be encoded is not limited, and any existing video encoding / decoding technology can be used. For example, each frame image in the video stream to be encoded (or each frame image after sampling / frame extraction) can be encoded sequentially or in parallel according to a preset strategy. The frame image currently being encoded is the current frame, which can be divided into several coding blocks (CBs), and each coding block can be encoded sequentially according to a set order.

[0090] The solution provided in this application can be applied to any application scenario with encoding and decoding requirements. That is, the video stream to be encoded can be any video that needs encoding in any application scenario, such as short videos, movies, etc. The video can be real-time video, such as live streaming video, online conference video, online call video, etc., or it can be non-real-time video, such as pre-recorded video, video from a video website (such as video in the video database of an application providing video viewing services), etc. The video to be encoded includes at least one video frame; that is, the method provided in this application is also applicable to encoding an image.

[0091] In this embodiment, the granularity of the coding block size is not limited. A coding block can be an image block of a set size in a video frame, and the size of the coding block can be pre-configured according to requirements. Optionally, the video frame to be encoded can be divided once or multiple times, and each sub-region after division can be used as a block to be encoded. For example, the frame to be encoded can be divided according to a preset size, or the frame to be encoded can be divided into n×n (n≥2) sub-regions, and each sub-region can be used as a block to be encoded, or each sub-region can be divided again, and each of the second divisions can be used as a block to be encoded. Optionally, a coding block can be a coding unit (CU).

[0092] For ease of description, the current block to be encoded in the following embodiments can be illustrated using the current CU or the current CB as an example. Multiple coded blocks in a video frame can have the same or different sizes. During the encoding process, the terms "coded block" and "image block" used below refer to the blocks that need to be encoded in the current frame.

[0093] Determining a suitable predictive coding mode for the block to be encoded is a crucial step in ensuring encoding quality. With the rapid development of applications such as screen sharing and cloud computing, video content generated by electronic devices like computers and mobile terminals has received increasing attention, leading to the development of corresponding screen content coding. Interpretive Coded (IBC) prediction mode (also known as IBC pattern) is a prediction mode primarily designed for screen content. Its basis is that screen content often contains the same patterns within the same frame, such as a single letter appearing multiple times in text. In this case, the encoder searches within the already encoded regions of the current frame and may find a region that is very similar to or perfectly matches the current block to predict it. In this situation, the encoder only needs to encode a very small prediction residual. While IBC prediction mode improves encoding performance for screen content, it's not suitable for all situations. Searching for similar regions within the already encoded areas of the current frame is computationally complex, requiring more resources and longer encoding times. This is highly unfriendly, or even unacceptable, in many business scenarios (such as high transcoding load services like video-on-demand). In some scenarios (such as low-latency services like live streaming), high encoding complexity can directly degrade the user experience. Therefore, quickly determining the IBC prediction mode is crucial for reducing computational complexity.

[0094] In this application, to improve coding efficiency while maintaining coding performance, based on the principle of IBC prediction mode, several alternative schemes are proposed to quickly determine whether to skip the IBC prediction mode (i.e., whether to perform IBC mode coding). Among these, considering that IBC prediction mode typically performs well for frames containing the same pattern, and that inter-frame prediction can achieve good pattern matching even without abrupt changes in image content, as an alternative scheme, a scheme is proposed to determine whether to skip the IBC prediction mode based on the inter-frame prediction effect of the current block to be coded.

[0095] Inter-frame prediction performance refers to the coding performance of the current frame to be coded in the inter-frame prediction mode. It can characterize the quality of the inter-frame prediction mode and reflect the applicability of the inter-frame prediction mode to the current block to be coded. The better the inter-frame prediction performance, the more suitable the inter-frame prediction mode is for the current block to be coded.

[0096] It is understandable that the inter-frame prediction effect is determined when the optional / candidate predictive coding modes of the current block to be coded include the inter-frame prediction mode. If the current frame belongs to an image that uses the intra-frame prediction mode, then the predictive coding mode of each coded block in the image is intra-frame prediction coding.

[0097] like Figure 2 In the flowchart shown, when encoding the video to be encoded, the frame type of the current frame can be determined first, and if the current frame supports inter-frame prediction mode (such as...), then... Figure 2 When the current frame shown is a P-frame (predictive frame) or a B-frame (bidirectional frame), a step of determining whether to skip the IBC mode can be performed on the current block in the current frame. The determined current block corresponds to the inter-frame prediction effect of at least one inter-frame prediction mode, and a decision is made on whether to skip the IBC prediction mode based on the inter-frame prediction effect. If the current frame is an I-frame (intra-frame / intra-coded frame), the prediction coding mode of the coded block in the frame image is intra-frame prediction coding, and the inter-frame prediction mode is not an optional prediction coding mode for the frame image. When encoding the current block in the current frame, the intra-frame prediction mode is used. For example, based on the coding cost, a target prediction mode can be selected from multiple intra-frame prediction modes (which may or may not include the IBC prediction mode). Alternatively, other optional embodiments provided in this application can be used to first determine whether the IBC mode can be skipped. If it is determined that it should be skipped, the IBC prediction mode can be excluded, and the target prediction mode for the current block to be encoded can be determined from the intra-frame prediction modes that do not include the IBC prediction mode.

[0098] Regarding the specific method for inter-frame prediction performance, this application does not impose a unique limitation. Theoretically, any information that can characterize the quality, applicability, or coding effect of the inter-frame prediction mode for the current prediction block is acceptable. Optionally, the inter-frame prediction performance can be characterized by at least one of the coding cost or the optimal inter-frame prediction mode for the current block to be coded; correspondingly, the inter-frame prediction performance must satisfy at least one of the following preset conditions:

[0099] The encoding cost is less than or equal to the set cost;

[0100] The optimal inter-frame prediction mode is one of the specified inter-frame prediction modes.

[0101] Optionally, the specified inter-frame prediction mode includes at least the inter-frame skip mode.

[0102] During the encoding process, to select the most suitable prediction mode for the current block to be encoded, the encoder determines the optimal prediction mode from multiple candidate prediction modes based on a certain strategy (such as rate-distortion cost). Since different prediction coding modes typically have their own more applicable scenarios and less applicable scenarios, determining the optimal inter-frame prediction mode for the current block reflects the applicability of the inter-frame prediction mode to the current block, and thus reflects the inter-frame prediction performance. The optimal inter-frame prediction mode can be used to characterize the inter-frame prediction performance. One or more inter-frame prediction modes can be pre-specified / configured, and the quality of the inter-frame prediction performance for the current block can be determined by judging whether the optimal inter-frame prediction mode for the current block is the specified mode.

[0103] The specific inter-frame prediction mode (IBC) can be configured based on empirical and / or experimental values. In practical applications, the encoder typically uses the IBC skip mode for prediction coding when the IBC prediction effect is good. Therefore, whether the skip mode was selected during the inter-frame prediction of the current block to be encoded can be used as a criterion for whether the prediction effect is good enough. Optionally, if the inter-frame prediction result (optimal inter-frame prediction mode) of the current block to be encoded is the skip mode, the IBC prediction mode is skipped.

[0104] This application does not limit the specific method for determining the optimal inter-frame prediction mode for the current block to be encoded; it can be determined according to a pre-configured prediction mode determination strategy. When the specified inter-frame prediction mode includes the skip mode, the specific implementation for determining whether the optimal inter-frame prediction mode for the current block to be encoded is the inter-frame skip mode can adopt any existing judgment scheme for determining whether the current block to be encoded belongs to the skip mode. For example, it can be determined whether the skip mode can be used as the optimal prediction mode based on whether the residual of the motion vector corresponding to the current block to be encoded is 0 or whether the change coefficient is 0. As an optional solution, the optimal prediction mode can be determined based on whether the image region of the current block to be encoded is the background region of the video frame. If the image region of the current block to be encoded is the background region, the skip mode can be used as the optimal inter-frame prediction mode for the current block to be encoded.

[0105] The coding cost can characterize the coding effect of the prediction mode. Therefore, the coding cost of the inter-frame prediction mode can be used as a condition or one of the conditions for determining whether to skip the IBC prediction mode. If the coding cost is less than or equal to the set cost, it means that the inter-frame prediction mode can also achieve a good coding effect, and the inter-frame prediction mode can be used as the target prediction mode to skip the IBC prediction mode. Alternatively, other information can be combined to determine whether the IBC prediction mode can be skipped. Optionally, the coding cost can be represented by one or more coding effect evaluation metrics. For example, it can be represented by rate-distortion loss (rate-distortion cost). The smaller the rate-distortion loss, the smaller the coding loss. It can also be represented by the matching degree between the current block to be coded and its prediction block. The higher the matching degree, the better the effect, and the lower the coding cost is considered. For example, the prediction block of the current block to be coded can be searched in the reference frame of the current frame, and the sum of the differences between the pixel values ​​of the corresponding pixels in the current block to be coded and the prediction block is used as the matching degree. The smaller the sum of the differences, the higher the matching degree.

[0106] In this embodiment, when determining whether the coding cost of the current block to be coded is less than or equal to a set cost, the coding cost here can be the cost of the current block to be coded corresponding to a specific inter-frame prediction mode. That is, a coding cost less than or equal to a set cost means that there exists at least one inter-frame prediction mode for the block to be coded with a coding cost less than the set cost. The coding cost can also be a general coding cost that can characterize the coding effect of the current block to be coded corresponding to the inter-frame prediction type (such as relative to the intra-frame prediction type). For example, when using the matching degree between the block to be coded and its prediction block to characterize the coding loss, it is not necessary to calculate the rate-distortion loss of the current block to be coded corresponding to one or more inter-frame prediction modes.

[0107] The specific value of the aforementioned setting cost can be pre-configured based on experimental and / or empirical values, and can also be adjusted according to different application scenarios. When the coding cost is the coding cost of various specific inter-frame prediction modes (such as skip mode), the setting cost values ​​corresponding to different inter-frame prediction modes can be the same or different.

[0108] In practice, the aforementioned coding cost and optimal inter-frame prediction mode can be used individually or in combination. For example, a candidate mode set can be pre-configured, which includes one or more inter-frame prediction modes, such as mode a and mode b. If the coding cost of the current block to be coded corresponding to mode a is less than a set cost A, or the coding cost of the current block to be coded corresponding to mode b is less than a set cost B, it can be considered that using the inter-frame prediction mode for the current block to be coded can already achieve a good coding effect, and the IBC prediction mode can be skipped. Optionally, the prediction mode with a cost less than the set cost can be used as the target prediction mode for predictive coding of the current block to be coded. Alternatively, if the coding costs corresponding to both modes are less than their respective set costs, the mode with the smaller coding cost can be used as the target prediction mode for predictive coding of the block to be coded.

[0109] For example, when coding cost and best inter-frame prediction mode are used in combination, the coding cost of the current block to be coded can be calculated for each of the specified inter-frame prediction modes. The mode with the lowest coding cost is determined as the best inter-frame prediction mode. If the coding cost of this mode is less than the set cost, the inter-frame prediction effect of the current block to be coded is considered to meet the preset condition (the best inter-frame prediction mode is the specified inter-frame prediction mode and the coding cost corresponding to the best inter-frame prediction mode is less than the set cost), and the IBC prediction mode can be skipped.

[0110] Since the IBC prediction mode is primarily designed for screen content, in this embodiment, the decision to skip the IBC prediction mode can be based on the characteristics of the image content of the current block to be encoded. Optionally, the method provided in this application further includes:

[0111] Determine the content type of the current block to be encoded, which is either natural content or screen content;

[0112] If the content type is natural content, skip the IBC prediction mode and use a prediction coding mode other than the IBC prediction mode as the target prediction mode for the current block to be coded; or, if the inter-frame prediction effect meets the preset conditions, skip the IBC prediction mode, including: if the inter-frame prediction effect meets the preset conditions and the content type is natural content, skip the IBC prediction mode.

[0113] In other words, when determining whether to skip the IBC prediction mode, one can base the decision on either the inter-frame prediction performance of the current block to be encoded or the content type of the current block to be encoded, or a joint decision on both. A specific implementation can be as follows:

[0114] 1) If the content type of the block to be encoded is natural content or the inter-frame prediction effect of the block to be encoded meets the preset conditions, then skip the IBC prediction mode.

[0115] 2) If the content type of the block to be encoded is natural content and the inter-frame prediction effect of the block to be encoded meets the preset conditions, then skip the IBC mode.

[0116] For the above scheme 1), the step of determining the content type of the current block to be encoded can be performed before or after the step of determining whether the inter-frame prediction effect of the current block to be encoded meets the preset conditions. That is, if the content type of the current block to be encoded is determined to be screen content, the inter-frame prediction effect of the current frame to be encoded can be determined again to determine whether the prediction effect meets the preset conditions. Alternatively, if the inter-frame prediction effect of the current block to be encoded does not meet the preset conditions, the content type of the current block to be encoded can be determined again, and whether to skip the IBC prediction mode can be determined based on the content type.

[0117] For scheme 2) above, there is no restriction on the execution order of the step of determining whether the content type is natural content and the step of determining whether the inter-frame prediction effect meets the preset conditions. Either step can be executed first. For example, the content type can be determined first. If it is natural content, the judgment can continue based on the inter-frame prediction effect. If it is not natural content, the IBC prediction mode is not skipped, and the step of determining the inter-frame prediction effect can be omitted. Of course, the two steps can also be executed in parallel. Only if both the content type is natural content and the inter-frame prediction effect meets the preset conditions will the IBC prediction mode be skipped. Otherwise, the IBC prediction mode will not be skipped and will be used as a candidate prediction mode, or the IBC prediction mode can be directly used as the target prediction mode.

[0118] Optionally, the content type of the current block to be encoded can be obtained in any of the following ways:

[0119] Based on the pixel values ​​of each pixel in the image content of the current block to be encoded, determine the color statistical features of the current block to be encoded, and based on the color statistical features, determine the content type of the current block to be encoded.

[0120] The content type of each block to be encoded in the video frame to which the current block to be encoded belongs is predicted by the trained classification model. Each block to be encoded includes the current block to be encoded.

[0121] Because screen content and natural content are generated in different ways, screen content has unique content characteristics compared to natural content captured by image acquisition devices. Therefore, it is possible to determine whether the current block to be encoded is an image block of natural content or an image block of screen content based on the content characteristics of the current block to be encoded.

[0122] In this embodiment, the content feature of the block to be encoded can be a statistical feature of the image content. For example, it can be a statistical feature of each pixel in the block to be encoded corresponding to at least one pixel attribute parameter. Optionally, the content feature can be a color statistical feature of the current block to be encoded. This feature can be derived statistically from the pixel values ​​of each pixel in the block to be encoded. For example, the color statistical feature can be a histogram of the image content of the block to be encoded. This histogram includes the number of each pixel value appearing in the block to be encoded, that is, the number of pixels with each pixel value. According to the histogram, the number of pixel values ​​appearing in the block to be encoded, as well as the number of pixels with each pixel value, can be known. Compared with natural content, the color information of screen content is generally more concentrated, the continuity of pixel value distribution in the image is poor, and the number of pixel values ​​appearing is also less (most pixels have the same pixel value). Therefore, the histogram of the block to be encoded can be used to distinguish whether the block to be encoded is an image block of natural content or an image block of screen content.

[0123] It is understood that in the embodiments of this application, the content type of a block to be encoded being natural content or screen content does not mean that the image block is 100% natural content or screen content. Rather, it is the result identified based on its content characteristics. A block to be encoded may contain both natural content and screen content. The content type of a block to be encoded being natural content can be interpreted as a high probability that the image content contained in the block to be encoded is natural content or that the main image content was generated by a camera.

[0124] As an alternative method to determine the content type of the block to be encoded, a trained neural network model (classification model) can be used to identify whether the image content is natural content or screen content. This classification model can be trained using labeled training samples. Each training sample can include a sample image block and a label representing the true content type of that sample image block; for example, a label of 1 indicates that the true content type is natural content, and a label of 0 indicates that the true content type is screen content.

[0125] This application does not limit the method of obtaining training samples. Optionally, a large number of sample images can be obtained, and the sample images can be divided into multiple image blocks. For example, each sample image can be divided into multiple sample image blocks according to the size of the block to be encoded during the encoding process. The labels of the sample image blocks can be obtained through manual annotation or other methods.

[0126] As an alternative, multiple training samples can be obtained in the following ways:

[0127] Acquire multiple sample image patches;

[0128] For each sample image block, determine the color statistical features of the image content of the sample image block, and obtain the label of the sample image block based on the color statistical features of the sample image block.

[0129] This approach reduces the manual costs of sample acquisition, improves acquisition efficiency, and allows for the convenient and rapid acquisition of a large number of training samples. In this embodiment, the specific model structure of the classification model is not limited; theoretically, any binary classification model based on any type of network can be used. During the training phase, the model input can be individual sample image patches, and the output is the classification result of the sample image patch. This classification result is a prediction of the content type of the sample image patch. For example, a classification result of 1 represents natural content, and a classification result of 0 represents screen content. Alternatively, the classification result can be two probability values ​​corresponding to natural content and screen content, respectively, with the sum of the two probability values ​​being 1. Each probability value represents the likelihood that the sample image patch belongs to the content type corresponding to that probability value. Based on the classification results of a large number of sample image patches and the true results represented by the labels, the model's training loss can be calculated, and the model parameters can be optimized and adjusted. Through continuous training, a classification model that meets the application requirements can be obtained. After obtaining the trained classification model, for the frames to be encoded in the video stream (such as p-frames and b-frames), each encoded block of the frame can be input into the model, and the model can predict the content type of each encoded block. By using model prediction, the content type of each block to be encoded in the frame to be encoded can be known conveniently and quickly, thereby improving the efficiency of determining whether to skip the IBC prediction mode and increasing the encoding speed.

[0130] In an optional embodiment of this application, the color statistical features of the block to be encoded or the sample image block in the training samples may include at least one of global statistical features or local statistical features. For ease of description, the block to be encoded and the sample image block can be collectively referred to as the block to be processed. The global statistical features of the block to be processed refer to the color statistical features of the entire image block obtained by statistically analyzing the pixel values ​​of all pixels in the block to be processed, while the local statistical features are the color statistical features of the sub-blocks obtained by statistically analyzing the pixel values ​​of the pixels contained in each sub-block after dividing the block to be processed into multiple sub-blocks.

[0131] The distribution of various pixel values ​​in an image block can vary greatly depending on the application scenario, the predictive coding method, or the application requirements. For different scenarios, predictive coding methods, or requirements, one or two of the global statistical features or local statistical features can be used to determine the content type of the image block in order to better ensure the judgment results.

[0132] Optionally, the global statistical features of the block to be processed (the current block to be encoded or the sample image block) can be determined in the following ways:

[0133] Determine the pixel value of each pixel in the image content of the block to be processed;

[0134] For each pixel, determine the color category to which the pixel value belongs among multiple first color categories, where each first color category corresponds to a first pixel value range;

[0135] Based on the first color category to which each pixel belongs, determine the number of first categories of the first color category appearing in the image content of the block to be processed. The global statistical features include the number of first categories.

[0136] Accordingly, the content type of the block to be processed can be determined in the following ways:

[0137] If the number of items in the first category is not less than the first preset value or the number of items in the first category is not greater than the second preset value, the content type of the block to be processed is determined to be natural content.

[0138] The first preset value is greater than the second preset value. The specific values ​​for these two preset values ​​can be based on experimental values, empirical values, or statistical values ​​obtained by statistically analyzing the pixel value categories of sample images. Optionally, the second preset value can be 1. In this case, if the total number of color categories appearing in a block to be processed is greater than the second preset value but less than the first preset value, the block to be processed can be considered screen content. Each color category corresponds to a pixel value interval, and a pixel value interval can include multiple consecutive pixel values; different pixel value intervals do not contain duplicate pixel values.

[0139] The specific method for dividing pixel value intervals is not limited in this application. As an optional solution, the distribution range of pixel values ​​can be divided into multiple color categories, i.e., color intervals, according to preset pixel value intervals. For example, a pixel value can refer to the grayscale value of a pixel. Taking a pixel value distribution range of 0 to 255 as an example, every 8 consecutive pixel values ​​can be considered as one color category, resulting in 32 color categories. For example, pixel values ​​0 to 7 belong to the 0th color category, pixel values ​​8 to 15 belong to the 1st color category, and so on, with pixel values ​​248 to 255 belonging to the 32nd color category. For the current block to be encoded or sample image block, the color categories existing in the block to be encoded or image block can be statistically determined based on the pixel values ​​of each pixel it contains and the pixel value intervals corresponding to each color category.

[0140] Taking the current block as an example, Figure 3 This diagram illustrates the process of determining the number of color categories present in the current block using the aforementioned alternative methods, as shown below. Figure 3As shown, each of the 32 color categories can have its own corresponding flag bit, with one flag bit corresponding to one color category. Each flag bit has an initial value, for example, an initial value of 0. When classifying the image content by color, if a pixel of a certain color category exists, the value of the flag bit corresponding to that color category will be updated, such as to 1. For the current block, when starting classification, the flag bits of the 32 color categories are first reset, for example, all flag bits are reset to the initial value of 0. Then, each pixel in the current block is classified sequentially according to a preset order, or multiple pixels are processed in parallel (classified simultaneously). If there are still unclassified pixels in the current block, the pixel value of the pixel to be classified is divided by 8 and rounded down to obtain the value s. The s-th color category is the color category to which the pixel belongs. The flag bit of the s-th color category is set to 1. If the flag bit is already 1, its value is kept at 1. After classifying the pixel values ​​of all pixels in the current block, the number of color classes with a flag value of 1 out of the 32 color classes is the number of color classes that exist in the current block.

[0141] For a block to be encoded, if the number of color categories appearing in the block is greater than a first preset value, it indicates that the color information of the image content in the block is relatively rich. The block can be considered to contain natural content and can be treated as an image block of natural content, thus IBC encoding is not required. If the number of color categories appearing in a block is not greater than a second preset value (e.g., the second preset value is 1), then the pixel values ​​of all pixels in the block are very similar, and the block is an image block with virtually no texture. In this case, even without IBC encoding, a target prediction mode with good encoding performance can be selected from inter-frame prediction modes or other intra-frame prediction modes. Therefore, IBC encoding, which has higher computational complexity, can still be used. Although the block may not be an image block of natural content, it can still be treated in the same way as an image block of natural content, and its content type can be considered as natural content.

[0142] Since global statistical features reflect the color distribution characteristics of all pixels in the entire image patch, they sometimes cannot reveal the distribution of local areas within the patch. For example, an image patch may contain many colors, but only a small local area may contain multiple colors. If the content in this local area has little impact on practical applications, classifying it as natural content might not be entirely accurate. Considering these potential issues, determining the content type of an image patch can also employ local statistical features, or a combination of local and global statistical features.

[0143] Optionally, the local statistical characteristics of the block to be processed are determined in the following ways:

[0144] Divide the block to be processed into multiple sub-blocks;

[0145] For each sub-block, based on the pixel value of each pixel in the sub-block, determine the color category to which the pixel value of each pixel in the sub-block belongs among multiple second color categories, and determine the number of second categories of the second color categories appearing in the sub-block; each second color category corresponds to a second pixel value range;

[0146] Among them, the local statistical features include the number of second categories corresponding to each sub-block.

[0147] Accordingly, the content type of the block to be processed can be determined in the following ways:

[0148] The content type of the block to be processed is determined to be natural content if at least one of the following conditions is met:

[0149] The proportion of sub-blocks with a corresponding second category number greater than or equal to the set value among all sub-blocks is not less than the set proportion; the number of sub-blocks with a corresponding second category number greater than or equal to the set value is not less than the set number.

[0150] The classification methods for the first and second color categories can be the same or different. By counting the number of second categories corresponding to each sub-block, the color distribution characteristics of each local area in the block to be processed can be obtained. If there are many sub-blocks with rich color categories in a block to be processed, the main content of the block to be processed can be considered as natural content, and its content type can be determined as natural content. Conversely, if there are few sub-blocks with rich color categories, the main content of the block to be processed can be considered as screen content.

[0151] The specific values ​​of the above-mentioned settings, percentages, and quantities can be configured according to actual needs, and this application embodiment does not impose any limitations.

[0152] As an optional approach, color statistical features can include both global and local statistical features. In this case, determining whether the content type of a block to be processed is natural content can be achieved by the global statistical features meeting certain conditions, the local statistical features meeting certain conditions, or both meeting their respective conditions. For example, if the number of the first category is not less than a first preset value and the aforementioned proportion is not less than a set proportion, the content type of the block to be processed is determined to be natural content. If the number of the first category is not less than the first preset value but the aforementioned proportion is less than the set proportion, the content type of the block to be processed can be considered not to be natural content.

[0153] As an alternative, the method provided in this application may also include:

[0154] Determine the block size of the block to be processed;

[0155] If the block size is less than or equal to the set size, the global statistical features will be used as the color statistical features;

[0156] If the block size is larger than the set size, the color statistical features include local statistical features.

[0157] Using this approach, if the size of the block to be processed is relatively small, meaning the number of pixels is relatively small, only the global statistical features of the block need to be calculated, reducing the amount of data processing. If the size of the block to be processed is relatively large, at least the local statistical features of the block must be calculated to ensure the accuracy of the judgment results as much as possible. The size setting can be pre-configured according to actual needs and can also be adjusted according to different scenarios.

[0158] Figure 4 and Figure 5 The diagram illustrates two optional schemes provided in the embodiments of this application. In these two schemes, the inter-frame prediction mode is specified as the inter-frame skip mode. The distribution range of pixel values ​​is divided into 32 color categories (first color category). Referring to the previous text, one color category corresponds to 8 pixel values. Pixel values ​​0-7 belong to category 0, pixel values ​​8-15 belong to category 1, and so on, with pixel values ​​248-255 belonging to category 31. The first preset value is 4, and the second preset value is 1.

[0159] like Figure 4 As shown, for the current block of the current frame, inter-frame predictive coding can be performed first, and the best inter-frame predictive mode can be determined based on the coding effect. Alternatively, it can be determined directly whether the skip mode is applicable according to the preset strategy (if applicable, the skip mode can be determined as the skip mode). Through this inter-frame predictive process, it can be determined whether the skip mode is the best prediction mode. If so, the inter-frame skip mode can be directly used as the target prediction mode for the current block, and the current block can be encoded using this mode.

[0160] If the skip mode is not the optimal prediction mode, further color classification is performed on the pixel values ​​of all pixels contained in the current block to determine the number of color categories appearing in the current block. If the number of color categories is not less than 4, the current block is considered to contain natural content, and IBC prediction is skipped. Another prediction mode is used to encode the current block. Since it has been determined that the inter-frame skip mode is not the optimal prediction mode, the target prediction mode for the current block can be any prediction mode other than the skip mode and IBC. For example, the optimal prediction mode determined during the inter-frame prediction process can be directly used as the target prediction mode, or the target prediction mode can be determined from other prediction modes based on encoding cost or other strategies. If the current block contains only 1 color category, inter-frame prediction can still achieve very good coding results. The same processing method is used for image blocks containing natural content, employing a prediction mode other than the skip mode and IBC to encode the current block.

[0161] If the best prediction mode for the current block is not the inter-frame skip mode, and the number of color categories appearing in the current block is greater than 1 and less than 4, then the IBC mode is not skipped. The IBC mode can be directly used to encode the current block, or the target prediction mode can be selected from the candidate prediction modes that include the IBC mode but do not include the inter-frame skip mode.

[0162] for Figure 5 The optional solutions shown are compared. Figure 4 and Figure 5 It can be seen that, Figure 5 The solution first performs the step of determining the number of color categories appearing in the current block. If the number of color categories cannot determine the number, IBC can be skipped. Figure 5 If the number of color categories is not equal to 1 or not less than 4, then the step of determining the best inter-frame prediction mode for the current block is executed. Based on whether the best inter-frame prediction mode is the skip mode, it is then determined whether the IBC mode can be skipped.

[0163] The scheme provided in this application utilizes at least one of the inter-frame prediction effect or content characteristics of the block to be encoded to determine whether to perform IBC prediction mode encoding. By adopting the scheme provided in this application, the IBC prediction mode can be skipped when it is not frequently used, reducing the computational complexity caused by the mode and accelerating encoding while maintaining encoding performance.

[0164] For the current block to be encoded, after determining that the IBC prediction mode can be skipped through the scheme provided in any embodiment of this application, a prediction coding mode other than the IBC prediction mode can be used as the target prediction mode for the block to be encoded. The block to be encoded is then encoded using the target prediction mode to obtain the encoding result. Wherein, this application does not limit the method of determining the target prediction mode when it is determined that the IBC prediction mode should be skipped. For example, it can be selected from candidate prediction modes other than the IBC prediction mode, such as selecting based on rate-distortion loss, choosing the prediction mode with the smallest rate-distortion loss as the target prediction mode; or, for example, if the optimal prediction mode (such as the optimal inter-frame prediction mode) for the block to be encoded has been determined during the process of determining whether to skip the IBC prediction mode, the optimal prediction mode can also be used as the target prediction mode. If, according to the scheme provided in this application, it is not determined that the IBC prediction mode should be skipped, the IBC prediction mode can be used as the target prediction mode, or the target prediction mode can be selected from candidate prediction modes that include the IBC prediction mode.

[0165] As an optional scheme, the inter-frame prediction effect of the current block to be encoded meets the preset conditions, including: the best inter-frame prediction mode of the current block to be encoded is one of the specified inter-frame prediction modes, for example, the best inter-frame prediction mode is the inter-frame skip mode. When the inter-frame prediction effect meets the preset conditions, the best inter-frame prediction mode is used as the target prediction mode of the current block to be encoded.

[0166] Optionally, without skipping the IBC prediction mode (e.g., when the inter-frame prediction effect of the current block to be encoded does not meet the preset conditions and the content type of the current block to be encoded is screen content), the IBC prediction mode is used as the target prediction mode, or the target prediction mode is determined from the first candidate mode set, wherein the first candidate mode set includes at least one prediction coding mode other than the specified inter-frame prediction mode and the IBC prediction mode.

[0167] Optionally, when the step of determining the inter-frame prediction effect is performed if the content type of the current block to be encoded is determined to be screen content, and if the content type of the current block to be encoded is natural content, the target prediction mode is determined from the second candidate mode set, wherein the second candidate mode set includes at least one predictive coding mode other than the IBC prediction mode. That is, if the content type of the current block to be encoded is determined to be natural content, the step of determining the inter-frame prediction effect of the current block to be encoded may not be performed, and the target prediction mode may be directly determined from the prediction modes other than the IBC prediction mode.

[0168] To more quickly determine whether the IBC prediction mode can be skipped, the method provided in this application may optionally include at least one of the following:

[0169] If the current encoding / decoding application scenario is a scenario in the first scenario set, then the process of determining whether to skip the IBC prediction mode is not executed, and the target prediction mode of the current block to be encoded is determined from the candidate prediction coding mode set including the IBC prediction coding mode.

[0170] If the current encoding / decoding application scenario is a scenario in the second scenario set, then skip the IBC prediction mode;

[0171] If the current block to be encoded is the background region of a video frame, skip the IBC prediction mode.

[0172] In practical applications, there are many video / image encoding and decoding application scenarios. In some application scenarios, the probability of the IBC prediction mode being selected as the target prediction mode for the block to be encoded is very small, or only a very small number of blocks to be encoded in the image / video are suitable for the IBC prediction mode. Considering this situation, the above-mentioned second scenario set can be pre-configured. This set includes one or more encoding and decoding scenarios that rarely use the IBC prediction mode. If the current encoding and decoding scenario belongs to the scenario in this set, all blocks to be encoded in all video frames in the video to be encoded will no longer need to determine whether the IBC prediction mode is applicable. That is, the IBC prediction mode is not used as a candidate prediction mode for the encoding and decoding scenarios in this set.

[0173] Similarly, a first set of scenarios can be used, where the encoding / decoding scenarios in this set are those with a high probability of having their pre-configured IBC prediction modes selected as the target prediction modes. If the current encoding / decoding application scenario belongs to this set, the decision on whether to skip the IBC prediction mode can be skipped, and the IBC prediction mode can be directly used as one of the prediction modes in the candidate prediction coding mode set. The target prediction mode can be selected from this set based on the rate-distortion loss. Optionally, the rate-distortion loss corresponding to the IBC prediction mode can be determined first. If the loss is less than a set threshold, the IBC prediction mode can be directly used as the target prediction mode. The aforementioned candidate prediction coding mode set can include multiple prediction modes.

[0174] In addition, for coded blocks that belong to the background region in a video frame, the computationally complex IBC prediction mode can be skipped, and other prediction modes can be used as the target prediction mode.

[0175] It should be noted that, in the absence of conflicts between the schemes, the various embodiments provided in this application can be used in combination. The schemes provided in the embodiments of this application can also be used in combination with other coding schemes that can speed up the selection of target prediction modes. For example, they can be combined with other schemes that determine whether to skip the IBC prediction mode, or with schemes that determine whether to skip a certain inter-frame prediction mode.

[0176] Corresponding to the video encoding method provided in the embodiments of this application, the embodiments of this application also provide a video decoding method, which can be executed by any electronic device, such as a user terminal or a server, for example... Figure 6 As shown, the decoding method may include the following steps:

[0177] Step S610: Obtain the encoding result corresponding to the current block to be decoded.

[0178] Step S620: Determine the target prediction mode corresponding to the current block to be encoded by parsing the encoding result;

[0179] Step S630: Analyze the encoding result based on the target prediction mode to obtain the decoding result of the current block to be decoded.

[0180] It is understood that the current block to be decoded is an image block that needs to be decoded, which can be any video frame in the video that needs to be parsed. The encoding result corresponding to the current block to be decoded is the encoding result (encoded data, also known as the bitstream) of the image block before encoding, which is the block to be encoded. This encoding result can be obtained by encoding the block to be encoded using any of the encoding methods provided in the embodiments of this application.

[0181] Optionally, the encoding result of the current block to be decoded can be obtained by parsing the bitstream corresponding to the video to be decoded. This bitstream can be a real-time received bitstream, such as the bitstream transmitted in real-time during a video call or video conference, or the bitstream of the currently viewed video received from a video server. It can also be a non-real-time bitstream, such as the bitstream stored locally after the video has been encoded.

[0182] For any block to be decoded, i.e. the image block to be reconstructed, its corresponding encoding result should contain the relevant information required by the image block to be reconstructed. The encoding result may include the identifier of the target prediction mode corresponding to the block to be reconstructed. The decoder can know the prediction coding mode used during encoding based on the identifier. The encoding result should also include other information required for reconstruction. The encoder continues to parse the encoding result according to the target prediction mode and reconstructs the image block corresponding to the encoding result through steps including but not limited to entropy decoding, inverse quantization, inverse transformation, and prediction.

[0183] The solution provided in this application can be applied to any application scenario with encoding requirements. For encoders / electronic devices that support IBC prediction mode, this solution can quickly determine whether to skip IBC prediction mode, reduce the proportion of IBC prediction mode encoding complexity in the overall complexity of video encoding, thereby reducing encoding time and computational resource overhead, and better meeting actual needs.

[0184] To better understand and illustrate the practical value of the solutions provided in the embodiments of this application, the optional implementation methods of this application will be described below in conjunction with specific scenario embodiments.

[0185] Scenario 1: Screen sharing scenario

[0186] In a multi-person online meeting scenario, participant A can trigger screen / desktop sharing on their user terminal's application interface, and other participants' user terminals will then be able to see the participant's screen content. For example, in an online meeting, participant A can open a document on their terminal, and through screen sharing, all participants can see the document, facilitating communication and discussion.

[0187] Figure 7 This application provides a schematic diagram of the system structure for an online meeting screen sharing scenario, as illustrated in an embodiment of the present application. Figure 7 As shown, the system architecture in this scenario may include an online meeting server 10 and user terminals that communicate with the meeting server 10, such as... Figure 7 Terminal 1, ..., Terminal N in the diagram, wherein the user terminal can be any form of terminal capable of watching video, such as a user's mobile phone or computer, and the conference server 10 can be any server capable of providing users with applications including but not limited to online conferencing functions.

[0188] Assuming terminal 1 is the terminal of participant a who initiates screen sharing, and terminals 2, ..., N are the terminals of other participants, the application supports the video encoding and decoding method provided in this application embodiment. Optionally, in this application scenario, terminal 1 can be the encoding end, and other terminals can be the decoding ends. Participant a can operate on the document content displayed on their terminal 1, such as turning pages, marking, etc., and other participants on terminal N will also see the document content after the operation. During the online meeting, terminal 1 can encode the content displayed on terminal 1 by executing the encoding method provided in this application embodiment, generate a bitstream, and send the bitstream to the terminals of other participants, such as terminal N, through server 10. Terminal N decodes the received bitstream to reconstruct and display the image of the content displayed on terminal 1.

[0189] For the screen sharing application scenario described above, terminal 1 mainly needs to encode the screen content, and the IBC prediction mode is more suitable. In this case, terminal 1 can determine whether to skip the IBC prediction mode by executing the scheme provided in the embodiments of this application. However, if the skipping condition is not met, the IBC prediction mode can be used as the target prediction mode for encoding, and an identifier indicating that the target prediction mode is the IBC prediction mode will be carried in the encoding result. After receiving the bitstream, terminal N can use the prediction encoding mode indicated by the identifier to decode the bitstream and obtain the reconstructed image. Optionally, the screen sharing scenario can also be used as one of the application scenarios in the first scenario set described above. In this case, terminal 1 can quickly determine that the encoding process in the current scenario does not skip the IBC prediction mode based on the set. Optionally, if terminal 1 determines that the current encoding scenario is a screen sharing scenario, it can directly select the IBC prediction mode as the target prediction mode, which can improve the compression efficiency of the screen content.

[0190] Scenario 2: Live streaming or live streaming-like scenarios

[0191] Among them, "live-stream-like scenarios" refer to videos to be encoded that are similar to those in live-stream scenarios. For example, the video streams to be encoded are both video streams captured by a camera, such as a video call scenario implemented through a camera on a terminal. The following example is a real-time video call scenario.

[0192] Figure 8 This paper illustrates a system architecture diagram of a video call application scenario provided by an embodiment of this application, as shown below. Figure 8 As shown, application server 20 is a server for an application that provides video call functionality. The application can run on the first terminal 121 of object 1 and the second terminal 122 of object 2. Object 1 and object 2 can conduct real-time video calls through the video call service provided by application server 20. In this application scenario, each terminal participating in the video call is both an encoding end and a decoding end. The application also supports the encoding and decoding scheme provided in this application embodiment. Both the video encoding method and the decoding method need to be deployed on the first terminal 121 and the second terminal 122. After the first terminal 121 and the second terminal 122 establish a video call through application server 20, the first terminal 121 collects video from object 1 in real time and can encode the video frames in the collected video using the video encoding method provided in this application embodiment. The encoded bitstream is then sent to the second terminal 121 through application server 20. The second terminal 122 decodes the bitstream sent by the first terminal 121 to obtain the video image from object 1 and displays it to object 2. Similarly, after the second terminal 122 encodes the video stream collected from one side of object 2, it sends it to the first terminal 121 through the application server 20. The first terminal 121 decodes the bit stream and presents the video image from one side of object 2 to object 1.

[0193] For scenario 2 above, since the video streams that need to be encoded are all video streams of natural content captured in real time by the camera, the possibility of using the IBC prediction mode as the target prediction mode is not high. When each terminal encodes its captured video stream, the solution provided in the embodiments of this application can identify the blocks to be encoded that are not applicable to the IBC prediction mode, thereby skipping the prediction mode and avoiding the problem of increased encoding complexity caused by the prediction mode, thus speeding up the encoding efficiency. This also allows the other end to display the other party's video screen faster, improving the user experience.

[0194] Other scenarios

[0195] In practical applications, there are many other complex application scenarios. For example, in online meeting scenarios, some participants share their screens (e.g., a document), and one or more participants simultaneously enable video display. The participants' real-time video feeds can be displayed in a small window or other ways above the shared document content. In this scenario, the video frame to be encoded contains both screen content and natural content captured in real time by the camera. At this time, some blocks to be encoded in a single frame may be natural content image blocks, while others may be screen content image blocks. Using the solution provided in the embodiments of this application, by determining whether to skip the IBC mode, some blocks to be encoded can be encoded using the IBC prediction mode, while others can be encoded using other prediction modes. This ensures encoding performance while improving the overall encoding effect.

[0196] In order to further improve the encoding efficiency of video in a variety of application scenarios, Figure 9 The diagram shows a flowchart of an optional scheme for a video encoding method provided by this application. In this optional scheme, the second scenario set includes one or more encoding and decoding application scenarios that do not require the selection of IBC mode, such as the video call scenario in scenario 2 above. The videos in these scenarios are basically natural content videos captured in real time by the camera. The IBC mode can be skipped for the entire video encoding and decoding process. Inter-frame predictive coding or intra-frame predictive coding other than IBC mode can be selected according to the frame type.

[0197] The first scenario set includes one or more encoding / decoding application scenarios that are very likely to choose the IBC mode, such as the screen sharing scenario in scenario 1 above. For these application scenarios, it is not necessary to determine whether to skip the IBC mode. The IBC mode can be directly used as a candidate prediction mode. The IBC mode can be selected from the candidates based on the encoding cost or other strategies. Of course, if the scenario in the first scenario set is also a scenario that is determined to adopt the IBC mode, if it is determined that the current encoding / decoding application scenario belongs to the set, the IBC mode can be directly used for encoding.

[0198] If the current encoding / decoding application scenario is neither a scenario in the first scenario set nor a scenario in the second scenario set, such as the application scenario mentioned above that simultaneously involves screen sharing and real-time video display, these scenarios are relatively complex. The method provided in the previous embodiment can be used to determine whether to skip the IBC mode or not based on the inter-frame prediction effect and / or content features (color statistical features) of the current block. Based on the determination result, the current block can be encoded using the IBC mode or the non-IBC mode.

[0199] By differentiating between different application scenarios, the entire encoding process in certain specific / specified scenarios can skip the IBC mode, or the step of determining whether to skip the IBC mode can be omitted, thereby further accelerating the encoding speed in these specific scenarios and saving computing resources.

[0200] Based on the same principles as the video encoding method provided in this application, embodiments of this application also provide a video encoding apparatus, which can be implemented as any electronic device, such as... Figure 10 As shown, the video encoding device 100 may include a mode decision information determination module 110, a mode decision module 120, and an encoding module 130. Wherein:

[0201] The pattern decision information determination module 110 is used to determine the inter-frame prediction effect of the current block to be encoded;

[0202] The mode decision module 120 is used to skip the IBC prediction mode and adopt a prediction coding mode other than the IBC prediction mode as the target prediction mode of the current block to be coded when the inter-frame prediction effect meets the preset conditions.

[0203] The encoding module 130 is used to encode the current block to be encoded using the target prediction mode to obtain the encoding result of the current block to be encoded.

[0204] Optionally, the inter-frame prediction effect is characterized by at least one of the coding cost or the optimal inter-frame prediction mode of the current block to be coded, and the inter-frame prediction effect satisfies at least one of the following preset conditions:

[0205] The coding cost of inter-frame prediction is less than or equal to the set cost;

[0206] The optimal inter-frame prediction mode is one of the specified inter-frame prediction modes.

[0207] Optionally, the specified inter-frame prediction mode includes at least the inter-frame skip mode.

[0208] Optionally, the pattern decision information determination module can be used to: determine the content type of the current block to be encoded, wherein the content type is natural content or screen content;

[0209] The mode decision module can be used to: skip the IBC prediction mode and adopt a prediction coding mode other than the IBC prediction mode as the target prediction mode for the current block to be encoded when the content type of the current block to be encoded is natural content; or, skip the IBC prediction mode when the inter-frame prediction effect of the current block to be encoded meets the preset conditions and the content type of the current block to be encoded is natural content.

[0210] Optionally, the content type of the current block to be encoded is obtained in any of the following ways:

[0211] Based on the pixel values ​​of each pixel in the image content of the current block to be encoded, the color statistical features of the current block to be encoded are determined, and based on the color statistical features, the content type of the current block to be encoded is determined;

[0212] The content type of each block to be encoded in the video frame to which the current block to be encoded belongs is predicted by the trained classification model, and each block to be encoded includes the current block to be encoded.

[0213] Optionally, the trained classification model is obtained by training the initial classification model based on multiple training samples. Each training sample includes a sample image patch and a label for the sample image patch. The label of a sample image patch indicates whether the content type of the sample image patch is natural content or screen content.

[0214] The multiple training samples were obtained in the following way:

[0215] Obtain multiple sample image blocks; for each sample image block, determine the color statistical features of the image content of the sample image block, and obtain the label of the sample image block based on the color statistical features of the sample image block.

[0216] Optionally, the color statistical features include at least one of global statistical features or local statistical features. The global statistical features of the block to be processed are determined by the following method, wherein the block to be processed is the current block to be encoded or a sample image block:

[0217] Determine the pixel value of each pixel in the image content of the block to be processed;

[0218] For each pixel, determine the color category to which the pixel value belongs among a plurality of first color categories, wherein each first color category corresponds to a pixel value range;

[0219] Based on the color category to which each pixel belongs, the number of the first category of color categories appearing in the image content of the block to be processed is determined, and the global statistical feature includes the number of the first category.

[0220] Optionally, the content type of the block to be processed is determined in the following way:

[0221] If the number of the first category is not less than the first preset value or the number of the first category is not greater than the second preset value, the content type of the block to be processed is determined to be natural content, and the second preset value is less than the first preset value.

[0222] Optionally, the color statistical features include local statistical features, which are determined in the following ways:

[0223] The block to be processed is divided into multiple sub-blocks; for each sub-block, based on the pixel value of each pixel in the sub-block, the color category to which the pixel value of each pixel in the sub-block belongs in multiple second image categories is determined, and the number of second categories of color categories appearing in the sub-block is determined;

[0224] The local statistical features include the number of second categories corresponding to each sub-block.

[0225] Optionally, the content type of the block to be processed is determined in the following way:

[0226] The content type of the block to be processed is determined to be natural content if at least one of the following conditions is met:

[0227] The proportion of the corresponding second category number of sub-blocks in all sub-blocks is not less than the set value;

[0228] The number of sub-blocks in the corresponding second category that is not less than a set value is not less than a set number.

[0229] Optionally, the pattern decision information determination module can be used to: determine the block size of the block to be processed; if the block size is less than or equal to a set size, use the global statistical features as the color statistical features; if the block size is greater than the set size, the color statistical features include local statistical features.

[0230] Optionally, the inter-frame prediction effect satisfies preset conditions, including the optimal inter-frame prediction mode for the current block to be encoded being one of the specified inter-frame prediction modes.

[0231] Optionally, the target prediction mode of the current block to be encoded is determined in the following way:

[0232] If the inter-frame prediction effect meets the preset conditions, the best inter-frame prediction mode is taken as the target prediction mode.

[0233] Without skipping the IBC prediction mode, the IBC prediction mode is used as the target prediction mode, or the target prediction mode is determined from a first candidate mode set, wherein the first candidate mode set includes at least one prediction coding mode other than the specified inter-frame prediction mode and the IBC prediction mode.

[0234] When the step of determining the inter-frame prediction effect is performed when the content type of the current block to be encoded is determined to be screen content, if the content type of the current block to be encoded is natural content, the target prediction mode is determined from the second candidate mode set, wherein the second candidate mode set includes at least one prediction coding mode other than the IBC prediction mode.

[0235] Optionally, the pattern decision module can also be used to perform at least one of the following:

[0236] If the current encoding / decoding application scenario is a scenario in the first scenario set, then the process of determining whether to skip the IBC prediction mode is not performed. Instead, the target prediction mode of the current block to be encoded is determined from the candidate prediction coding set that includes the IBC prediction coding mode. The first scenario set includes at least one encoding / decoding application scenario.

[0237] If the current encoding / decoding application scenario is a scenario in the second scenario set, then skip the IBC prediction mode;

[0238] If the current block to be encoded is the background region of a video frame, then skip the IBC prediction mode.

[0239] Based on the same principle as the video encoding method provided in this application, embodiments of this application also provide a video decoding device, which can be implemented as any electronic device, such as... Figure 11 As shown, the video decoding device 200 may include an encoding result acquisition module 210 and a decoding module 220. Wherein:

[0240] The encoding result acquisition module 210 is used to acquire the encoding result corresponding to the current block to be decoded, wherein the encoding result is obtained by encoding the block to be encoded using the encoding method provided in the embodiments of this application;

[0241] The decoding module 220 is used to determine the target prediction mode corresponding to the current block to be decoded by parsing the encoding result, and to obtain the decoding result of the current block to be decoded by parsing the encoding result based on the target prediction mode.

[0242] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0243] This application also provides an electronic device including at least one processor configured to execute the steps of the method provided in any optional embodiment of this application. Optionally, the electronic device may further include a transceiver and / or a memory coupled to the processor, the memory storing a computer program, which the processor can use to implement the scheme provided in any optional embodiment of this application when running the computer program. Optionally, the electronic device may be a user terminal or a server.

[0244] Figure 12 A schematic diagram of the structure of an electronic device to which an embodiment of the present invention applies is shown, such as... Figure 12 As shown, the electronic device can be a standalone encoder (such as a general encoder), a standalone decoder (general decoder), or an electronic device that has both encoding and decoding functions. For example, the electronic device can be a server or a user terminal. The electronic device can be used to implement the method provided in any embodiment of the present invention.

[0245] like Figure 12 As shown, the electronic device 2000 may primarily include at least one processor 2001. Figure 12 The diagram shows components such as a memory 2002, a communication module 2003, and an input / output interface 2004. Optionally, these components can be connected and communicate with each other via a bus 2005. It should be noted that... Figure 12 The structure of the electronic device 2000 shown is merely illustrative and does not constitute a limitation on the electronic devices to which the methods provided in the embodiments of this application are applicable.

[0246] The memory 2002 can be used to store operating systems and applications, etc. The applications can include computer programs that implement the methods shown in the embodiments of the present invention when invoked by the processor 2001, and can also include programs for implementing other functions or services. The memory 2002 can be ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices that can store information and computer programs, or it can be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disk storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0247] Processor 2001 is connected to memory 2002 via bus 2005, and implements corresponding functions by calling application programs stored in memory 2002. Processor 2001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 2001 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0248] Electronic device 2000 can connect to a network via communication module 2003 (which may include, but is not limited to, components such as a network interface) to communicate with other devices (such as user terminals or servers) through the network and achieve data interaction, such as sending data to or receiving data from other devices. Communication module 2003 may include wired network interfaces and / or wireless network interfaces, meaning the communication module may include at least one of wired or wireless communication modules.

[0249] Electronic device 2000 can connect to required input / output devices, such as keyboards and display devices, via input / output interface 2004. Electronic device 2000 itself may have a display device, and other display devices can also be connected externally via interface 2004. Optionally, storage devices, such as hard drives, can also be connected via interface 2004 to store data from electronic device 2000, retrieve data from storage devices, or store data from storage devices into memory 2002. It is understood that input / output interface 2004 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to input / output interface 2004 can be a component of electronic device 2000 or an external device connected to electronic device 2000 when needed.

[0250] The bus 2005 used to connect the various components may include a pathway for transmitting information between these components. The bus 2005 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Depending on its function, the bus 2005 can be divided into address bus, data bus, control bus, etc.

[0251] Optionally, for the solution provided in the embodiments of the present invention, the memory 2002 can be used to store a computer program that executes the solution of the present invention, and the processor 2001 runs the computer program. When the processor 2001 runs the computer program, it implements the operation of the method or apparatus provided in the embodiments of the present invention.

[0252] Based on the same principle as the method provided in the embodiments of this application, the embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the corresponding content of the aforementioned method embodiments.

[0253] This application also provides a computer program product, which includes a computer program that, when executed by a processor, can implement the corresponding content of the aforementioned method embodiments.

[0254] It should be noted that the terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0255] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0256] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A video encoding method, characterized in that, The method includes: Determine the inter-frame prediction effect of the current block to be encoded and / or determine the content type of the current block to be encoded, wherein the content type is natural content or screen content; If the inter-frame prediction effect meets the preset conditions and / or the content type is natural content, then skip the intra-block copy (IBC) prediction mode and use a prediction coding mode other than the IBC prediction mode as the target prediction mode for the current block to be coded. The target prediction mode is used to encode the current block to be encoded to obtain the encoding result of the current block to be encoded; The inter-frame prediction effect is characterized by at least one of the coding cost or the best inter-frame prediction mode of the current block to be coded, and the inter-frame prediction effect satisfies at least one of the following preset conditions: The coding cost of inter-frame prediction is less than or equal to the set cost; The optimal inter-frame prediction mode is one of the specified inter-frame prediction modes.

2. The method according to claim 1, characterized in that, The specified inter-frame prediction mode includes at least the inter-frame skip mode.

3. The method according to claim 1, characterized in that, The content type of the current block to be encoded is obtained through any of the following methods: Based on the pixel values ​​of each pixel in the image content of the current block to be encoded, the color statistical features of the current block to be encoded are determined, and based on the color statistical features, the content type of the current block to be encoded is determined; The content type of each block to be encoded in the video frame to which the current block to be encoded belongs is predicted by the trained classification model, and each block to be encoded includes the current block to be encoded.

4. The method according to claim 3, characterized in that, The trained classification model is obtained by training an initial classification model based on multiple training samples. Each training sample includes a sample image patch and a label for the sample image patch. The label of a sample image patch indicates whether the content type of the sample image patch is natural content or screen content. The multiple training samples were obtained in the following way: Acquire multiple sample image patches; For each sample image block, determine the color statistical features of the image content of the sample image block, and obtain the label of the sample image block based on the color statistical features of the sample image block.

5. The method according to claim 3 or 4, characterized in that, The color statistical features include at least one of global statistical features or local statistical features, and the global statistical features of the block to be processed are determined in the following way: Determine the pixel value of each pixel in the image content of the block to be processed, wherein the block to be processed is the current block to be encoded or a sample image block; For each pixel, determine the color category to which the pixel value belongs among a plurality of first color categories, wherein each first color category corresponds to a first pixel value range; Based on the first color category to which each pixel belongs, determine the number of first categories of the first color category appearing in the image content of the block to be processed, and the global statistical features include the number of first categories; The content type of the block to be processed is determined in the following way: If the number of the first category is not less than the first preset value or the number of the first category is not greater than the second preset value, the content type of the block to be processed is determined to be natural content, and the second preset value is less than the first preset value.

6. The method according to claim 5, characterized in that, The color statistical features include local statistical features, which are determined in the following ways: The block to be processed is divided into multiple sub-blocks; For each sub-block, based on the pixel value of each pixel in the sub-block, determine the color category to which the pixel value of each pixel in the sub-block belongs among multiple second color categories, and determine the number of second categories of color categories appearing in the sub-block; each second color category corresponds to a second pixel value range; The local statistical features include the number of second categories corresponding to each sub-block; The content type of the block to be processed is determined in the following way: The content type of the block to be processed is determined to be natural content if at least one of the following conditions is met: The proportion of the corresponding second category number of sub-blocks in all sub-blocks is not less than the set value and is not less than the set proportion. The number of sub-blocks in the corresponding second category that is not less than the set value is not less than the set quantity.

7. The method according to claim 5, characterized in that, The method further includes: Determine the block size of the block to be processed; If the block size is less than or equal to the set size, the global statistical feature will be used as the color statistical feature; If the block size is larger than the set size, the color statistical features include local statistical features.

8. The method according to claim 1, characterized in that, The inter-frame prediction effect meets the preset conditions, including: the best inter-frame prediction mode of the current block to be encoded is one of the specified inter-frame prediction modes; The target prediction mode of the current block to be encoded is determined in the following way: If the inter-frame prediction effect meets the preset conditions, the best inter-frame prediction mode is taken as the target prediction mode. Without skipping the IBC prediction mode, the IBC prediction mode is used as the target prediction mode, or the target prediction mode is determined from a first candidate mode set, wherein the first candidate mode set includes at least one prediction coding mode other than the specified inter-frame prediction mode and the IBC prediction mode. When the step of determining the inter-frame prediction effect is performed when the content type of the current block to be encoded is determined to be screen content, if the content type of the current block to be encoded is natural content, the target prediction mode is determined from the second candidate mode set, wherein the second candidate mode set includes at least one prediction coding mode other than the IBC prediction mode.

9. The method according to any one of claims 1 to 4, characterized in that, The method further includes at least one of the following: If the current encoding / decoding application scenario is a scenario in the first scenario set, then the process of determining whether to skip the IBC prediction mode is not performed. Instead, the target prediction mode of the current block to be encoded is determined from the candidate prediction coding mode set including the IBC prediction coding mode. The first scenario set includes at least one encoding / decoding application scenario. If the current encoding / decoding application scenario is a scenario in the second scenario set, then skip the IBC prediction mode; If the current block to be encoded is the background region of a video frame, then skip the IBC prediction mode.

10. A video decoding method, characterized in that, The method includes: Obtain the encoding result corresponding to the current block to be decoded, wherein the encoding result is obtained by encoding the block to be decoded using the method described in any one of claims 1 to 9; By analyzing the encoding results, the target prediction mode corresponding to the current block to be decoded is determined; Based on the target prediction pattern, the encoding result is parsed to obtain the decoding result of the current block to be decoded.

11. A video encoding device, characterized in that, The encoding device includes: The pattern decision information determination module is used to determine the inter-frame prediction effect of the current block to be encoded and / or determine the content type of the current block to be encoded, wherein the content type is natural content or screen content; The mode decision module is used to skip the IBC prediction mode and adopt a prediction coding mode other than the IBC prediction mode as the target prediction mode of the current block to be encoded when the inter-frame prediction effect meets the preset conditions and / or the content type is natural content. The encoding module is used to encode the current block to be encoded using the target prediction mode to obtain the encoding result of the current block to be encoded. The inter-frame prediction effect is characterized by at least one of the coding cost or the best inter-frame prediction mode of the current block to be coded, and the inter-frame prediction effect satisfies at least one of the following preset conditions: The coding cost of inter-frame prediction is less than or equal to the set cost; The optimal inter-frame prediction mode is one of the specified inter-frame prediction modes.

12. The apparatus according to claim 11, characterized in that, The specified inter-frame prediction mode includes at least the inter-frame skip mode.

13. The apparatus according to claim 11, characterized in that, The content type of the current block to be encoded is obtained through any of the following methods: Based on the pixel values ​​of each pixel in the image content of the current block to be encoded, the color statistical features of the current block to be encoded are determined, and based on the color statistical features, the content type of the current block to be encoded is determined; The content type of each block to be encoded in the video frame to which the current block to be encoded belongs is predicted by the trained classification model, and each block to be encoded includes the current block to be encoded.

14. The apparatus according to claim 13, characterized in that, The trained classification model is obtained by training an initial classification model based on multiple training samples. Each training sample includes a sample image patch and a label for the sample image patch. The label of a sample image patch indicates whether the content type of the sample image patch is natural content or screen content. The multiple training samples were obtained in the following way: Acquire multiple sample image patches; For each sample image block, determine the color statistical features of the image content of the sample image block, and obtain the label of the sample image block based on the color statistical features of the sample image block.

15. The apparatus according to claim 13 or 14, characterized in that, The color statistical features include at least one of global statistical features or local statistical features, and the global statistical features of the block to be processed are determined in the following way: Determine the pixel value of each pixel in the image content of the block to be processed, wherein the block to be processed is the current block to be encoded or a sample image block; For each pixel, determine the color category to which the pixel value belongs among a plurality of first color categories, wherein each first color category corresponds to a first pixel value range; Based on the first color category to which each pixel belongs, determine the number of first categories of the first color category appearing in the image content of the block to be processed, and the global statistical features include the number of first categories; The content type of the block to be processed is determined in the following way: If the number of the first category is not less than the first preset value or the number of the first category is not greater than the second preset value, the content type of the block to be processed is determined to be natural content, and the second preset value is less than the first preset value.

16. The apparatus according to claim 15, characterized in that, The color statistical features include local statistical features, which are determined in the following ways: The block to be processed is divided into multiple sub-blocks; For each sub-block, based on the pixel value of each pixel in the sub-block, determine the color category to which the pixel value of each pixel in the sub-block belongs among multiple second color categories, and determine the number of second categories of color categories appearing in the sub-block; each second color category corresponds to a second pixel value range; The local statistical features include the number of second categories corresponding to each sub-block; The content type of the block to be processed is determined in the following way: The content type of the block to be processed is determined to be natural content if at least one of the following conditions is met: The proportion of the corresponding second category number of sub-blocks in all sub-blocks is not less than the set value and is not less than the set proportion. The number of sub-blocks in the corresponding second category that is not less than the set value is not less than the set quantity.

17. The apparatus according to claim 15, characterized in that, The pattern decision information determination module is also used for: Determine the block size of the block to be processed; If the block size is less than or equal to the set size, the global statistical feature will be used as the color statistical feature; If the block size is larger than the set size, the color statistical features include local statistical features.

18. The apparatus according to claim 11, characterized in that, The inter-frame prediction effect meets the preset conditions, including: the best inter-frame prediction mode of the current block to be encoded is one of the specified inter-frame prediction modes; The target prediction mode of the current block to be encoded is determined in the following way: If the inter-frame prediction effect meets the preset conditions, the best inter-frame prediction mode is taken as the target prediction mode. Without skipping the IBC prediction mode, the IBC prediction mode is used as the target prediction mode, or the target prediction mode is determined from a first candidate mode set, wherein the first candidate mode set includes at least one prediction coding mode other than the specified inter-frame prediction mode and the IBC prediction mode. When the step of determining the inter-frame prediction effect is performed when the content type of the current block to be encoded is determined to be screen content, if the content type of the current block to be encoded is natural content, the target prediction mode is determined from the second candidate mode set, wherein the second candidate mode set includes at least one prediction coding mode other than the IBC prediction mode.

19. The apparatus according to any one of claims 11 to 14, characterized in that, The pattern decision module is also used to perform at least one of the following: If the current encoding / decoding application scenario is a scenario in the first scenario set, then the process of determining whether to skip the IBC prediction mode is not performed. Instead, the target prediction mode of the current block to be encoded is determined from the candidate prediction coding mode set including the IBC prediction coding mode. The first scenario set includes at least one encoding / decoding application scenario. If the current encoding / decoding application scenario is a scenario in the second scenario set, then skip the IBC prediction mode; If the current block to be encoded is the background region of a video frame, then skip the IBC prediction mode.

20. A video decoding device, characterized in that, The decoding device includes: The encoding result acquisition module is used to acquire the encoding result corresponding to the current block to be decoded, wherein the encoding result is obtained by encoding the block to be encoded using the method described in any one of claims 1 to 9; The decoding module is used to determine the target prediction mode corresponding to the current block to be decoded by parsing the encoding result, and to obtain the decoding result of the current block to be decoded by parsing the encoding result based on the target prediction mode.

21. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the method according to any one of claims 1 to 10 when running the computer program.

22. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 10.

Images