Screen content coding method and device, equipment, storage medium and product

Abstract

The embodiment of the invention provides a screen content coding method and device, equipment, a storage medium and a product. According to the technical scheme provided by the embodiment of the invention, the to-be-coded key frame is divided into the plurality of image blocks, the pure color blocks, the simple blocks and the screen blocks are determined from the plurality of image blocks according to the number of color types of the brightness components in each image block, and the proportion information of the pure color blocks, the simple blocks and the screen blocks in the to-be-coded key frame is determined; determining the frame type of the to-be-coded key frame according to the proportion information, and under the condition that the frame type of the to-be-coded key frame is a screen frame, performing a coding mode decision comprising a screen content coding mode on the to-be-coded key frame, according to the coding mode decision result, the to-be-coded key frame and the subsequent non-key frame are coded to obtain the coding result, the coding mode decision including the screen content coding mode decision is accurately carried out on the screen frame, the coding efficiency and the image compression quality of the screen image are effectively improved, and the image coding quality is improved.

Description

Technical Field

[0001] Embodiments of the present application relate to image coding technology, and more particularly to a screen content coding method, apparatus, device, storage medium and product. Background Art

[0002] With the development of Internet technology and computer technology, screen content (SC) including computer-generated graphics, texts and animations is becoming more and more popular and has more and more applications in daily life.

[0003] In order to meet people's demand for high image quality and high performance and to achieve efficient coding and compression of screen content videos, the Screen Content Coding (SCC) standard was proposed based on the High Efficiency Video Coding (HEVC) standard and the content characteristics of screen content videos. Screen content coding has added screen content coding modes such as intra-frame block copy (IBC), palette mode (PLT), and adaptive multiple core transform (AMT) to address the characteristics of screen content videos, such as intra-frame content repetition, limited number of colors in coding blocks, and edge sharpening, which significantly improves the coding efficiency of screen content.

[0004] Since screen content encoding tools often only have good benefits for compressing screen content images, the traditional encoding method of turning on the screen content encoding mode for all images is prone to performance loss and load increase, and the image encoding quality is poor. Summary of the invention

[0005] The embodiments of the present application provide a screen content encoding method, device, equipment, storage medium and product to solve the technical problem of poor image encoding quality in related technologies, which can effectively improve the encoding efficiency and image compression quality of screen images and improve the image encoding quality.

[0006] In a first aspect, an embodiment of the present application provides a screen content encoding method, comprising:

[0007] Divide the key frame to be encoded into a plurality of image blocks, and determine a pure color block, a simple block, and a screen block from the plurality of image blocks according to the number of color types of brightness components in each of the image blocks;

[0008] Determine the proportion information of the pure color block, the simple block and the screen block in the key frame to be encoded, and determine the frame type of the key frame to be encoded according to the proportion information;

[0009] In a case where the frame type of the key frame to be encoded is a screen frame containing screen content, performing a coding mode decision including a screen content coding mode on the key frame to be encoded, wherein the coding mode decision includes a screen content coding mode decision;

[0010] The key frame to be encoded and subsequent non-key frames are encoded according to the encoding mode decision result to obtain an encoding result.

[0011] In a second aspect, an embodiment of the present application provides a screen content encoding device, including a block analysis module, a type analysis module, a mode decision module and an encoding processing module, wherein:

[0012] The block analysis module is configured to divide the key frame to be encoded into a plurality of image blocks, and determine a pure color block, a simple block and a screen block from the plurality of image blocks according to the number of color types of the brightness components in each of the image blocks;

[0013] The type analysis module is configured to determine the proportion information of the pure color block, the simple block and the screen block in the key frame to be encoded, and determine the frame type of the key frame to be encoded according to the proportion information;

[0014] The mode decision module is configured to, when the frame type of the key frame to be encoded is a screen frame containing screen content, perform a coding mode decision including a screen content coding mode on the key frame to be encoded, wherein the coding mode decision includes a screen content coding mode decision;

[0015] The encoding processing module is configured to perform encoding processing on the key frame to be encoded and subsequent non-key frames according to the encoding mode decision result to obtain an encoding result.

[0016] In a third aspect, an embodiment of the present application provides a screen content encoding device, including: a memory and one or more processors;

[0017] The memory is used to store one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the screen content encoding method as described in the first aspect.

[0019] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the screen content encoding method as described in the first aspect when executed by a computer processor.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium, and at least one processor of a device reads and executes the computer program from the computer-readable storage medium, so that the device performs the screen content encoding method of the first aspect.

[0021] The embodiment of the present application divides the key frame to be encoded into multiple image blocks, and determines the pure color blocks, simple blocks and screen blocks from the multiple image blocks according to the number of color types of the brightness components in each image block, determines the proportion information of the pure color blocks, simple blocks and screen blocks in the key frame to be encoded, and determines the frame type of the key frame to be encoded according to the proportion information, and in the case that the frame type of the key frame to be encoded is a screen frame containing screen content, performs a coding mode decision including a screen content coding mode on the key frame to be encoded, and encodes the key frame to be encoded and subsequent non-key frames according to the coding mode decision result to obtain a coding result, by accurately determining the frame type of the key frame to be encoded according to the proportion information of the pure color blocks, simple blocks and screen blocks in the key frame to be encoded, and accurately performs a coding mode decision including a screen content coding mode decision on the screen frame, the coding efficiency and image compression quality of the screen image are effectively improved, and the image coding quality is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a screen content encoding method provided by an embodiment of the present application;

[0023] Figure 2 is a flowchart of another screen content encoding method provided by an embodiment of the present application;

[0024] Figure 3 is a structural diagram of a screen content encoding device provided in an embodiment of the present application;

[0025] Figure 4 It is a structural diagram of a screen content encoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0027] The screen content encoding method provided in the present application can be applied to the detection of screen frames and image encoding based on the screen content encoding mode. It aims to accurately determine the frame type of the key frame to be encoded based on the proportion of pure color blocks, simple blocks and screen blocks in the key frame to be encoded, and accurately make encoding mode decisions for the screen frames including screen content encoding mode decisions, so as to effectively improve the encoding efficiency and image compression quality of the screen image and improve the image encoding quality.

[0028] In the existing image coding scheme, considering the actual application of screen content coding, most image coding scenarios cannot predetermine whether the sequence to be coded is a screen sequence, and often can only use one encoder and one set of coding parameters to be compatible with screen content scenarios and ordinary content scenarios. In general, the encoder (such as the SVT-AV1 encoder) first performs screen frame detection on the frame to be coded, divides each frame into several 16x16 blocks, and counts the number of luminance component pixels in each block. When the number of pixels is between 2 and 4, the block is considered to belong to the screen block. When the screen block of a frame accounts for more than 10% of all 16x16, the frame is determined to be a screen frame and the screen content coding tool is turned on. However, this screen frame detection method can only detect about 20% of the screen frames, resulting in the screen content coding tool not being turned on, and the image coding compression efficiency is low. If the screen frame detection threshold is simply increased, many ordinary frames will be detected as screen frames, which increases unnecessary coding complexity and may also cause loss of coding quality. Moreover, when static sequences, pure color sequences, and simple sequences are detected as screen sequences, since the contents of these sequences are too simple, the SKIP mode of inter-frame prediction will generally be selected after mode decision, and the screen content encoding mode will not be selected. Therefore, the time of analyzing the screen algorithm of these sequences is wasted, and the image encoding quality and efficiency are poor. Based on this, a screen content encoding method of an embodiment of the present application is provided to solve the technical problems that the existing screen content encoding has low detection accuracy for screen frames and poor image encoding quality.

[0029] Figure 1 A flowchart of a screen content encoding method provided in an embodiment of the present application is given. The screen content encoding method provided in an embodiment of the present application can be executed by a screen content encoding device, which can be implemented in hardware and / or software and integrated in a screen content encoding device.

[0030] The following description is made by taking the screen content encoding method executed by the screen content encoding device as an example. Figure 1 , the screen content encoding method includes:

[0031] S110: Divide the key frame to be encoded into a plurality of image blocks, and determine a pure color block, a simple block, and a screen block from the plurality of image blocks according to the number of color types of brightness components in each image block.

[0032] In one embodiment, when it is necessary to encode a sequence to be encoded, the frame type of the key frame to be encoded (I frame) in each group of pictures (GOP) in the sequence to be encoded is detected, and it is determined whether to enable the screen content encoding tool for the key frame to be encoded according to the frame type detection result. Wherein, for non-key frames (such as B frames / P frames), it can be determined whether to enable the screen content encoding tool for the non-key frames according to the frame type detection result of the key frame to be encoded in the same group of pictures (wherein, the screen content encoding tool enabled for the non-key frames can be the same as the screen content encoding tool enabled for the key frame to be encoded, or it can be different), so as to improve the image encoding efficiency while ensuring the image encoding quality.

[0033] Exemplarily, the key frame to be encoded is divided into multiple image blocks of preset sizes, and pure color blocks, simple blocks and screen blocks are determined from the multiple image blocks according to the number of color types of the brightness component in each image block. The range of the number of color types of the corresponding brightness component can be set according to the color characteristics corresponding to the pure color block, the simple block and the screen block, wherein the pure color block can be understood as an image block with a single color and / or brightness, and the number of color types corresponding to the brightness component can be 1, the simple block can be understood as an image block with a relatively single color and / or brightness combination, and the number of color types corresponding to the brightness component can be 2, and the screen block can be understood as an image block with a color and / or brightness combination that meets the characteristics of the screen content, and the number of color types corresponding to the brightness component can be 3 to 5.

[0034] For example, the number of types of brightness components contained in each image block is detected. If there is only one type, it may belong to the background content of an ordinary image and not a screen content block, so the image block can be considered a pure color block; if there are more types (for example, 2), it does not meet the characteristics of a single color of the screen sequence content, and the image block can be considered a simple block; if the number of types is within the range of the number of types that meet the characteristics of a single color of the screen sequence content (for example, 3 to 5), the image block can be considered to meet the characteristics of the screen sequence. Optionally, when the image block meets the characteristics of the screen sequence, the variance of the brightness value of the image block can be further calculated. If the variance of the brightness value of the image block is greater than a preset threshold, it is considered that the brightness pixel of the image block tends to be discontinuous and meets the characteristics of the screen image; and if the variance of the brightness value of the image block is small, it can be considered that the image block may be a simple background block of an ordinary image, and the screen content encoding strategy is not used. In one embodiment, the total number of pure color blocks, simple blocks and screen blocks is less than or equal to the total number of image blocks. For example, when there are image blocks with a large number of types of brightness components (for example, more than 5), these image blocks do not belong to pure color blocks, simple blocks and screen blocks. At this time, the total number of pure color blocks, simple blocks and screen blocks is less than the total number of image blocks.

[0035] In one embodiment, considering that the characteristics of screen image content are relatively complex, a block of size 16*16 cannot reflect the content of the current block in detail and accurately. Based on this, this solution can set the preset size of the image block to 8*8, and divide each key frame to be encoded into several 8*8 image blocks for traversal. The image block can reflect the content of the current block in detail and accurately, and more accurately determine the image block type of each image block.

[0036] S120: Determine the proportion information of the pure color block, the simple block and the screen block in the key frame to be encoded, and determine the frame type of the key frame to be encoded according to the proportion information.

[0037] Exemplarily, after determining pure color blocks, simple blocks, and screen blocks from multiple image blocks, the proportion information of the pure color blocks, simple blocks, and screen blocks in the key frame to be encoded is calculated respectively, and the frame type of the key frame to be encoded is determined according to the proportion information of the pure color blocks, simple blocks, and screen blocks. The frame type includes a screen frame and a non-screen frame.

[0038] For example, when all image blocks in the key frame to be encoded are simple blocks (the proportion of simple blocks is 1 at this time), the proportion of pure color blocks reaches a set proportion (for example, in the range of greater than 0.4 to 0.8), or the sum of the proportions of screen blocks and simple blocks / or the weighted sum of the proportions of screen blocks and simple blocks reaches a preset proportion (for example, in the range of corresponding proportions greater than 0.05 to 0.1), or the sum of the proportions of screen blocks and pure color blocks / or the weighted sum of the proportions of screen blocks and pure color blocks reaches a preset proportion (for example, in the range of corresponding proportions greater than 0.05 to 0.1), the frame type of the key frame to be encoded can be considered to be a screen frame containing screen content (SC, Screen Content), otherwise the frame type of the key frame to be encoded is considered to be a non-screen frame.

[0039] Among them, the screen content can be computer-generated graphics, texts, animations, etc., and can be encoded using screen content coding tools (SCC tools) such as intra-frame block copying, palette, and adaptive transform kernel to effectively improve the coding efficiency, targeting the characteristics of intra-frame content repetition, limited number of colors in the coding block, and edge sharpening in the image frames containing the screen content.

[0040] S130: When the frame type of the key frame to be encoded is a screen frame including screen content, a coding mode decision including a screen content coding mode is performed on the key frame to be encoded.

[0041] Exemplarily, when the frame type of the key frame to be encoded is a screen frame containing screen content, a coding mode decision including a screen content coding mode may be made for the key frame to be encoded based on a pre-set coding mode decision strategy. The screen content coding mode may include an intra block copy mode (IBC) and / or a palette mode (PLT). In addition to the screen content coding mode, the coding mode decision may also include other intra coding modes, inter coding modes, hybrid coding modes, etc. When a coding mode decision including a screen content coding mode is made for the key frame to be encoded, the coding mode decision result of the coding unit (CU) may be a screen content coding mode or other pre-set image coding modes.

[0042] In a possible embodiment, when the key frame to be encoded is a non-screen frame, the screen content encoding tool for the key frame to be encoded can be turned off, and a coding mode decision that does not include the screen content coding mode is made for the key frame to be encoded. Subsequently, the key frame to be encoded and the non-key frames in the same image group as the key frame to be encoded can be encoded according to the coding mode decision result that does not include the screen content coding mode to obtain the encoding result.

[0043] S140: encoding the key frame to be encoded and subsequent non-key frames according to the encoding mode decision result to obtain an encoding result.

[0044] Exemplarily, after a coding mode decision including a screen content coding mode is made for a key frame to be encoded, the key frame to be encoded is encoded according to the coding mode decision result to obtain a key frame encoding result, and multiple non-key frames following the key frame to be encoded in the same image group are encoded according to the coding mode decision result to obtain a non-key frame encoding result.

[0045] Among them, when the coding mode decision result is the screen content coding mode, according to the content characteristics of the screen content video, that is, the repeated content in the key frame to be encoded, the limited number of colors in the coding block, the edge sharpening and other characteristics, the screen content coding tools such as intra-frame block copy, palette, adaptive transform kernel, etc. are used to encode the key frame to be encoded to obtain the key frame encoding result, which can effectively improve the encoding quality and encoding efficiency.

[0046] This solution can accurately distinguish between ordinary frames and screen frames based on the proportion of pure color blocks, simple blocks and screen blocks in the key frames to be encoded, and only enable the screen encoding algorithm for screen frames to reduce the occurrence of screen frame misjudgment and missed judgment. For image frames that are confirmed to be screen frames, more screen encoding tools are enabled to make the encoding efficiency higher. The encoding modes of frames that have been encoded and detected as screen frames can be counted to reduce the situation where ordinary frames are misjudged as screen frames, thereby improving image encoding efficiency while ensuring image encoding quality.

[0047] In one embodiment, it can be determined whether to enable a preset parameter set for multiple non-key frames after the key frame to be encoded in the same image group according to the result of the coding mode decision, and the preset parameter set is used to configure the coding processing method for the non-key frame. For example, when the result of the coding mode decision is the screen content coding mode, it can be determined to enable the preset parameter set, and when the result of the coding mode decision is the non-screen content coding mode, it can be determined to disable the preset parameter set. It can also be that when the proportion of the target coding tree unit in the key frame to be encoded to all coding tree units reaches a preset proportion threshold, the preset parameter set is enabled, and when the proportion of the target coding tree unit in the key frame to be encoded to all coding tree units does not reach the preset proportion threshold, the preset parameter set is applied. The target coding tree unit can be a coding tree unit whose proportion corresponding to the coding unit using the screen content coding mode as the target coding mode reaches a preset proportion, and whether to use the screen content coding mode as the target coding mode can be determined according to the result of the coding mode decision.

[0048] The preset parameter set provided by the present solution records the encoding processing methods for the image, such as pre-processing, motion estimation, reference frame selection, etc. These encoding processing methods can effectively improve the encoding efficiency of the screen frame. Among them, the preset parameter set can be used to configure the encoding parameters of the non-key frames in the image group after the key frame to be encoded, such as the corresponding configuration parameters of pre-processing, motion estimation, and reference frame selection. That is, when the non-key frame in the same image group as the key frame to be encoded is encoded, when the preset parameter set is enabled, the screen encoding method recorded in the preset parameter set is configured for the non-key frame with the corresponding encoding parameters, and the non-key frame is encoded based on the encoding parameters. When the preset parameter set is disabled, the screen encoding method recorded in the preset parameter set is not encoded for the non-key frame, and other preset encoding methods applicable to non-screen frames can be used for encoding to obtain the encoding result of the non-key frame.

[0049] When it is determined to enable the preset parameter set, when the encoding mode decision result is the screen content encoding mode, the non-key frame subsequent to the key frame to be encoded is encoded, and the preset parameter set can be enabled to encode the non-key frame to obtain the non-key frame encoding result. The encoding of the non-key frame can be based on the preset parameter set and encoded according to the difference between the current non-key frame and the previous key frame or non-key frame.

[0050] In the above, by dividing the key frame to be encoded into multiple image blocks, and determining the pure color block, simple block and screen block from the multiple image blocks according to the number of color types of the brightness component in each image block, determining the proportion information of the pure color block, simple block and screen block in the key frame to be encoded, and determining the frame type of the key frame to be encoded according to the proportion information, in the case that the frame type of the key frame to be encoded is a screen frame containing screen content, a coding mode decision including a screen content coding mode is performed on the key frame to be encoded, and the key frame to be encoded and subsequent non-key frames are encoded according to the coding mode decision result to obtain a coding result, by accurately determining the frame type of the key frame to be encoded according to the proportion information of the pure color block, simple block and screen block in the key frame to be encoded, and accurately performing a coding mode decision including a screen content coding mode decision on the screen frame, the coding efficiency of the screen image and the image compression quality are effectively improved, and the image coding quality is improved.

[0051] Based on the above embodiments, Figure 2 A flowchart of another screen content encoding method provided by an embodiment of the present application is given, and the screen content encoding method is a specific embodiment of the above screen content encoding method. Figure 2 , the screen content encoding method includes:

[0052] S210: Divide the key frame to be encoded into a plurality of image blocks, and determine a pure color block, a simple block, and a screen block from the plurality of image blocks according to the number of color types of brightness components in each image block.

[0053] In a possible embodiment, the screen content encoding method provided by the present solution determines a pure color block, a simple block, and a screen block from a plurality of image blocks according to the number of color types of brightness components in each image block, including:

[0054] S211: Determine the number of color types of the brightness component in each image block.

[0055] S212: When the number of color types is within a first number range, determine that the image block is a pure color block.

[0056] S213: When the number of color types is within the second number range, determine that the image block is a simple block.

[0057] S214: when the number of color types is within a third number range, determine the brightness variance of the image block, and when the brightness variance is greater than a preset variance threshold, determine that the image block is a screen block.

[0058] Exemplarily, for the key frames to be encoded in each image group in the sequence to be encoded, the key frames to be encoded are divided into multiple image blocks, and the number of color types of the brightness components in these image blocks is determined. For example, when the number of color types of brightness is 256 (for example, the numerical range corresponding to brightness is 0 to 255), the number of color types contained in the brightness component of each image block is counted. For example, for an 8*8 image block, the minimum number of color types is 1 (that is, the brightness values ​​of each pixel in the image block are the same), and the maximum number of color types is 64 (that is, the brightness values ​​of each pixel in the image block are different).

[0059] In one embodiment, after determining the number of color types of the brightness component in the image block, determine the preset number range corresponding to the number of color types, and determine the type of the image block according to the corresponding preset number range. The preset number range includes a first number range, a second number range, and a third number range, and the numerical values ​​corresponding to the first number range, the second number range, and the third number range increase in sequence. For example, the first number range is 1, the second number range is 2, and the third number range is 3 to 5. Then, when the number of color types is 1, the number of color types is in the first number range; when the number of color types is 2, the number of color types is in the second number range; when the number of color types is 2 to 5, the number of color types is in the third number range.

[0060] Exemplarily, when the number of color types is within a first number range, the image block can be determined to be a pure color block. When the number of color types is within a second number range, the image block can be determined to be a simple block. When the number of color types is within a third number range, whether the image block is a screen block can be further determined based on the brightness variance of the image block.

[0061] When the number of color types is within the third number range, the brightness variance of the image block is calculated based on the brightness values ​​of each pixel in the image block, and the brightness variance is compared with a preset variance threshold. When the brightness variance is greater than the preset variance threshold, it is considered that the image block meets the characteristics of uneven color distribution of the screen block, and the image block can be determined as a screen block.

[0062] This scheme accurately determines pure color blocks, simple blocks and screen blocks from multiple image blocks according to the number range corresponding to the number of color types of brightness components in the image blocks and the brightness variance of the image blocks, and more accurately determines the distribution of pure color blocks, simple blocks and screen blocks in the key frames to be encoded, thereby improving the detection accuracy of screen frames and improving the encoding quality of screen content.

[0063] S220: Determine the proportion information of the pure color block, the simple block and the screen block in the key frame to be encoded, and determine the frame type of the key frame to be encoded according to the proportion information.

[0064] In a possible embodiment, the screen content encoding method provided by the present solution determines the proportion information of the pure color block, the simple block and the screen block in the key frame to be encoded, including:

[0065] S221: Determine a first number of pure color blocks, a second number of simple blocks, a third number of screen blocks, and an image block number of image blocks.

[0066] S222: Determine the proportion of pure color blocks according to the first number and the number of image blocks.

[0067] S223: Determine the simple block ratio according to the second number and the number of image blocks.

[0068] S224: Determine the screen block proportion according to the third number and the number of image blocks.

[0069] Exemplarily, after determining the pure color blocks, simple blocks and screen blocks from multiple image blocks, the corresponding numbers of the pure color blocks, simple blocks, screen blocks and image blocks are counted, that is, the first number of pure color blocks, the second number of simple blocks, the third number of screen blocks and the number of image blocks of the image blocks.

[0070] The number of image blocks of the image block can be determined according to the size of the image block, for example, by dividing the product of the width and height of the key frame to be encoded by the product of the width and height of the image block to obtain the number of image blocks. For example, the number of image blocks can be determined based on the following formula:

[0071] blocksNum=height*width / pixNumInblock

[0072] pixNumInblock=blk_w*blk_h

[0073] Among them, blocksNum is the number of image blocks, height is the height of the key frame to be encoded, width is the width of the key frame to be encoded, pixNumInblock is the size of the image block, blk_w is the width of the image block, and blk_h is the height of the image block.

[0074] In one embodiment, the proportion information in the key frame to be encoded provided by this solution includes the proportion of pure color blocks, the proportion of simple blocks and the proportion of screen blocks. The ratio of the first number to the number of image blocks can be used as the proportion of pure color blocks relative to the key frame to be encoded; the ratio of the second number to the number of image blocks can be used as the proportion of simple blocks relative to the key frame to be encoded; the ratio of the third number to the number of image blocks can be used as the proportion of screen blocks relative to the key frame to be encoded. This solution accurately determines the proportion of pure color blocks, the proportion of simple blocks and the proportion of screen blocks according to the first number of pure color blocks, the second number of simple blocks, the third number of screen blocks and the number of image blocks of image blocks, and accurately determines the distribution of pure color blocks, simple blocks and screen blocks on the frame to be encoded, thereby improving the detection accuracy of screen frames and the encoding quality of screen content.

[0075] In a possible embodiment, the screen content encoding method provided by the present scheme determines the frame type of the key frame to be encoded according to the proportion information, which may be: when at least one of the simple block proportion is a first preset proportion, the pure color block proportion is greater than the second preset proportion, and the sum of the product of the simple block proportion and the preset ratio and the screen block proportion is greater than the third preset proportion, the frame type of the key frame to be encoded is determined to be a screen frame.

[0076] After determining the proportion of pure color blocks, the proportion of simple blocks and the proportion of screen blocks, determine whether the proportion of simple blocks is the first preset proportion or greater than the first preset proportion (the value range of the first preset proportion can be 0.9-1, for example, the first preset proportion is 1), whether the proportion of pure color blocks is greater than the second preset proportion (the value range of the second preset proportion can be 0.4-0.8, for example, the second preset proportion is 0.6), and determine whether the product of the simple block proportion and the preset ratio (the value range of the preset ratio can be 0.1-0.4, for example, the preset ratio is 0.15) and the sum of the screen block proportion is greater than the third preset ratio (the value range of the third preset ratio can be 0.05-0.1, for example, the third preset ratio is 0.075). When at least one of the above conditions is met, the frame type of the key frame to be encoded can be considered to be a screen frame containing screen content. When all three conditions are not met, the frame type of the key frame to be encoded can be considered to be a non-screen frame (ordinary frame).

[0077] For example, the pure color block proportion p1, simple block proportion p2 and screen block proportion p3 can be expressed as:

[0078] p1=r1 / blocksNum

[0079] p2=r2 / blocksNum

[0080] p3=r3 / blocksNum

[0081] Wherein, r1 is the first number, r2 is the second number, r3 is the third number, and blocksNum is the number of image blocks. When the following conditions are met, the frame type of the key frame to be encoded is a screen frame containing screen content:

[0082] sc_class=(p2==a1)||(p1>a2)||((p3+k*p2)>a3)

[0083] Among them, sc_class is the frame type information, when c_class=1, the frame type is a screen frame, a1 is the first preset proportion, a2 is the second preset proportion, a3 is the third preset proportion, and k is the preset ratio. This solution accurately determines whether the frame type of the key frame to be encoded is a screen frame by satisfying whether the simple block proportion is the first preset proportion, whether the pure color block proportion is greater than the second preset proportion, and whether the sum of the product of the simple block proportion and the preset proportion and the screen block proportion is greater than the third preset proportion, thereby improving the detection accuracy of the screen frame and improving the encoding quality of the screen content.

[0084] S230: When the frame type of the key frame to be encoded is a screen frame including screen content, a coding mode decision including a screen content coding mode is performed on the key frame to be encoded.

[0085] S240: Determine, according to the coding mode decision result, a plurality of coding tree units of the key frame to be encoded, use the screen content coding mode as a target coding unit of the target coding mode, and determine the target coding tree unit from the plurality of coding tree units according to the number of the target coding units.

[0086] S250: Determine a first proportion of the target coding tree unit in the key frame to be encoded, and when the first proportion is greater than a fourth preset proportion, enable a preset parameter set, which is used to configure encoding parameters of non-key frames in the image group after the key frame to be encoded.

[0087] Exemplarily, after a coding mode decision including a screen content coding mode is made for a key frame to be encoded, the screen content coding mode (including intra block copy IBC and / or palette mode PLT) can be used as a target coding unit (CU) of the target coding mode in multiple coding tree units (CTUs) of the key frame to be encoded according to the coding mode decision result, and the number of target coding units can be counted.

[0088] In one embodiment, the target coding tree unit may be determined from a plurality of coding tree units according to the above-stated number of target coding units, wherein the target coding tree unit may be a coding tree unit whose number or proportion is greater than a set number or proportion.

[0089] Determine the number of target coding tree units determined above, calculate the ratio of the number of target coding tree units and the number of coding tree units divided by the number of target coding tree units, and determine the ratio as a first proportion of the target coding tree units in the key frame to be encoded. Determine whether the first proportion is greater than a fourth preset proportion (for example, the fourth preset proportion is 0.2). When the first proportion is greater than the fourth preset proportion, it is considered that the key frame to be encoded has a strong screen characteristic, and the preset parameter set can be enabled; and when the first proportion is less than or equal to the fourth preset proportion, the preset parameter set can be disabled.

[0090] When the first proportion of the target coding tree unit in the key frame to be encoded is greater than the fourth preset proportion, this scheme can determine that the key frame to be encoded has a strong screen characteristic, and the preset parameter set can be enabled to perform screen encoding processing on non-key frames, effectively improving the encoding efficiency of the screen.

[0091] In one embodiment, the screen content encoding method provided by the present scheme determines the target coding tree unit from multiple coding tree units according to the number of target coding units, which may be: determining the second proportion of the target coding unit in the corresponding coding tree unit; when the second proportion is greater than the fifth preset proportion, determining the coding tree unit as the target coding tree unit.

[0092] Exemplarily, after determining the target coding unit that takes the screen content coding mode as the target coding mode, for each coding tree unit, the ratio of the number of target coding units in the coding tree unit to the number of coding units in the coding tree unit is determined, and the ratio is used as the second proportion of the target coding unit in the corresponding coding tree unit.

[0093] In one embodiment, it is determined whether the second proportion is greater than the fifth preset proportion (for example, the fifth preset proportion is 0.1). When the second proportion is greater than the fifth preset proportion, the coding tree unit can be determined as the target coding tree unit, and it is considered that the current frame type judgment of the key frame to be encoded is accurate, and the frame type determination of the subsequent key frames to be encoded can continue, as well as the coding mode decision based on the frame type. This solution accurately determines the target coding tree unit according to the comparison result of the second proportion of the target coding unit in the coding tree unit and the fifth preset proportion, accurately enables the preset parameter set to perform screen coding processing on non-key frames, and effectively improves the coding efficiency of the screen.

[0094] S260: When the first proportion is less than the fifth preset proportion, determine to stop determining the frame type of a subsequent preset number of key frames, and turn off the screen content encoding mode decision for the subsequent preset number of key frames.

[0095] Exemplarily, after determining the first proportion of the target coding tree unit in the key frame to be encoded, it is determined whether the first proportion is greater than the fifth preset proportion. When the first proportion is less than the fifth preset proportion, it can be considered that the current key frame to be encoded has too few blocks with screen characteristics, and the ordinary image column may be mistakenly judged as a screen image. Due to the continuity of the video, the video characteristics will not change drastically in a short time. It can be determined to stop determining the frame type of a subsequent preset number of key frames (for example, 3-5 key frames), and to turn off the screen content coding mode decision for the subsequent preset number of key frames, thereby improving image coding efficiency.

[0096] S270: Encoding the key frame to be encoded and subsequent non-key frames according to the encoding mode decision result to obtain an encoding result.

[0097] In the above, by dividing the key frame to be encoded into multiple image blocks, and determining the pure color block, simple block and screen block from the multiple image blocks according to the number of color types of the brightness component in each image block, determining the proportion information of the pure color block, simple block and screen block in the key frame to be encoded, and determining the frame type of the key frame to be encoded according to the proportion information, in the case that the frame type of the key frame to be encoded is a screen frame containing screen content, a coding mode decision including a screen content coding mode is performed on the key frame to be encoded, and the key frame to be encoded and subsequent non-key frames are encoded according to the coding mode decision result to obtain a coding result, by accurately determining the frame type of the key frame to be encoded according to the proportion information of the pure color block, simple block and screen block in the key frame to be encoded, and accurately performing a coding mode decision including a screen content coding mode decision on the screen frame, the coding efficiency of the screen image and the image compression quality are effectively improved, and the image coding quality is improved. At the same time, when the first proportion of the target coding tree unit in the key frame to be encoded is greater than the fourth preset proportion, it can be determined that the key frame to be encoded has a stronger screen characteristic, and the preset parameter set can be enabled to perform screen encoding processing on non-key frames, effectively improving the encoding efficiency of the screen.

[0098] Figure 3 is a structural diagram of a screen content encoding device provided by an embodiment of the present application. Figure 3 The screen content encoding device includes a block analysis module 31, a type analysis module 32, a mode decision module 33 and an encoding processing module 34.

[0099] Among them, the block analysis module 31 is configured to divide the key frame to be encoded into multiple image blocks, and determine the pure color block, simple block and screen block from the multiple image blocks according to the number of color types of the brightness component in each image block; the type analysis module 32 is configured to determine the proportion information of the pure color block, simple block and screen block in the key frame to be encoded, and determine the frame type of the key frame to be encoded according to the proportion information; the mode decision module 33 is configured to, when the frame type of the key frame to be encoded is a screen frame containing screen content, make a coding mode decision including a screen content coding mode for the key frame to be encoded; the coding processing module 34 is configured to perform coding processing on the key frame to be encoded and subsequent non-key frames according to the coding mode decision result to obtain a coding result.

[0100] In the above, by dividing the key frame to be encoded into multiple image blocks, and determining the pure color block, simple block and screen block from the multiple image blocks according to the number of color types of the brightness component in each image block, determining the proportion information of the pure color block, simple block and screen block in the key frame to be encoded, and determining the frame type of the key frame to be encoded according to the proportion information, in the case that the frame type of the key frame to be encoded is a screen frame containing screen content, a coding mode decision including a screen content coding mode is performed on the key frame to be encoded, and the key frame to be encoded and subsequent non-key frames are encoded according to the coding mode decision result to obtain a coding result, by accurately determining the frame type of the key frame to be encoded according to the proportion information of the pure color block, simple block and screen block in the key frame to be encoded, and accurately performing a coding mode decision including a screen content coding mode decision on the screen frame, the coding efficiency of the screen image and the image compression quality are effectively improved, and the image coding quality is improved.

[0101] In a possible embodiment, the block analysis module 31 determines the pure color block, the simple block and the screen block from the multiple image blocks according to the number of color types of the brightness components in each image block, and is configured as follows:

[0102] Determining the number of color types of the luminance component in each image block;

[0103] When the number of color types is within a first number range, determining that the image block is a pure color block;

[0104] When the number of color types is within a second number range, determining the image block as a simple block;

[0105] When the number of color types is within a third number range, the brightness variance of the image block is determined, and when the brightness variance is greater than a preset variance threshold, the image block is determined to be a screen block.

[0106] In a possible embodiment, the type analysis module 32 determines the proportion information of pure color blocks, simple blocks and screen blocks in the key frame to be encoded, and is configured as follows:

[0107] determining a first number of pure color blocks, a second number of simple blocks, a third number of screen blocks, and a number of image blocks of the image block;

[0108] Determine the proportion of pure color blocks according to the first number and the number of image blocks;

[0109] Determine the proportion of simple blocks according to the second number and the number of image blocks;

[0110] The screen block proportion is determined according to the third number and the number of image blocks.

[0111] In a possible embodiment, the type analysis module 32 determines the frame type of the key frame to be encoded according to the proportion information, and is configured as follows:

[0112] When at least one of the following conditions is met: the proportion of simple blocks is the first preset proportion, the proportion of pure color blocks is greater than the second preset proportion, and the sum of the product of the simple block proportion and the preset ratio and the screen block proportion is greater than the third preset proportion, the frame type of the key frame to be encoded is determined to be a screen frame.

[0113] In a possible embodiment, the screen content encoding device further includes a parameter set configuration module, and the parameter set configuration module is configured as follows:

[0114] Determine, according to the coding mode decision result, a plurality of coding tree units of the key frame to be encoded, use the screen content coding mode as a target coding unit of the target coding mode, and determine the target coding tree unit from the plurality of coding tree units according to the number of the target coding units;

[0115] Determine a first proportion of the target coding tree unit in the key frame to be encoded, and when the first proportion is greater than a fourth preset proportion, enable a preset parameter set, which is used to configure encoding parameters of non-key frames in the image group after the key frame to be encoded.

[0116] In a possible embodiment, the screen content encoding device further includes a decision management module, and the decision management module is configured to:

[0117] When the first proportion is less than the fifth preset proportion, it is determined to stop determining the frame type of a subsequent preset number of key frames, and to close the screen content encoding mode decision for the subsequent preset number of key frames.

[0118] In a possible embodiment, the parameter set configuration module determines the target coding tree unit from the plurality of coding tree units according to the number of the target coding units, and is configured as follows:

[0119] Determining a second proportion of the target coding unit in the corresponding coding tree unit;

[0120] When the second proportion is greater than the fifth preset proportion, the coding tree unit is determined as the target coding tree unit.

[0121] It is worth noting that in the embodiment of the above-mentioned screen content encoding device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0122] The embodiment of the present application also provides a screen content encoding device, which can integrate the screen content encoding apparatus provided in the embodiment of the present application. Figure 4 is a structural diagram of a screen content encoding device provided by an embodiment of the present application. Figure 4 , the screen content encoding device is configured as: an input device 43, an output device 44, a memory 42 and one or more processors 41; the memory 42 is used to store one or more programs; when one or more programs are executed by one or more processors 41, the one or more processors 41 implement the screen content encoding method provided in the above embodiment. The screen content encoding device, equipment and computer provided above can be used to execute the screen content encoding method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0123] The embodiment of the present application also provides a non-volatile storage medium storing computer executable instructions, which are used to execute the screen content encoding method provided in the above embodiment when executed by a computer processor. Of course, the non-volatile storage medium storing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the screen content encoding method provided above, can also execute the related operations in the screen content encoding method provided in any embodiment of the present application. The screen content encoding device, equipment and storage medium provided in the above embodiments can execute the screen content encoding method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the screen content encoding method provided in any embodiment of the present application.

[0124] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium, including a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the screen content encoding method provided in each embodiment of the present application.

Claims

1. A screen content encoding method, characterized in that: include: Divide the key frame to be encoded into a plurality of image blocks, and determine a pure color block, a simple block, and a screen block from the plurality of image blocks according to the number of color types of brightness components in each of the image blocks; Determine the proportion information of the pure color block, the simple block and the screen block in the key frame to be encoded, and determine the frame type of the key frame to be encoded according to the proportion information; In a case where the frame type of the key frame to be encoded is a screen frame containing screen content, performing a coding mode decision including a screen content coding mode on the key frame to be encoded; The key frame to be encoded and subsequent non-key frames are encoded according to the encoding mode decision result to obtain an encoding result.

2. The screen content encoding method according to claim 1, characterized in that: The step of determining a pure color block, a simple block, and a screen block from the plurality of image blocks according to the number of color types of the brightness components in each of the image blocks comprises: Determining the number of color types of brightness components in each of the image blocks; When the number of color types is within a first number range, determining that the image block is a pure color block; When the number of color types is within a second number range, determining that the image block is a simple block; When the number of color types is within a third number range, a brightness variance of the image block is determined, and when the brightness variance is greater than a preset variance threshold, the image block is determined to be a screen block.

3. The screen content encoding method according to claim 1, characterized in that: The determining of the proportion information of the pure color block, the simple block and the screen block in the key frame to be encoded includes: determining a first number of the pure color blocks, a second number of the simple blocks, a third number of the screen blocks, and a number of image blocks of the image blocks; Determine the proportion of pure color blocks according to the first number and the number of image blocks; Determine the proportion of simple blocks according to the second number and the number of image blocks; The screen block ratio is determined according to the third number and the number of image blocks.

4. The screen content encoding method according to claim 3, characterized in that: The determining the frame type of the key frame to be encoded according to the proportion information includes: Under the condition that at least one of the following conditions is met: the proportion of simple blocks is the first preset proportion, the proportion of pure color blocks is greater than the second preset proportion, and the sum of the product of the proportion of simple blocks and the preset ratio and the proportion of screen blocks is greater than the third preset proportion, the frame type of the key frame to be encoded is determined to be a screen frame.

5. The screen content encoding method according to claim 1, characterized in that: After the coding mode decision including the screen content coding mode is made for the key frame to be coded, the method further includes: Determine, according to the coding mode decision result, a plurality of coding tree units of the key frame to be encoded, use the screen content coding mode as a target coding unit of the target coding mode, and determine the target coding tree unit from the plurality of coding tree units according to the number of the target coding units; Determine a first proportion of the target coding tree unit in the key frame to be encoded, and when the first proportion is greater than a fourth preset proportion, enable a preset parameter set, wherein the preset parameter set is used to configure encoding parameters of non-key frames in the image group that follow the key frame to be encoded.

6. The screen content encoding method according to claim 5, characterized in that: After determining the first proportion of the target coding tree unit in the key frame to be encoded, the method further includes: When the first proportion is less than the fifth preset proportion, it is determined to stop determining the frame type of a subsequent preset number of key frames, and to close the screen content encoding mode decision for the subsequent preset number of key frames.

7. The screen content encoding method according to claim 5, characterized in that: The determining a target coding tree unit from a plurality of the coding tree units according to the number of the target coding units comprises: Determining a second proportion of the target coding unit in the corresponding coding tree unit; When the second proportion is greater than a fifth preset proportion, the coding tree unit is determined as a target coding tree unit.

8. A screen content encoding device, characterized in that: It includes a block analysis module, a type analysis module, a mode decision module and an encoding processing module, among which: The block analysis module is configured to divide the key frame to be encoded into a plurality of image blocks, and determine a pure color block, a simple block and a screen block from the plurality of image blocks according to the number of color types of the brightness components in each of the image blocks; The type analysis module is configured to determine the proportion information of the pure color block, the simple block and the screen block in the key frame to be encoded, and determine the frame type of the key frame to be encoded according to the proportion information; The mode decision module is configured to make a coding mode decision including a screen content coding mode for the key frame to be encoded when the frame type of the key frame to be encoded is a screen frame containing screen content; The encoding processing module is configured to perform encoding processing on the key frame to be encoded and subsequent non-key frames according to the encoding mode decision result to obtain an encoding result.

9. A screen content encoding device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the screen content encoding method according to any one of claims 1 to 7.

10. A non-volatile storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute the screen content encoding method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the screen content encoding method according to any one of claims 1 to 7 is implemented.

Images