Screen content encoding method, device, equipment and medium

Abstract

This application belongs to the field of video coding technology and discloses a screen content encoding method, apparatus, device, and medium. The method includes screening target reference blocks that match a target coding unit to be encoded in a screen content image from all reference blocks in an encoded area of ​​the screen content image, and performing screen content encoding based on the target coding unit and the corresponding target reference blocks. This reduces data computation overhead, expands the effective search area of ​​the target coding unit, increases the number of reference blocks, and improves the quality of screen content encoding.

Description

Technical Field

[0001] The present application relates to the field of screen content encoding, and in particular to a screen content encoding method, apparatus, device and medium. Background Art

[0002] Screen content images are different from images captured by traditional cameras. They are generated by electronic devices and captured from the image display units of various devices (such as computers and mobile terminals). Examples include computer graphics and text, hybrid images combining natural images with graphics and text, and computer-generated animations.

[0003] In order to improve the compression efficiency of screen content coding, high efficiency video coding (HEVC) and screen content coding (Screen Content Coding, SCC) came into being.

[0004] In HEVC SCC encoding, a frame of screen content image is usually divided into multiple coding tree units (CTUs), which are then divided into multiple coding units (CUs). The screen content is encoded using a block-level coding mode (IntraBlock Copy, IBC). The so-called IBC technology is to determine the matching target reference block in the valid search area of ​​the target coding unit to be processed. The valid search area and invalid search area of ​​the target coding unit are determined according to the HEVC SCC standard.

[0005] However, in the existing IBC technology, it is necessary to determine whether the reference block in the screen content image is located in the effective search area of ​​the target coding unit, which has a high computational overhead, and the number of reference blocks in the effective search area is small, resulting in poor rate-distortion (RD) quality of the screen content image.

[0006] Therefore, a new technical solution is needed to optimize IBC to reduce computing overhead and improve the encoding quality of screen content. Summary of the Invention

[0007] The embodiments of the present application provide a screen content encoding method, apparatus, device, and medium for optimizing IBC when using IBC for screen content encoding, reducing computational overhead, and improving screen content encoding quality.

[0008] In one aspect, a method for encoding screen content is provided, comprising:

[0009] Filtering, from the reference blocks within the encoded area of ​​the screen content image, a target reference block matching the target coding unit to be encoded in the screen content image;

[0010] Screen content encoding is performed based on the target coding unit and the target reference block.

[0011] In one aspect, a screen content encoding device is provided, comprising:

[0012] A matching unit, configured to select, from the reference blocks in the encoded area of ​​the screen content image, a target reference block that matches a target coding unit to be encoded in the screen content image;

[0013] The processing unit is configured to perform screen content encoding based on the target coding unit and the target reference block.

[0014] Preferably, the matching unit is used to:

[0015] Obtain pixel information of the target coding unit;

[0016] For each reference block in the one-way linked list, perform the following steps until a reference block whose image similarity meets the preset matching conditions is determined:

[0017] Obtain pixel information of the reference block, and determine image similarity between the target coding unit and the reference block based on the pixel information of the target coding unit and the pixel information of the reference block, and if the image similarity is not lower than a first similarity threshold, determine that the reference block is a reference block that meets the preset matching condition.

[0018] Preferably, the matching unit is further used for:

[0019] When it is determined that there is no reference block whose similarity is not lower than the first similarity threshold, a reference block that meets the preset matching condition is determined based on the position distance and corresponding image similarity between the obtained target coding unit and each reference block.

[0020] On the one hand, a control device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of any one of the above-mentioned screen content encoding methods when executing the program.

[0021] In one aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned screen content encoding methods are implemented.

[0022] In a screen content encoding method, apparatus, device, and medium provided by embodiments of the present application, a target reference block matching a target coding unit to be encoded in a screen content image is screened out from all reference blocks within the encoded region of the screen content image, and screen content encoding is performed based on the target coding unit and the corresponding target reference block. This reduces position calculation overhead, expands the effective search region of the target coding unit, increases the number of reference blocks, and improves the quality of screen content encoding. Furthermore, if the target reference block is located within the invalid search region of the target coding unit, the search path length for the target reference block can be shortened, reducing computational complexity.

[0023] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 This is a schematic diagram of an application scenario in the implementation manner of this application;

[0026] Figure 2 A schematic diagram of a HEVC SCC encoding process in the prior art;

[0027] Figure 3 This is a flowchart of a method for encoding screen content in an embodiment of the present application;

[0028] Figure 4a A schematic diagram of the principle of quadtree partitioning in an embodiment of the present application;

[0029] Figure 4b This is a schematic diagram of a CTU in an embodiment of the present application;

[0030] Figure 4c A schematic diagram of a multi-threaded screen content encoding method in the prior art;

[0031] Figure 4d This is a schematic diagram of a search area division in an embodiment of the present application;

[0032] Figure 5 This is a flowchart of a method for determining a target reference block in an embodiment of the present application;

[0033] Figure 6A schematic diagram of multi-threaded screen content encoding in an embodiment of the present application;

[0034] Figure 7 This is a structural diagram of a screen content encoding device in an embodiment of the present application;

[0035] Figure 8 This is a structural diagram of a control device in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and beneficial effects of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0037] First, some of the terms involved in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0038] Terminal device: can be a mobile terminal, fixed terminal or portable terminal, such as a mobile phone, station, unit, device, multimedia computer, multimedia tablet, Internet node, communicator, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, personal communication system device, personal navigation device, personal digital assistant, audio / video player, digital camera / camcorder, positioning device, television receiver, radio broadcast receiver, e-book device, gaming device or any combination thereof, including accessories and peripherals of these devices or any combination thereof. It is also foreseeable that the terminal device can support any type of user interface (such as wearable device), etc.

[0039] Server: It can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms.

[0040] IBC: It is a block-level coding mode. At the encoding end, block matching technology is used to find the best matching block for each current CU and calculate the block vector between the current block and the best matching block.

[0041] Inter-frame prediction: Find the position that best matches the current block from the reference frame, referring to information in the time domain, including motion estimation (ME) and motion compensation (MC).

[0042] Motion estimation: Determine the best corresponding image block of the CU to be encoded in the encoded image (reference frame) and calculate the offset (motion vector) of the corresponding block.

[0043] Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network that provides these resources is called the "cloud." To users, these resources appear infinitely scalable and can be accessed at any time, used on demand, expanded at any time, and paid for on a per-use basis.

[0044] Cloud storage is a new concept that has evolved from the concept of cloud computing. A distributed cloud storage system (hereafter referred to as a storage system) is a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (also known as storage nodes) on a network through application software or application interfaces to work together and provide data storage and service access. Currently, storage systems create logical volumes. When creating a logical volume, physical storage space is allocated for each logical volume. This physical storage space may consist of disks on a single storage device or several storage devices. When a client stores data on a logical volume, it stores the data on a file system. The file system divides the data into multiple parts, each of which is an object. An object contains not only the data but also additional information such as the data identifier (ID). The file system writes each object to the physical storage space of the logical volume and records the storage location of each object. When a client requests data access, the file system can provide access based on the storage location information of each object.

[0045] The storage system allocates physical storage space to logical volumes by pre-dividing the physical storage space into stripes based on the estimated capacity of the objects to be stored in the logical volume (this estimate usually has a large margin relative to the actual capacity of the objects to be stored) and the Redundant Array of Independent Disks (RAID) groupings. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0046] The following introduces the design concept of the embodiment of this application.

[0047] To improve the compression efficiency of screen content encoding, HEVC SCC was developed. In HEVC SCC encoding, the encoder generates a corresponding one-way linked list based on the image hash value of the target coding unit. The one-way linked list is created based on the reference blocks within the coded area of ​​the CTU. Each reference block in a one-way linked list has the same image hash value.

[0048] It should be noted that in actual applications, a one-way linked list (including a one-way linked list) is formed by linking the positions of each reference block. In the embodiment of the present application, for ease of explanation, the reference blocks corresponding to each position in the one-way linked list are expressed as the reference blocks contained in the one-way linked list.

[0049] Then, the encoder determines whether each reference block in the one-way linked list is located in the valid search area of ​​the target coding unit, until a target reference block matching the target coding unit is obtained.

[0050] Since the deeper the one-way linked list, the greater the computational overhead of determining whether the reference block in the one-way linked list is within the valid search area, and the number of reference blocks within the valid search area is limited, the encoding quality of the screen content is poor.

[0051] Obviously, conventional technologies do not provide a technical solution that can optimize IBC in HEVC SCC, reduce computational overhead, and improve the quality of screen content coding. Therefore, a screen content coding technical solution is needed to optimize IBC to reduce computational overhead and improve screen content coding quality.

[0052] The applicant carefully analyzed the principles for dividing the effective search area and the invalid search area in conventional techniques and found that the criteria for determining the effective search area are set under the ideal premise that the number of encoding threads is infinite. The purpose is to use the effective search area to avoid affecting multi-threaded parallel processing. However, in actual applications, the number of encoding threads is usually limited. If the reference blocks in the invalid search area are located in the encoded area, they will not affect the parallel processing of the encoding threads. Therefore, the effective search area can be expanded to include the encoded area.

[0053] In view of the above analysis and considerations, an embodiment of the present application provides a screen content encoding scheme, in which, among all reference blocks in the encoded area of ​​the screen content image, target reference blocks that match the target coding unit to be encoded in the screen content image are screened out, and screen content encoding is performed based on the target coding unit and the corresponding target reference block.

[0054] To further illustrate the technical solutions provided by the embodiments of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of the present application provide the method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiments of the present application. During the actual processing process or when the control device is executed, the method can be executed in the order of the methods shown in the embodiments or drawings or in parallel.

[0055] See Figure 1 The following is a schematic diagram of an application scenario. Figure 1 This section describes an application scenario related to screen content encoding.

[0056] like Figure 1 The network shown includes a terminal device 10A, a terminal device 10B, a network 20 and a server 30 , wherein the terminal device 10A and the terminal device 10B are communicatively connected to the server 30 via the network 20 .

[0057] Figure 1 In this scenario, a user using terminal device 10A wants to push a video they are currently watching to terminal device 10B. In this scenario, terminal device 10A acts as the encoding end, and terminal device 10B acts as the decoding end. Terminal device 10A needs to encode the video to obtain an encoded video stream, which it then uploads to server 30. Server 30 then forwards the video stream to terminal device 10B, which decodes the video stream to enable normal playback of the video on terminal device 10B.

[0058] See Figure 2The figure shows a flow chart of HEVC SCC encoding under the prior art. In the HEVC SCC encoding process, a frame of screen content image is read from the frame buffer and sent to the encoder, and first undergoes intra-frame or inter-frame prediction to obtain a prediction value. After obtaining the prediction value, the prediction value is subtracted from the input data to obtain a residual, and then a discrete cosine transform (DCT) is performed and quantized to obtain a residual coefficient, which is then sent to the entropy coding module to output the bitstream. At the same time, the residual coefficient is inversely quantized and inversely transformed to obtain the residual value of the reconstructed image, which is then added to the prediction value obtained after intra-frame or inter-frame prediction to obtain a reconstructed image. The reconstructed image is then filtered in the loop and enters the reference frame queue as the reference image of the next frame, thereby encoding one frame backward. Among them, the in-loop filtering may include deblocking filtering (DBF) and sample adaptive offset (SAO).

[0059] Among them, intra-frame prediction uses IBC to find a matching target reference block for each target coding unit to be encoded, and calculates the block vector between the positions of the two, referring to the information in the spatial domain.

[0060] Since IBC technology requires determining whether each reference block is within the effective search area of ​​the target coding unit, the computational complexity and overhead are high, and the number of reference blocks within the effective search area is limited, resulting in poor IBC RD quality, which in turn leads to poor screen content encoding quality. Therefore, an embodiment of the present application provides a screen content encoding scheme to reduce computational complexity and overhead, and improve screen content encoding quality.

[0061] In the embodiments of the present application, the execution entity may be a control device for encoding screen content. The control device may be a server or a terminal device. Optionally, the control device may be a cloud server providing cloud storage and cloud computing. The screen content encoding scheme provided in the embodiments of the present application is applicable to scenarios where HEVC SCC is used to encode screen content images.

[0062] See Figure 3 The figure shows a flowchart of an implementation method of screen content encoding provided by this application. The specific process of the method is as follows:

[0063] Step 300: Control the device to obtain a target coding unit to be encoded in a screen content image.

[0064] In an embodiment of the present application, the screen content image is an image generated by an electronic device, which is captured from an image display unit of various devices (such as computers, mobile terminals, etc.). It can be a frame image in a video or a separate image.

[0065] The screen content image can be a text document, slideshow, webpage, or game screen. It can be color-coded using any of the YUV420, YUV422, or YUV444 formats. "Y" represents luminance (Luminance or Luma), or grayscale values; "U" and "V" represent chrominance (Chroma), which describes the color and saturation of the image and is used to specify the color of a pixel.

[0066] The target coding unit is a CU in the screen content image. The size of the target coding unit can be 8×8, or can be set according to the actual application scenario, which is not limited here.

[0067] In one embodiment, the control device first divides the screen content image into multiple coding tree unit (CTU) blocks according to a preset maximum CU size, and then divides each CTU block multiple times using a quadtree cyclic hierarchical structure until a preset minimum CU size is reached.

[0068] See Figure 4a The figure shows a schematic diagram of the principle of quadtree segmentation. The specific segmentation process of the quadtree is marked with two variables: depth (Depth) and split flag (Split_flag). The largest CU (LCU) can be 64×64 in size and 0 in depth, represented by CU0. CU0 can be divided into four CU1s of size 32×32 and depth 1, and so on, until it can be divided into CU3 of depth 3 and cannot be divided any further. For CUd of size 2N*2N and depth d, if its split_flag value is 0, CUd is no longer divided; otherwise, it is divided into four CUd+1s of size N×N and depth d+1.

[0069] It should be noted that the reference block is an image block of the same size (ie, the same dimensions) as the target coding unit. The reference block is obtained differently from the CU, and each reference block is generated based on each reference point in the screen content image.

[0070] In the embodiment of the present application, only the target reference block matching a target coding unit is determined as an example for explanation. Based on similar principles, the target reference blocks of other target coding units in the screen content image can be determined, which will not be repeated here.

[0071] See Figure 4b , which is a schematic diagram of a CTU. Figure 4b In the CTU, a CTU includes multiple CUs of different sizes. The control device uses a CU to be encoded in the CTU as a target coding unit.

[0072] In this way, the target coding unit to be encoded can be obtained.

[0073] Step 301: The control device selects a target reference block matching a target coding unit to be encoded in the screen content image from reference blocks in the encoded area of ​​the screen content image.

[0074] Specifically, the reference blocks in the coded area are all reference blocks in the coded area in the screen content image.

[0075] For example, see Figure 4b As shown, the reference blocks in the gray area (ie, the reference blocks in the first three rows and the reference block before the target coding unit in the fourth row) are all reference blocks in the coded area.

[0076] This is because the traditional technology sets the division criteria of the effective search area and the invalid search area under the ideal premise that the number of encoding threads is infinite. However, considering that in actual applications, the number of threads for encoding screen content is limited, there may also be target reference blocks matching the target encoding unit in the invalid search area. Therefore, in the embodiment of the present application, the effective search area is expanded.

[0077] See Figure 4c As shown, it is a schematic diagram of multi-threaded screen content encoding under a conventional technology. The control device uses 4 encoding threads to encode the CUs in the first to fourth rows respectively. The first thread currently encodes the 9th CU in the first row, the second thread currently encodes the 8th CU in the second row, the third thread currently encodes the 7th CU in the third row, and the fourth thread currently encodes the CU in the 6th coding block in the fourth row. In traditional technology, in order to avoid affecting the parallel processing of encoding threads, if the CU processed by the N+1th thread is the M+1th, the serial number of the CU processed by the Nth thread is the Mth. Therefore, when the CU in the 6th coding block in the fourth row is the target coding unit, the effective search area of ​​the target coding unit is Figure 4c Furthermore, in the conventional technology, when determining the target reference block that the target coding unit matches, the Figure 4c In the direction indicated by the arrow, each reference block in the one-way linked list is judged in turn whether it is located in the valid search area. Figure 4c The reference blocks in the non-gray area (i.e., the invalid search area) are found. The deeper the linked list is, the greater the computational overhead of the central processing unit (CPU). Obviously, this will consume a lot of system resources and reduce the efficiency of screen content encoding.

[0078] Considering that in actual applications, the number of threads for screen content encoding is limited, and there may be target reference blocks matching the target coding unit in the invalid search area, the embodiment of the present application improves the traditional IBC technology by expanding the valid search area to all reference blocks in the encoded area. Figure 4d As shown in the figure, it is a schematic diagram of search area division. In the traditional IBC technology, Figure 4d The gray area in the figure is used as the effective search area, and the black area is used as the invalid search area. Figure 4d The gray and black areas in the figure are both valid search areas.

[0079] That is, in the traditional method, the encoded area is divided into a valid search area and an invalid search area, while in the embodiment of the present application, the entire encoded area is expanded into the valid search area.

[0080] Step 302: The control device performs screen content encoding based on the target coding unit and the target reference block.

[0081] See Figure 5 FIG. 3 is a flowchart of a method for determining a target reference block. The following is a specific embodiment to illustrate the above step 301. When executing the above step 301, the following process can be used:

[0082] Step 500: The control device obtains image feature information of the target coding unit.

[0083] Specifically, the image feature information is determined according to the pixel value of each pixel in the target coding unit.

[0084] In one embodiment, the image feature information may be determined by the following steps:

[0085] S5001: The control device obtains the pixel value of each pixel in the target coding unit.

[0086] Among them, pixels are the smallest image units, and a picture is composed of multiple pixels.

[0087] For example, the image size of a picture is 500*338, which means that the picture is composed of a 500*338 pixel matrix. The width of the picture is 500 pixels long, and the height is 338 pixels long. There are a total of 500*338=149,000 pixels.

[0088] S5002: The control device determines the sum of each pixel value of each coding unit.

[0089] S5003: The control device determines the average horizontal pixel gradient of the target coding unit in the horizontal direction and the average vertical pixel gradient in the vertical direction according to each pixel value.

[0090] Specifically, the control device determines the average value of the horizontal pixel gradient according to each pixel value in the horizontal direction, and determines the average value of the vertical pixel gradient according to each pixel value in the vertical direction.

[0091] S5004: The control device obtains image feature information of the target coding unit based on the sum of the pixel values, the horizontal pixel gradient average, and the vertical pixel gradient average.

[0092] Specifically, the control device uses the sum of the pixel values, the horizontal pixel gradient average value, and the vertical pixel gradient average value as the image feature information of the target coding unit.

[0093] In practical applications, other methods may also be used to determine image feature information, which is not limited here.

[0094] In this way, the image feature information of the target coding unit can be obtained.

[0095] Step 501: The control device performs hash processing on the image feature information of the target coding unit to obtain the image hash value of the target coding unit.

[0096] Specifically, the control device uses a preset hash algorithm to perform hash processing on the image feature information of the target coding unit to obtain the image hash value of the target coding unit.

[0097] A hash algorithm is used to generate a fixed-length output based on an input. Changing a single character in the input will produce a completely different hash value.

[0098] Optionally, the hash algorithm can use a perceptual hash algorithm, which is a general term for a class of hash algorithms. Its function is to generate a "fingerprint" string for each image and compare the fingerprint information of different images to determine the similarity of the images. The closer the result, the more similar the images are. Perceptual hash algorithms include mean hash (aHash), perceptual hash (pHash), and difference hash (dHash). aHash is faster but less accurate; pHash takes the opposite approach, with higher accuracy but slower speed; dHash takes both into account, with higher accuracy and faster speed. After obtaining the 64-bit hash value, the Hamming distance is used to quantify the similarity between the two images. The larger the Hamming distance, the less similar the images are, and the smaller the Hamming distance, the greater the similarity between the images.

[0099] In this way, the image hash value of the target coding unit can be obtained.

[0100] Step 502: The control device obtains a one-way linked list corresponding to the image hash value of the target coding unit.

[0101] In one implementation, the control device determines the one-way linked list corresponding to the image hash value of the target coding unit through a hash index table.

[0102] The hash index table contains each image hash value and a corresponding one-way linked list. The one-way linked list is created based on the reference blocks within the coded region. Each reference block in a one-way linked list has the same image hash value. The reference blocks in the one-way linked list are arranged in descending order of distance from the target coding unit.

[0103] In this way, multiple reference blocks similar to the target coding unit can be efficiently screened out through the image hash value.

[0104] Step 503: The control device sequentially determines the image similarity between each reference block in the screened one-way linked list and the target coding unit, until it is determined that there is a reference block whose image similarity meets the preset matching condition.

[0105] Specifically, the control device obtains pixel information of the target coding unit and performs the following steps for each reference block in the one-way linked list until it is determined that there is a reference block whose image similarity meets the preset matching condition:

[0106] Obtain pixel information of the reference block, and determine image similarity between the target coding unit and the reference block based on the pixel information of the target coding unit and the pixel information of the reference block, and if the image similarity is not lower than a first similarity threshold, determine that the reference block is a reference block that meets the preset matching condition.

[0107] Optionally, the image similarity may be determined by using the sum of absolute differences (SAD) or other methods according to actual application scenarios, which are not limited here.

[0108] Pixel information refers to the pixel value of each pixel. In digital image processing, SAD is a measure of similarity between image blocks. It is calculated by calculating the absolute difference between each pixel in one image block and the corresponding pixel in another image block. The smaller the SAD, the higher the image similarity.

[0109] In actual applications, the first similarity threshold can be set according to actual application scenarios. For example, the first similarity threshold can be 100%, which is not limited here.

[0110] Furthermore, when it is determined that there is no reference block with a similarity not lower than the first similarity threshold, the reference block that meets the preset matching conditions can also be determined based on the position distance and / or corresponding image similarity between the obtained target coding unit and each reference block.

[0111] In one embodiment, a reference block corresponding to the maximum value of each image similarity is determined. When the image similarity corresponding to the determined reference block is greater than a second preset threshold, it is determined whether the determined reference block is one. If so, it is determined that the reference block meets the preset matching condition. Otherwise, the reference block with the smallest position distance to the target coding unit among the multiple reference blocks determined is determined as the reference block that meets the preset matching condition.

[0112] In one embodiment, reference blocks whose image similarity is greater than a third preset threshold are screened out. If only one reference block is determined, it is determined that the reference block meets the preset matching conditions. Otherwise, the reference block with the smallest position distance to the target coding unit among the multiple reference blocks determined is determined as the reference block that meets the preset matching conditions.

[0113] In actual applications, the first preset threshold, the second threshold, and the third preset threshold can be set according to actual application scenarios, for example, they can be 1, 0.9, and 0.8, respectively, and there is no limitation here.

[0114] The embodiment of the present application can be applied to single-threaded screen content coding scenarios, and can also be applied to multi-threaded screen content coding scenarios. Furthermore, the IBC mode can be used not only in I frames, but also in inter-frame coded frames P or B frames.

[0115] In a specific application scenario, in single-threaded mode, the QQ265 SCC YUV420 encoder adopts the solution provided by the embodiment of the present application and achieves a bit rate saving of 1.65% ( The QQ265 SCC YUV444 encoder, using the solution provided in this embodiment, achieved a 0.05% improvement in BD-BR quality and a 1% reduction in encoding complexity. BD-BR represents the bitrate savings achieved by the two methods at the same objective quality. BD-BR is a metric for measuring video quality reduction (RD).

[0116] The following uses a specific application scenario to specifically illustrate the expansion of the effective search area for multi-threading.

[0117] See Figure 6As shown, it is a schematic diagram of a multi-threaded screen content encoding provided by an embodiment of the present application. For a multi-threaded mode in which Wavefront Parallel Processing (WPP) is greater than 1, taking WPP=2 as an example, that is, taking two threads as an example, the first thread encodes the CU in the first row in the direction indicated by the arrow, and all reference blocks in the first row are reference blocks within the encoded area. The second thread encodes the CU in the second row in parallel in the direction of the arrow, and the N+1th CU is the target coding unit, then the encoded area of ​​the target coding unit includes: the area of ​​the entire first row and the area before the N+1th CU in the second row. The first thread encodes the CU in the third row in parallel in the direction of the arrow, and the Nth CU is the target coding unit, then the encoded area of ​​the target coding unit includes: the area of ​​the entire first row, the area before the N+1th CU in the second row, and the area before the Nth CU in the third row. The second thread encodes the CU in the fourth row, and the N-1th CU is the target coding unit. The encoded area of ​​the target coding unit includes: the area of ​​the entire first row, the area before the N+1th CU in the second row, the area before the Nth CU in the third row, and the area before the N-1th CU in the fourth row.

[0118] It can be seen that compared with the traditional effective search area, even if only the effective search area of ​​the first row is expanded, the encoding efficiency and encoding quality of the screen content will be improved.

[0119] In the embodiment of the present application, the IBC technology is optimized, the effective search area is expanded, the number of reference blocks is increased, and the IBC RD quality is improved. There is no need to determine whether the reference blocks in the one-way linked list are within the effective search area, which reduces the system resources consumed by position calculation. Furthermore, if a matching target reference block is obtained in the expanded search area, the actual search path length can be shortened, thereby reducing the computational complexity.

[0120] Based on the same inventive concept, an embodiment of the present application also provides a device for screen content encoding. Since the principles of solving the problems of the above-mentioned devices and equipment are similar to those of a screen content encoding method, the implementation of the above-mentioned device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0121] like Figure 7 , which is a schematic diagram of the structure of a device for encoding screen content provided by an embodiment of the present application. A device for encoding screen content includes:

[0122] A matching unit 701 is configured to select, from reference blocks within an encoded region of a screen content image, a target reference block that matches a target coding unit to be encoded in the screen content image;

[0123] The processing unit 702 is configured to perform screen content encoding based on the target coding unit and the target reference block.

[0124] Preferably, the matching unit 701 is used to:

[0125] Obtaining image feature information of a target coding unit;

[0126] Performing hash processing on the image feature information of the target coding unit to obtain an image hash value of the target coding unit;

[0127] Obtain a one-way linked list corresponding to the image hash value of the target coding unit. The one-way linked list is created based on the reference blocks in the coded area. The image hash value corresponding to each reference block included in the one-way linked list is the same.

[0128] Determining the image similarity between each reference block in the one-way linked list and the target coding unit in sequence until a reference block whose image similarity meets a preset matching condition is determined;

[0129] The determined reference block is determined as the target reference block matched by the target coding unit.

[0130] Preferably, the matching unit 701 is used to:

[0131] Obtain the pixel value of each pixel in the target coding unit;

[0132] Determining a sum of pixel values ​​of a target coding unit;

[0133] Determine, based on each pixel value, an average value of horizontal pixel gradients in the horizontal direction and an average value of vertical pixel gradients in the vertical direction of the target coding unit;

[0134] Image feature information of the target coding unit is obtained based on the sum of the pixel values, the horizontal pixel gradient average, and the vertical pixel gradient average.

[0135] Preferably, the matching unit 701 is used to:

[0136] Obtain pixel information of the target coding unit;

[0137] For each reference block in the one-way linked list, perform the following steps until a reference block whose image similarity meets the preset matching conditions is determined:

[0138] Obtain pixel information of the reference block, and determine image similarity between the target coding unit and the reference block based on the pixel information of the target coding unit and the pixel information of the reference block, and if the image similarity is not lower than a first similarity threshold, determine that the reference block is a reference block that meets the preset matching condition.

[0139] Preferably, the matching unit 701 is further configured to:

[0140] When it is determined that there is no reference block whose similarity is not lower than the first similarity threshold, a reference block that meets the preset matching condition is determined based on the position distance and corresponding image similarity between the obtained target coding unit and each reference block.

[0141] In a screen content encoding method, apparatus, device, and medium provided by embodiments of the present application, a target reference block matching a target coding unit to be encoded in the screen content image is screened out from all reference blocks within the encoded area of ​​the screen content image, and screen content encoding is performed based on the target coding unit and the corresponding target reference block. This eliminates the need to determine whether the reference block is within the valid search area of ​​the target coding unit, reducing computational overhead, expanding the valid search area of ​​the target coding unit, increasing the number of reference blocks, and improving the quality of screen content encoding. Furthermore, if the target reference block is within the invalid search area of ​​the target coding unit, the search path length for the target reference block can be reduced, reducing computational complexity.

[0142] Figure 8 Schematic diagram of the structure of a control device 8000 is shown. Figure 8 As shown, the control device 8000 includes: a processor 8010, a memory 8020, a power supply 8030, a display unit 8040, and an input unit 8050.

[0143] The processor 8010 is the control center of the control device 8000. It uses various interfaces and lines to connect various components, and performs various functions of the control device 8000 by running or executing software programs and / or data stored in the memory 8020, thereby monitoring the control device 8000 as a whole.

[0144] In the embodiment of the present application, the processor 8010 executes the following when calling the computer program stored in the memory 8020: Figure 3 The embodiment shown in provides a method for encoding screen content.

[0145] Optionally, the processor 8010 may include one or more processing units. Preferably, the processor 8010 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and applications, and the modem processor primarily processes wireless communications. It is understood that the modem processor may not be integrated into the processor 8010. In some embodiments, the processor and memory may be implemented on a single chip. In some embodiments, they may also be implemented on separate chips.

[0146] The memory 8020 may primarily include a program storage area and a data storage area. The program storage area may store an operating system, various applications, and the like; the data storage area may store data generated based on the use of the control device 8000. Furthermore, the memory 8020 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0147] The control device 8000 also includes a power supply 8030 (such as a battery) for supplying power to various components. The power supply can be logically connected to the processor 8010 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0148] The display unit 8040 can be used to display information input by the user or information provided to the user, as well as various menus of the control device 8000. In the embodiment of the present invention, it is mainly used to display the display interface of each application in the control device 8000 and objects such as text and images displayed on the display interface. The display unit 8040 may include a display panel 8041. The display panel 8041 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.

[0149] The input unit 8050 can be used to receive information such as numbers or characters input by the user. The input unit 8050 may include a touch panel 8051 and other input devices 8052. The touch panel 8051, also known as a touch screen, can receive user touch operations on or near it (for example, operations performed by the user using a finger, a stylus, or any other suitable object or accessory on or near the touch panel 8051).

[0150] Specifically, the touch panel 8051 can detect user touch operations and the signals generated by the touch operations, convert these signals into touch point coordinates, and send them to the processor 8010. It can also receive and execute commands sent by the processor 8010. In addition, the touch panel 8051 can be implemented using various types, such as resistive, capacitive, infrared, and surface acoustic wave. Other input devices 8052 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, a joystick, etc.

[0151] Of course, the touch panel 8051 can cover the display panel 8041. When the touch panel 8051 detects a touch operation on or near it, it transmits it to the processor 8010 to determine the type of touch event. Then the processor 8010 provides corresponding visual output on the display panel 8041 according to the type of touch event. Figure 8 In the embodiment, the touch panel 8051 and the display panel 8041 are two independent components to realize the input and output functions of the control device 8000, but in some embodiments, the touch panel 8051 and the display panel 8041 can be integrated to realize the input and output functions of the control device 8000.

[0152] The control device 8000 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity light sensor, etc. Of course, according to the needs of specific applications, the control device 8000 may also include other components such as a camera. Since these components are not the key components used in the embodiments of this application, Figure 8 It is not shown and will not be described in detail.

[0153] Those skilled in the art will understand that Figure 8 This is merely an example of a control device and does not constitute a limitation on the control device. The control device may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0154] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the screen content encoding method in any of the above method embodiments is implemented.

[0155] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the relevant technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a control device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or certain parts of the embodiment.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A screen content encoding method, characterized in that: include: Filtering, from reference blocks within an encoded region of a screen content image, a target reference block that matches a target coding unit to be encoded in the screen content image; wherein the encoded region is divided into at least two rows, and the number of rows divided is an integer multiple of the number of at least two enabled threads; and each row in the encoded region corresponds to one enabled thread; Performing screen content encoding based on the target coding unit and the target reference block; The step of screening out a target reference block matching a target coding unit to be encoded in the screen content image includes: Determining an image hash value corresponding to the target coding unit and obtaining a one-way linked list corresponding to the image hash value; the one-way linked list is created based on reference blocks within the coded area, and each reference block included in the one-way linked list has the same image hash value; For the at least two rows divided by the encoded area in the screen content image, perform the following operations respectively: Through the corresponding enabled threads, according to the preset direction, the hash value information corresponding to each reference block in the row in the one-way linked list is filtered and processed until the corresponding target reference block is obtained; the one-way linked list contains: the hash value information and position information corresponding to each reference block in the encoded area.

2. The method according to claim 1, wherein Screening out a target reference block matching a target coding unit to be encoded in the screen content image from reference blocks in an encoded area of ​​the screen content image includes: Acquiring image feature information of the target coding unit; Performing hash processing on the image feature information of the target coding unit to obtain an image hash value of the target coding unit; Obtain a one-way linked list corresponding to the image hash value of the target coding unit; sequentially determining the image similarity between each reference block in the one-way linked list and the target coding unit until a reference block whose image similarity meets a preset matching condition is determined; The determined reference block is determined as the target reference block matching the target coding unit.

3. The method according to claim 2, wherein Acquiring image feature information of the target coding unit, including: Obtaining a pixel value of each pixel in the target coding unit; Determining a sum of pixel values ​​of the target coding unit; Determine, based on each pixel value, an average value of horizontal pixel gradients in a horizontal direction and an average value of vertical pixel gradients in a vertical direction of the target coding unit; Image feature information of the target coding unit is obtained based on the sum of the pixel values, the horizontal pixel gradient average value, and the vertical pixel gradient average value.

4. The method according to claim 2, wherein Determining the image similarity between each reference block in the one-way linked list and the target coding unit in sequence until it is determined that there is a reference block whose image similarity meets a preset matching condition, including: Obtaining pixel information of the target coding unit; For each reference block in the one-way linked list, the following steps are performed until it is determined that there is a reference block whose image similarity meets the preset matching condition: Obtain pixel information of the reference block, and determine image similarity between the target coding unit and the reference block based on the pixel information of the target coding unit and the pixel information of the reference block, and if the image similarity is not lower than a first similarity threshold, determine that the reference block is a reference block that meets a preset matching condition.

5. The method according to claim 3, wherein Further including: When it is determined that there is no reference block whose similarity is not lower than the first similarity threshold, a reference block that meets the preset matching condition is determined based on the acquired position distance and corresponding image similarity between the target coding unit and each reference block.

6. A device for encoding screen content, characterized in that: include: a matching unit, configured to screen, from reference blocks within an encoded region of a screen content image, a target reference block that matches a target coding unit to be encoded in the screen content image; wherein the encoded region is divided into at least two rows, the number of rows being an integer multiple of the number of at least two enabled threads; and each row in the encoded region corresponds to one enabled thread; a processing unit, configured to perform screen content encoding based on the target encoding unit and the target reference block; When the matching unit is used to screen out the target reference block that matches the target coding unit to be encoded in the screen content image, it is specifically used to: determine the image hash value corresponding to the target coding unit, and obtain a one-way linked list corresponding to the image hash value; the one-way linked list is created based on the reference blocks in the encoded area, and the image hash value corresponding to each reference block contained in a one-way linked list is the same; for the at least two rows divided by the encoded area in the screen content image, the following operations are performed respectively: through the corresponding enabled thread, according to the preset direction, the hash value information corresponding to each reference block in the row recorded in the one-way linked list is screened and processed until the corresponding target reference block is obtained; the one-way linked list contains: the hash value information and position information corresponding to each reference block in the encoded area.

7. The device according to claim 6, characterized in that The matching unit is used for: Acquiring image feature information of the target coding unit; Performing hash processing on the image feature information of the target coding unit to obtain an image hash value of the target coding unit; Obtain a one-way linked list corresponding to the image hash value of the target coding unit; sequentially determining the image similarity between each reference block in the one-way linked list and the target coding unit until a reference block whose image similarity meets a preset matching condition is determined; The determined reference block is determined as the target reference block matching the target coding unit.

8. The device according to claim 7, wherein The matching unit is used for: Obtaining a pixel value of each pixel in the target coding unit; Determining a sum of pixel values ​​of the target coding unit; Determine, based on each pixel value, an average value of horizontal pixel gradients in a horizontal direction and an average value of vertical pixel gradients in a vertical direction of the target coding unit; Image feature information of the target coding unit is obtained based on the sum of the pixel values, the horizontal pixel gradient average value, and the vertical pixel gradient average value.

9. A control device, characterized in that: include: at least one memory for storing program instructions; At least one processor is configured to call the program instructions stored in the memory and execute the steps of the method according to any one of claims 1 to 5 according to the obtained program instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images