Region perception coding and decoding method for screen content image

Abstract

The invention provides a region perception coding and decoding method for a screen content image. The method comprises the following steps: respectively detecting text information and graphic information in an input screen content image at a coding end to obtain a text region and a graphic region; at a coding end, extracting structured information of the text region, and carrying out lossless coding on the structured information to obtain a coded text region; at the coding end, chromaticity conversion is carried out on the graphic area, lossless coding is carried out on the converted graphic information, and the coded graphic area is obtained; and encoding the residual background region in the screen content image at the encoding end to obtain an encoded background region, and synthesizing the screen content image at the decoding end according to the encoded text region, graphic region and background region. According to the method, the text and graph areas are identified through the area detection and cutting mechanism, and differential coding strategies are adopted for different areas, so that the coding efficiency and the visual reconstruction quality of the screen content image are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of image encoding technology, and in particular to a region-aware encoding and decoding method for screen content images. Background Technology

[0002] Screen content images differ significantly from natural images in terms of content composition and structural features. Screen content typically contains a large number of computer-generated areas such as text, lines, geometric shapes, and interface components. These areas have distinct edge structures, flat regions, and high-contrast boundaries. Therefore, traditional natural image encoding algorithms often suffer from problems such as blurred characters, broken edges, and quality degradation when processing screen content, making it difficult to meet the needs of scenarios such as office document transmission, distance education, meeting collaboration, and interface recording.

[0003] Currently, existing deep learning-based image compression methods primarily target natural images, and their feature extraction and transformation structures do not consider the regional structural characteristics of screen content images. Furthermore, existing image compression models often rely on a single global processing approach, lacking the ability to differentiate and model highly sensitive regions such as text and icon regions, resulting in insufficient reconstruction quality after compression, especially noticeable in high compression ratio scenarios. Summary of the Invention

[0004] This invention provides a region-aware encoding and decoding method for screen content images, so as to effectively improve the encoding and decoding efficiency of screen content images.

[0005] To achieve the above objectives, the present invention adopts the following technical solution.

[0006] A region-aware encoding and decoding method for screen content images includes: At the encoding end, an adaptive detection and cropping mechanism is used to detect text and graphic information in the input screen content image, respectively, to obtain text regions and graphic regions; At the encoding end, the structured information of the text region is extracted, and the structured information is losslessly encoded to obtain the encoded text region; At the encoding end, the graphic region is chromaticity converted to construct a unified luminance-chromaticity mapping table, and the converted graphic information is losslessly encoded to obtain the encoded graphic region. At the encoding end, the remaining background area in the screen content image is encoded using a screen content encoder to obtain the encoded background area. At the decoding end, the screen content image is synthesized based on the encoded text area, graphic area, and background area.

[0007] Preferably, at the encoding end, an adaptive detection and cropping mechanism is used to detect text and graphic information in the input screen content image, respectively, to obtain text regions and graphic regions, including: At the encoding end, a depthwise convolutional-based detection network is trained using a certain amount of screen content image data containing text and graphics. The deep convolutional-based detection network learns the structural features of the screen content image data, resulting in a trained deep convolutional-based detection network. The input screen content image is then fed into the trained deep convolutional-based detection network. ( ); Detection Networks Based on Depth Convolution ( ) Input screen content image Text prior extraction is performed to obtain the text prior map. The calculation process is expressed as follows: Formula (1) Among them, U( ) indicates an upsampling operation. For network parameters, the text prior graph Used to characterize the confidence that each location in an image belongs to a text region; The text prior map With screen content images By fusing the images, a text-aware enhanced image is obtained. : Formula (2) Text-aware enhanced images Output multiple candidate regions, their category identifiers, and location information: Formula (3) in, Identify the region categories, which include text regions and graphic regions. For the corresponding area coordinates, based on the area category identifier and regional coordinates The original image is cropped to obtain the corresponding text and graphic regions.

[0008] Preferably, the step of extracting the structured information of the text region at the encoding end and performing lossless encoding on the structured information to obtain the encoded text region includes: At the encoding end, the character sequence in the text region is extracted by a text recognition network, and the position, font category, font size and text color of the character sequence are identified to obtain structured text information. The structured text information is encoded into a structured string, and then the structured string is losslessly encoded by dictionary matching and entropy encoding to obtain the encoded text region.

[0009] Preferably, at the encoding end, the graphic region undergoes chromaticity conversion to construct a unified luminance-chromaticity mapping table, and the converted graphic information is losslessly encoded to obtain the encoded graphic region, including: At the encoding end, the region bounding boxes corresponding to the cropped graphic regions are chromaticity converted, transforming the graphic regions from the red-green-blue color space to the luminance-chrominance space. Then, the color of the graphic regions is standardized using chromaticity consistency. The correspondence between luminance and chrominance is statistically analyzed on all graphic regions to construct a shared luminance-chrominance mapping table. During encoding, only the luminance information is losslessly encoded to obtain the encoded graphic regions, while the chrominance information is stored through the luminance-chrominance mapping table.

[0010] Preferably, the step of encoding the remaining background area in the screen content image using a screen content encoder at the encoding end to obtain the encoded background area, and then synthesizing the screen content image at the decoding end based on the encoded text area, graphic area, and background area, includes: The area in the screen content image other than the text area and the graphic area is taken as the background area. At the encoding end, the background area is compressed and encoded using a screen content dedicated encoder to obtain the encoded background area. At the decoding end, the text region is reconstituted based on the encoded text region information, the graphic region is reconstituted based on the encoded graphic region and the shared color mapping table, the background region is reconstituted based on the encoded background region using a dedicated screen content decoder, and the reconstituted text region, graphic region and background region are combined into a complete screen content image based on the regional position coordinates of the screen content image.

[0011] As can be seen from the technical solutions provided by the embodiments of the present invention above, the method of the present invention identifies text and graphic regions through region detection and cropping mechanisms, and adopts differentiated encoding strategies for different regions, thereby significantly improving the encoding efficiency and visual reconstruction quality of screen content images.

[0012] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram illustrating the implementation principle of a region-aware encoding and decoding method for screen content images provided in an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram illustrating the processing flow of a region-aware encoding and decoding method for screen content images provided in an embodiment of the present invention.

[0016] Figure 3 This is a schematic diagram illustrating the implementation principle of a text region detection and information extraction module provided in an embodiment of the present invention.

[0017] Figure 4 This is a schematic diagram illustrating the implementation principle of constructing a colorimetric unification and sharing mapping table for a graphic region, as provided in an embodiment of the present invention. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0019] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0020] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0021] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0022] Example 1: The implementation principle diagram of a region-aware encoding and decoding method for screen content images provided in this embodiment of the invention is shown below. Figure 1 As shown, the specific processing flow is as follows: Figure 2 As shown, it includes the following steps: Step S10: At the encoding end, the text information and graphic information in the input screen content image are detected by an adaptive detection and cropping mechanism to obtain the text region and graphic region respectively.

[0023] Step S20: At the encoding end, extract the structured information such as text content, font, size and color of the text region, and perform lossless encoding on the structured information to obtain the encoded text region.

[0024] Step S30: At the encoding end, perform chromaticity conversion on the graphic region, construct a unified luminance-chromaticity mapping table, and perform lossless encoding on the converted graphic information to obtain the encoded graphic region.

[0025] Step S40: At the encoding end, the remaining background area is encoded using a screen content encoder to obtain the encoded background area. At the decoding end, the complete image is finally synthesized based on the encoded text area, graphic area, and background area.

[0026] Specifically, step S10 above includes: Figure 3 This diagram illustrates the implementation principle of a text region detection and information extraction module provided in this embodiment of the invention. This step aims to identify text and graphic regions in screen content images through an adaptive detection and cropping mechanism for subsequent differentiated processing. First, a depthwise convolutional-based detection network is trained using a certain amount of screen content image data containing text and graphics. The depthwise convolutional-based detection network learns the structural features of the screen content image data, resulting in a trained depthwise convolutional-based detection network. ( ).

[0027] The adaptive detection and cropping mechanism employs a depthwise convolution-based object detection network and adapts it to the text characteristics of screen content images, enhancing the detectability of text regions by introducing prior text information.

[0028] First, a detection network based on depthwise convolution. ( ) Input screen content image Text prior extraction is performed to obtain the text prior map. The calculation process is expressed as follows: Formula (1) Among them, U( ) indicates an upsampling operation. These are network parameters. The text prior map is used to characterize the confidence that each location in the image belongs to a text region.

[0029] Subsequently, the text prior image is fused with the original image to obtain a text-aware enhanced image. : Formula (2) Building upon this, a detection network based on depthwise convolution... ( Text-aware augmented images Output multiple candidate regions, their category identifiers, and location information: Formula (3) in, Identify the region categories, which include text regions and graphic regions. For the corresponding area coordinates, finally, based on the area category identifier... and regional coordinates The original image is cropped to obtain the corresponding text area, graphic area, and background area.

[0030] The input screen content image is fed into the trained depthwise convolutional-based detection network. This network identifies text regions (such as characters, paragraphs, and titles) and graphic regions (such as icons, buttons, and graphs) in the screen content image data and outputs the corresponding bounding boxes for the text and graphic regions. The bounding boxes for the text and graphic regions are cropped into independent sub-images for independent encoding.

[0031] After cropping the bounding boxes corresponding to the text and graphic regions, the remaining portion of the screen content image is considered the background region. Adaptive detection and cropping are key preliminary steps in this invention, significantly reducing redundant areas in subsequent pixel-level processing and enabling accurate extraction of text and graphic structural information. This step also requires storing the region location coordinates of the input screen content image.

[0032] Specifically, step S20 includes: extracting information and lossless encoding the region bounding boxes corresponding to the text region. For each text region, this invention extracts character sequences through a text recognition network and identifies the position, font type, font size, and text color of the character sequences to obtain structured text information. This structured text information together constitutes complete text description information. To achieve high-fidelity reconstruction, this invention encodes the structured text information into a structured string, and then performs lossless encoding on the structured string using dictionary matching and entropy encoding to obtain the encoded text region.

[0033] Because text itself is highly regular, structured text encoding offers significantly higher compression efficiency compared to pixel compression, while avoiding issues like character blurring caused by traditional compression. On the decoding side, the encoded text regions can be precisely synthesized into images, achieving completely lossless text reconstruction.

[0034] Specifically, step S30 above includes: Figure 4 This diagram illustrates the implementation principle of a unified chromaticity mapping table for graphic regions, as provided in this invention. At the encoding end, the region bounding boxes corresponding to the cropped graphic regions undergo chromaticity conversion. This chromaticity conversion includes chromaticity unification and mapping table encoding. The edges and colors of graphic regions (such as icons, buttons, and interface elements) are crucial in screen content images. This invention first converts the graphic regions from a red-green-blue color space to a luminance-chromaticity space, and then standardizes the colors of the graphic regions using chromaticity consistency. Subsequently, the correspondence between luminance and chromaticity is statistically analyzed across all graphic regions to construct a shared luminance-chromaticity mapping table. During encoding, only luminance information is compressed, while chromaticity information is uniformly stored through the shared chromaticity mapping table, resulting in the encoded graphic regions. Since the chromaticity of graphic regions is usually relatively stable, the shared chromaticity mapping table method can significantly reduce redundant storage while maintaining color fidelity. The aforementioned shared chromaticity mapping table is used at the decoding end to recover the chromaticity of the graphic regions, ensuring the consistency of the interface graphic appearance and color accuracy.

[0035] Specifically, step S40 includes: encoding the background region and reconstructing the image. The remaining background region after region detection typically contains natural textures or complex mixed content. At the encoding end, this invention uses a screen content-specific encoder to compress and encode the background region, obtaining the encoded background region to ensure the restoration quality and compatibility of the background region.

[0036] At the decoding end, the text region is reconstructed based on the encoded text region information. The graphic region is reconstructed based on the encoded graphic region and the shared colorimetric map. The background region is then reconstructed using a dedicated screen content decoder based on the encoded background region.

[0037] Finally, based on the region position coordinates of the input screen content image stored in step S10, the resynthesized text region, graphic region, and background region are combined into a complete screen content image, achieving lossless or high-quality restoration of structured regions and significantly improving the overall compression rate.

[0038] Comparative test

[0039] (1) Training and testing process

[0040] The experiments were conducted on an NVIDIA RTX 3090 GPU platform, with the region detection network trained and tested using a deep learning framework. To ensure the usability of the overall encoding process, both the text region recognition module and the image region processing module were jointly trained using publicly available datasets and artificially synthesized data. The proposed region-aware encoding algorithm was ultimately integrated into a screen content image compression framework, and its performance was compared with several existing encoding algorithms.

[0041] For text region detection, supervised training was performed using the screen content document detection dataset constructed in this invention. For image region detection, multi-source image synthesis data was used for enhanced training, thereby improving the model's robustness in complex interface scenarios.

[0042] To comprehensively evaluate the coding performance of the method of this invention, the experiment adopted two mainstream evaluation metrics for screen content: Accelerated Screen Image Quality Evaluator (ASIQE) and Deep Feature Statistics (DFSS).

[0043] Tables 1 and 2 present the comparison results of the coding efficiency of the proposed method and several existing coding algorithms on three typical screen content datasets, using ASIQE and DFSS as indicators. As can be seen from Tables 1 and 2, the region-aware coding algorithm proposed in this invention significantly improves the readability of text regions, the color consistency of graphic regions, and the overall structural feature restoration quality at the same bitrate, especially in text-intensive or icon-intensive scenarios. This indicates that the proposed invention can fully utilize the regional characteristics of screen content images to achieve a more efficient coding method.

[0044] Table 1. Comparison of Screen Content Encoding Efficiency (ASIQE)

[0045] Table 2 Comparison of Screen Content Encoding Efficiency (DFSS)

[0046] (2) Coding performance verification under different content scenarios

[0047] To further verify the encoding robustness of the method of this invention under different content types, experiments were conducted in the test set to evaluate the performance of each method in scenarios of overlapping text and graphics, anti-aliased elements, multi-resolution scenarios, and semi-transparent graphics. The ASIQE metric was used to uniformly test different encoding algorithms, and the specific results are shown in Table 3.

[0048] As can be seen from Table 3, the method of the present invention achieves excellent performance in all four types of scenarios, effectively avoiding common problems such as character blurring, jagged edges, and color shift. In anti-aliased elements and semi-transparent scenarios, the present invention maintains the color consistency of interface elements by sharing a colorimetric mapping table. In multi-resolution scenarios, the present invention improves the overall structure restoration quality by using regional encoding.

[0049] Table 3 Comparison of coding efficiency in complex scenarios (ASIQE)

[0050] In summary, the method of this invention significantly enhances text readability: storing text descriptions instead of pixels avoids blurring or breakage. High chromaticity consistency across graphic regions is ensured: a shared chromaticity mapping table guarantees consistent graphic colors, improving reconstruction quality. Overall bitrate is significantly reduced: structured representation of text and graphic regions drastically reduces redundant pixel storage.

[0051] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0052] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0053] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0054] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A region-aware encoding and decoding method for screen content images, characterized in that, include: At the encoding end, an adaptive detection and cropping mechanism is used to detect text and graphic information in the input screen content image, respectively, to obtain text regions and graphic regions; At the encoding end, the structured information of the text region is extracted, and the structured information is losslessly encoded to obtain the encoded text region; At the encoding end, the graphic region is chromaticity converted to construct a unified luminance-chromaticity mapping table, and the converted graphic information is losslessly encoded to obtain the encoded graphic region. At the encoding end, the remaining background area in the screen content image is encoded using a screen content encoder to obtain the encoded background area. At the decoding end, the screen content image is synthesized based on the encoded text area, graphic area, and background area.

2. The method according to claim 1, characterized in that, At the encoding end, an adaptive detection and cropping mechanism is used to detect text and graphic information in the input screen content image, respectively, to obtain text regions and graphic regions, including: At the encoding end, a depthwise convolutional-based detection network is trained using a certain amount of screen content image data containing text and graphics. The deep convolutional-based detection network learns the structural features of the screen content image data, resulting in a trained deep convolutional-based detection network. The input screen content image is then fed into the trained deep convolutional-based detection network. ( ); Detection Networks Based on Depth Convolution ( ) Input screen content image Text prior extraction is performed to obtain the text prior map. The calculation process is expressed as follows: Official (1) Among them, U( ) indicates an upsampling operation. For network parameters, the text prior graph Used to characterize the confidence that each location in an image belongs to a text region; The text prior map With screen content images By fusing the images, a text-aware enhanced image is obtained. : Official (2) Text-aware enhanced images Output multiple candidate regions, their category identifiers, and location information: Official (3) in, Identify the region categories, which include text regions and graphic regions. For the corresponding area coordinates, based on the area category identifier and regional coordinates The original image is cropped to obtain the corresponding text and graphic regions.

3. The method according to claim 2, characterized in that, The process of extracting structured information from the text region at the encoding end, and performing lossless encoding on the structured information to obtain the encoded text region includes: At the encoding end, the character sequence in the text region is extracted by a text recognition network, and the position, font category, font size and text color of the character sequence are identified to obtain structured text information. The structured text information is encoded into a structured string, and then the structured string is losslessly encoded by dictionary matching and entropy encoding to obtain the encoded text region.

4. The method according to claim 2, characterized in that, At the encoding end, the graphic region undergoes chromaticity conversion to construct a unified luminance-chromaticity mapping table. The converted graphic information is then losslessly encoded to obtain the encoded graphic region, including: At the encoding end, the region bounding boxes corresponding to the cropped graphic regions are chromaticity converted, transforming the graphic regions from the red-green-blue color space to the luminance-chrominance space. Then, the color of the graphic regions is standardized using chromaticity consistency. The correspondence between luminance and chrominance is statistically analyzed on all graphic regions to construct a shared luminance-chrominance mapping table. During encoding, only the luminance information is losslessly encoded to obtain the encoded graphic regions, while the chrominance information is stored through the luminance-chrominance mapping table.

5. The method according to claim 4, characterized in that, The process of encoding the remaining background area in the screen content image using a screen content encoder at the encoding end to obtain the encoded background area, and then synthesizing the screen content image at the decoding end based on the encoded text area, graphic area, and background area, includes: The area in the screen content image other than the text area and the graphic area is taken as the background area. At the encoding end, the background area is compressed and encoded using a screen content dedicated encoder to obtain the encoded background area. At the decoding end, the text region is reconstituted based on the encoded text region information, the graphic region is reconstituted based on the encoded graphic region and the shared color mapping table, the background region is reconstituted based on the encoded background region using a dedicated screen content decoder, and the reconstituted text region, graphic region and background region are combined into a complete screen content image based on the regional position coordinates of the screen content image.

Images