A screen mixed content classification method and system for efficient compression

Abstract

The application provides a screen mixed content classification method and system for efficient compression, and relates to the technical field of screen mixed content classification.The method comprises the following steps: obtaining a screen mixed content image; performing block processing on the screen mixed content image to obtain a plurality of target blocks; determining a first category classification result of each target block in a feature-based classification manner; constructing an improved convolutional neural network model; determining a second category classification result of each target block through the improved convolutional neural network model; and determining a final category of each target block based on the first category classification result and the second category classification result through a preset classification rule.In the application, the first category and the second category classification result are fused, the traditional feature and the deep feature are cooperatively optimized, and thus the image content recognition accuracy in a complex scene is significantly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of screen mixed content classification, and in particular to a screen mixed content classification method and system oriented to efficient compression. Background Art

[0002] With the rapid advancement of integrated industrial internet applications, scenarios such as remote collaborative office, intelligent manufacturing, cloud gaming, the metaverse, multi-screen conferencing, and telemedicine are generating a large amount of mixed-screen content image data. This type of content, typically composed of a mixture of natural images and computer-generated graphics in arbitrary proportions, exhibits high hybridity, ultra-high definition, and complex structures. This also places new demands on encoding systems, such as high compression rate, low latency, and low power consumption. In this context, developing an efficient mixed-screen content classification method that can effectively identify and adapt to different content characteristics is crucial for improving video image coding efficiency.

[0003] Currently, compression methods for mixed screen image content can be roughly divided into two categories. One is a composite encoding method based on image hierarchical division, which divides the image into background layer, foreground layer, and mask layer for separate encoding, or uses OCR technology to divide the image into text layer and graphic layer, thereby improving the compression efficiency of specific types of content. The other is a content type recognition method based on frame-level judgment, such as the frame-level screen content encoding tool switch mechanism in the AVS3 standard, which determines whether to enable screen content-specific encoding tools by analyzing the number of colors, texture complexity, hash collisions, and other characteristics of the entire frame image.

[0004] However, existing methods often rely on single feature metrics, such as the number of colors or hash hits with fixed thresholds. These methods are susceptible to interference from changes in image structure and edge transitions, making it difficult to accurately identify the local mixed distribution of natural and screen content within an image. Furthermore, existing classification strategies primarily employ manual rules or shallow feature extraction methods, lacking the ability to integrate deep neural networks for high-level semantic modeling and multi-feature fusion. This makes it difficult to guarantee classification accuracy in complex image scenarios. Summary of the Invention

[0005] In view of the above shortcomings of the existing technologies, the purpose of the embodiments of the present invention is to provide a method for classifying screen mixed content for efficient compression. This method can solve the problem that existing methods mostly rely on single feature indicators such as the number of colors with fixed thresholds and the number of hash hits, are easily affected by changes in image structure and edge transition areas, and have difficulty in accurately identifying the local mixed distribution state of natural content and screen content in the image. At the same time, existing classification strategies mainly use manual rules or shallow feature extraction methods, lacking the ability to combine deep neural networks to achieve high-level semantic modeling and multi-feature fusion, resulting in technical problems such as difficulty in ensuring classification accuracy in complex image scenarios.

[0006] A first aspect of an embodiment of the present invention provides a method for classifying screen mixed content for efficient compression, comprising:

[0007] S1: Get the screen mixed content image;

[0008] S2: performing block processing on the screen mixed content image to obtain multiple target blocks;

[0009] S3: Determine a first category classification result for each target block in a feature-based classification manner;

[0010] S4: Build an improved convolutional neural network model;

[0011] S5: Determine the second category classification result of each target block through the improved convolutional neural network model;

[0012] S6: Based on the first category classification result and the second category classification result, the final category of each target block is determined by a preset classification rule.

[0013] A second aspect of an embodiment of the present invention provides a screen mixed content classification system for efficient compression, comprising: a processor and a memory;

[0014] The memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the screen mixed content classification method for efficient compression as described in the first aspect are implemented.

[0015] According to a third aspect of an embodiment of the present invention, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the screen mixed content classification method for efficient compression as described in the first aspect are implemented.

[0016] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0017] In an embodiment of the present invention, by adopting a multi-feature-based classification method to determine the first category classification results of each target block, the adaptability to complex image areas is enhanced, and the local mixed distribution state of natural content and screen content in the image can be accurately identified. An improved convolutional neural network model is introduced to determine the second category classification results of each target block, thereby realizing high-level semantic classification of the target blocks. By fusing the first category and second category classification results, the coordinated optimization of traditional features and deep features is realized, thereby significantly improving the image content recognition accuracy in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are only for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. Throughout the drawings, the same reference symbols represent the same components. Obviously, the drawings described below are only some embodiments of the present invention. It is clear that those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0019] Figure 1 This is a flow chart of a method for classifying screen mixed content for efficient compression provided by an embodiment of the present invention;

[0020] Figure 2 This is a structural diagram of a screen mixed content classification system for efficient compression provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all of the embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the scope of protection of the present invention.

[0022] The screen mixed content classification method for efficient compression provided by the embodiment of the present invention will be described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios.

[0023] Reference Manual Figure 1 , which shows a flow chart of a screen mixed content classification method for efficient compression provided by an embodiment of the present invention.

[0024] An embodiment of the present invention provides a method for classifying screen mixed content for efficient compression, which may include the following steps:

[0025] S1: Acquire a screen mixed content image.

[0026] S2: performing block processing on the screen mixed content image to obtain multiple target blocks.

[0027] Among them, block processing is a basic and key image preprocessing strategy, the core of which is to divide the entire image into several smaller areas according to a predetermined spatial size.

[0028] Optionally, the target block size is specifically: 256x256 or 128x128 or 64x64.

[0029] In the embodiments of the present application, the block strategy can more finely identify the local mixed area of natural content and screen content in the image, help to improve the classification accuracy, and is especially suitable for complex scenes with structured graphics such as text, charts, icons and the like embedded. At the same time, the flexible setting of the block size supports the trade-off between accuracy and processing efficiency, which is suitable for high-precision classification scenes and meets the actual application requirements sensitive to computing performance. At the same time, the uniform size of the block input facilitates batch training and inference calculation of the deep learning model, and improves the overall processing efficiency and model convergence performance of the system.

[0030] S3: determining the first category classification result of each target block in a feature-based classification manner.

[0031] The feature-based classification manner refers to a method of extracting artificially designed features with discriminative ability in an image region, and classifying image content according to a pre-set judgment rule.

[0032] In a possible implementation, S3 specifically includes:

[0033] S301: performing block processing on each target block to obtain a plurality of target sub-blocks.

[0034] S302: calculating the feature value of each target sub-block.

[0035] In a possible implementation, the feature value specifically includes:

[0036] The color number, entropy, horizontal gradient mutation value, vertical gradient mutation value, uniformity and hash value. The calculation method of the uniformity is specifically:

[0037] ;

[0038] Wherein, Evenness The uniformity is represented by evenness, pix max The maximum value in the block is represented by max, pix min The minimum value in the block is represented by min, pix aver The average value in the block is represented by avg.

[0039] The calculation method of the hash value is specifically:

[0040] ;

[0041] Wherein, HashValue represents the hash value, evenness represents the uniformity in the block, Ttcolor represents the color number in the block, gradXsp represents the horizontal gradient mutation value in the block, gradYsp represents the vertical gradient mutation value in the block, and avgDC represents the average color number per pixel in the block.

[0042] It should be noted that the bit length of a hash value is 20 bits, the first 4 bits are uniformity, the next 4 bits are the number of colors, the next 8 bits are the horizontal gradient mutation value and the vertical gradient mutation value respectively, and the last 4 bits are the average number of pixel unit colors.

[0043] Optionally, one calculation method of the horizontal gradient mutation value is that if the absolute value of the difference between the values of horizontally adjacent pixels exceeds a threshold value, it is recorded as a horizontal gradient mutation, and the horizontal gradient mutation value is the total number of horizontal gradient mutations.

[0044] One calculation method of the vertical gradient mutation value is that if the absolute value of the difference between the values of vertically adjacent pixels exceeds a threshold value, it is recorded as a vertical gradient mutation, and the vertical gradient mutation value is the total number of vertical gradient mutations.

[0045] S303: According to the characteristic value of each target sub-block and the preset condition, the classification category of the current target sub-block is determined.

[0046] In one possible implementation, the preset condition specifically includes:

[0047] The hash hit number formed by counting the total number of pixels with the same hash value in each target sub-block is taken as the characteristic value. In the case where the hash hit number is less than a preset hash hit number, the target sub-block is classified as natural content. Otherwise, the target sub-block is classified as computer content.

[0048] And / or the occurrence numbers of a plurality of pixels in each frame of screen mixed content image are arranged in descending order, and the occurrence numbers of the top 10 pixels are counted. In the case where the occurrence number is less than a preset occurrence number, the target sub-block is classified as natural content. Otherwise, the target sub-block is classified as computer content.

[0049] For example: Step 1: Starting from the top-left block, traverse from left to right and from top to bottom in 4x4 sub-block units. For the current traversed sub-block, proceed to Step 2.

[0050] Step 2: Calculate the hash characteristic value of the current traversed sub-block, compare the characteristic value of the sub-block with a threshold value A, and when the characteristic value of the sub-block is greater than a preset characteristic value, the sub-block is considered as natural content, otherwise it is considered as screen content. After traversing all sub-blocks of the current block, proceed to Step 3.

[0051] Step 3: Determine the specific category of the block. Let Hxy be the percentage of the number of sub-blocks considered as natural content in the entire block to the total number of blocks. If Hxy is greater than or equal to 98%, the block is considered as natural content, otherwise it is considered as screen content.

[0052] In the embodiment of the present application, according to the preset conditions such as the hash hit number and the pixel occurrence number of each target sub-block, the classification is performed in combination with the characteristics of the actual image. The rule-based classification manner can ensure the calculation efficiency of the system, and determine different content types according to the predetermined threshold, which helps to improve the stability and reliability of the classification result.

[0053] S304: Determine the first category classification result of each target sub-block according to the classification category of each target sub-block.

[0054] In a possible implementation, S304 specifically includes:

[0055] S3041: Calculate the target ratio of the number of target sub-blocks whose category is natural content to the total number of sub-blocks.

[0056] S3042: Determine whether the target ratio is greater than or equal to a preset ratio. If yes, classify the target sub-block corresponding to each target sub-block as natural content. Otherwise, classify the target sub-block as screen content.

[0057] It should be noted that the size of the preset ratio can be set by the person skilled in the art according to actual needs, and the present application does not limit it.

[0058] S3043: Aggregate the classification results of each target sub-block to obtain the first category classification result.

[0059] In the embodiment of the present application, the proportion of the natural content category in the target sub-block is counted, and compared with the preset threshold to finally determine the classification result of the target sub-block. Through this method, the classification system can make a judgment based on the "majority rule" for the target sub-block, avoiding the influence of the abnormality of individual sub-blocks on the overall classification result. If most of the sub-blocks belong to natural content, the whole image is also determined as natural content, ensuring the stability and rationality of the classification result.

[0060] S4: Construct an improved convolutional neural network model.

[0061] In a possible implementation, the improved convolutional neural network model specifically includes: a first feature extraction layer, a batch normalization layer, an activation layer, and a maximum pooling layer connected in sequence.

[0062] The maximum pooling layer is connected with a second feature extraction layer and a third feature extraction layer respectively.

[0063] The second feature extraction layer and the third feature extraction layer are both connected with the batch normalization layer.

[0064] The maximum pooling layer is connected with an output layer through a classification layer and a full connection layer in sequence.

[0065] Optionally, training the improved convolutional neural network model specifically involves collecting 229,729 natural images (e.g., camera footage) and 7,890 images of screen content (e.g., text, graphics) in MbxNb blocks (e.g., 128x128). Each block is classified using a feature-based classification method to generate a classification label. The classification labels include two categories: Category A for natural content blocks and Category B for screen content blocks. Each data pair consists of a pixel matrix of an MbxNb block (e.g., 128x128 PNG format) and its corresponding classification label, which is used for subsequent model training.

[0066] In an embodiment of the present invention, the classification labels (natural content and screen content) generated by the feature-based classification method provide accurate supervision information for the model, which helps to improve training efficiency.

[0067] An improved convolutional neural network model was constructed. Its input layer receives pixel data in MbxNb blocks (e.g., 128x128 PNG). The feature extraction layer consists of three layers: the first layer uses a 5x5 convolution kernel (with 3 input channels and 32 output channels), with a stride of 1 and padding of 2, to maintain the feature map size. This layer is followed by a batch normalization layer (BatchNorm) and a ReLU activation function, and a 2x2 max pooling layer to reduce the feature map size to 32x32. The second layer also uses a 5x5 convolution kernel (with 32 input and output channels), with the same stride and padding as the first layer. After batch normalization and ReLU activation, a 2x2 max pooling layer further reduces the feature map size to 16x16. The third layer uses a 5x5 convolution kernel (with 32 input channels and 64 output channels), with the same stride and padding. After batch normalization and ReLU activation, a 2x2 max pooling layer compresses the feature map size to 8x8. The classification layer first converts the 8x8x64 feature map into a 16384-dimensional vector through a flattening operation, then compresses the dimension to 64 dimensions through a fully connected layer, and finally maps the 64-dimensional vector to a 2-dimensional classification result (corresponding to category A / B) through the output layer.

[0068] A binary cross-entropy loss function (BCEWithLogitsLoss) was used for classification tasks, and network parameters were updated using a stochastic gradient descent optimizer (SGD), with an initial learning rate of 0.01. During training, the model inputs block data for forward propagation, calculates the loss between the predicted values ​​and the true labels, and optimizes the model weights through backpropagation to improve classification accuracy.

[0069] After inputting the current MbxNb block into the trained model, the raw output value is obtained through forward propagation. The category is determined by the maximum probability index of the output value (0 and 1 correspond to category A and B, respectively). The confidence G is converted into a probability distribution (range 0.5-1) using the Softmax function to quantify the reliability of the prediction result.

[0070] In the embodiment of the present invention, the combination of forward propagation and back propagation enables the network to automatically adjust in each training, gradually optimize the model, reduce errors and ultimately achieve ideal classification results.

[0071] S5: Determine the second category classification results of each target block through the improved convolutional neural network model.

[0072] Convolutional Neural Networks (CNNs) are deep learning models widely used in fields such as image processing, video analysis, and natural language processing. The core concept of CNNs is to mimic the structure of biological visual systems. They extract spatial features from data through multiple layers of convolutional and pooling layers and map them to higher-level semantics through fully connected layers. CNNs are particularly well-suited for image recognition tasks because they automatically extract local features from images and gradually construct more complex feature representations through a hierarchical structure.

[0073] In a possible implementation, S5 specifically includes:

[0074] S501: Input each target block into the first feature extraction layer for feature extraction, and pass through the batch normalization layer, activation layer and maximum pooling layer in sequence to output the first pooling feature map.

[0075] S502: Input the first pooled feature map into the second feature extraction layer for feature extraction, and sequentially perform batch normalization layer, activation layer and maximum pooling layer to output the second pooled feature map.

[0076] S503: Input the second pooled feature map into the third feature extraction layer for feature extraction, and sequentially perform batch normalization layer, activation layer and maximum pooling layer to output the third pooled feature map.

[0077] In an embodiment of the present invention, each layer of convolution and pooling operations gradually extracts complex features, enabling the network to gradually build a deeper understanding of the image content, from simple low-level features to more abstract high-level semantics.

[0078] S504: Flatten the third pooled feature map through the flattening operation of the classification layer, and compress the dimension through the fully connected layer.

[0079] In the embodiment of the present invention, the flattening and full connection operations integrate the features extracted from multiple layers, providing a rich feature representation for the final classification decision, which helps to improve the accuracy of classification.

[0080] S505: Map the feature map after the compressed dimension to two dimensions through the output layer to obtain the second category classification result.

[0081] In the embodiment of the present invention, through the Softmax output layer, the model not only gives the classification results, but also provides confidence for each category to help make decisions.

[0082] S6: Based on the first category classification result and the second category classification result, the final category of each target block is determined by a preset classification rule.

[0083] In a possible implementation, the preset classification rules specifically include:

[0084] When the first category classification result is consistent with the second category classification result, and the confidence of the improved convolutional neural network model is greater than the preset confidence, the second category classification result is used as the final category; otherwise, the processing method when the first category classification result is inconsistent with the second category classification result is adopted.

[0085] Optionally, the preset confidence level is 0.9.

[0086] In an embodiment of the present invention, when the classification result of the first category is consistent with the classification result of the second category, and the confidence of the CNN model is greater than the preset confidence, directly adopting the classification result of CNN helps to give full play to the advantages of deep learning models in complex pattern recognition.

[0087] Optionally, fusion confidence F = weight S1×H (fixed confidence of feature classification) + weight S2×G (dynamic confidence of learning classification), where weight S1 is the credible proportion of feature classification confidence in fusion confidence, and weight S2 is the credible proportion of learning classification dynamic confidence in fusion confidence.

[0088] Specifically, the dynamic confidence of the learned classification is the confidence obtained through the Softmax output layer.

[0089] Specifically, the feature classification is fixed with confidence = 1, (weight S1 > weight S2).

[0090] Optionally, when the strong condition of feature classification is met, the weight S1=0.8 and the weight S2=0.2.

[0091] It should be noted that those skilled in the art can set the ratio of the weights S1 and S2 according to actual needs, and the present invention is not limited thereto.

[0092] If the first and second classification results are inconsistent, the fusion confidence is calculated to determine whether it is greater than or equal to a preset confidence ratio. If so, the first classification result is used as the final classification. Otherwise, the second classification result is used as the final classification.

[0093] Optionally, the confidence ratio is preset to 0.7.

[0094] It should be noted that those skilled in the art can set the size of the preset confidence ratio according to actual needs, and the present invention is not limited thereto.

[0095] For example, in case 1, the judgment when the feature classification (first category classification result) is consistent with the learning classification (second category classification result) is as follows:

[0096] If both the feature classification and the learned classification are natural content (classification A=1 and class B=1), or both are computer-generated content (classification A=0 and class B=0) and the model's confidence G is greater than or equal to 0.9, the confidence calculation step is skipped and the learned classification result is directly used as the final output category C. Otherwise, dynamic adjustments will be made.

[0097] Case 2: Dynamic adjustment when feature classification conflicts with learning classification:

[0098] If the feature classification result is 0 (classification A) and the learned classification result is 1 (classification B), the fused confidence is calculated using Formula 1: Confidence = 0.8 × 1 (fixed confidence of feature classification) + 0.2 × G (dynamic confidence of learned classification). For other cases, the fused confidence is calculated using Formula 2: Confidence = 0.2 × 1 (fixed confidence of feature classification) + 0.8 × G (dynamic confidence of learned classification).

[0099] If the learned classification result (classification B=1) and the real-time confidence G of the learned classification is greater than 0.6, the confidence is calculated according to Formula 2 for judgment: when the new confidence is greater than or equal to the confidence ratio K, the final category C is adopted as natural content (C=1); if the new confidence is less than the confidence ratio 0.7, computer-generated content is adopted (C=0).

[0100] If all of the above conditions are not met, the new confidence level is calculated according to Formula 2 for judgment: when the new confidence level is greater than or equal to the confidence ratio of 0.7, the final category C is adopted as computer-generated content (C=0); if the new confidence level is less than the confidence ratio of 0.7, natural content is adopted (C=1).

[0101] In an embodiment of the present invention, by setting a preset confidence ratio, the system can adaptively select the most appropriate classification result when processing complex or uncertain scenarios, so that the system can dynamically adjust the classification decision when facing different data or complex scenarios, thereby improving the flexibility and adaptability of the system.

[0102] Optionally, after S6 , the method further includes: using different preset encoders for encoding according to the classification type, using preset encoder 1 when category C=1 (natural content), and using preset encoder 2 when category C=0 (computer content).

[0103] It should be noted that the preset encoder 1 is an encoder that does not include (or disables) the screen content-specific tool, and the preset encoder 2 is an encoder that includes (or enables) the screen content-specific tool.

[0104] In the embodiment of the present invention, different encoders are selected to optimize the encoding strategy in a targeted manner, thereby achieving higher compression efficiency and avoiding unnecessary computational waste.

[0105] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0106] In an embodiment of the present invention, by adopting a multi-feature-based classification method to determine the first category classification results of each of the target blocks, the adaptability to complex image areas is enhanced, and the local mixed distribution state of natural content and screen content in the image can be accurately identified. An improved convolutional neural network model is introduced to determine the second category classification results of each of the target blocks, thereby realizing high-level semantic classification of the target blocks. By fusing the first category and second category classification results, the coordinated optimization of traditional features and deep features is achieved, thereby significantly improving the image content recognition accuracy in complex scenes.

[0107] Reference Manual Figure 2 , showing a structural diagram of a screen mixed content classification system for efficient compression provided by an embodiment of the present invention.

[0108] The embodiment of the present invention provides a screen mixed content classification system 20 for efficient compression, comprising: a processor 201 and a memory 202;

[0109] The memory 202 stores programs or instructions that can be run on the processor 201. When the programs or instructions are executed by the processor 201, the steps of the above-mentioned screen mixed content classification method for efficient compression are implemented, and the same technical effect can be achieved. To avoid repetition, the present invention will not be described in detail.

[0110] It should be understood that the processor 201 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0111] It should also be understood that the memory 202 in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0112] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0113] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0114] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0115] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0116] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0117] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0118] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0119] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0120] An embodiment of the present invention provides a readable storage medium including: a program or instruction is stored on the readable storage medium, and when the program or instruction is executed by the processor, the steps of the above-mentioned screen mixed content classification method for efficient compression are implemented, and the same technical effect can be achieved. To avoid repetition, the present invention will not be described in detail.

[0121] Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents; and such modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or replacements that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be covered by the scope of protection of the present invention.

Claims

1. A screen mixed content classification method for efficient compression, characterized in that: include: S1: Get the screen mixed content image; S2: performing block processing on the screen mixed content image to obtain multiple target blocks; S3: Determine a first category classification result for each target block in a feature-based classification manner; S4: Build an improved convolutional neural network model; S5: Determine the second category classification result of each target block through the improved convolutional neural network model; S6: Based on the first category classification result and the second category classification result, determine the final category of each of the target blocks according to a preset classification rule; The preset classification rules specifically include: When the first category classification result is consistent with the second category classification result, and the confidence of the improved convolutional neural network model is greater than a preset confidence, the second category classification result is used as the final category; otherwise, the processing method when the first category classification result is inconsistent with the second category classification result is adopted; Preset reliability = 0.9; Fusion confidence F = weight S1 × H + weight S2 × G, where H represents the fixed confidence of feature classification, weight S1 represents the credible proportion of feature classification confidence in fusion confidence, G represents the dynamic confidence of learning classification, and weight S2 represents the credible proportion of dynamic confidence of learning classification in fusion confidence. The dynamic confidence of the learned classification is the confidence obtained through the Softmax output layer; The feature classification fixed confidence = 1, weight S1> weight S2; When the strong condition of feature classification is met, the weight S1=0.8 and the weight S2=0.2; When the first category classification result is inconsistent with the second category classification result, calculate the fusion confidence and determine whether the fusion confidence is greater than or equal to a preset confidence ratio; if so, take the first category classification result as the final category; otherwise, take the second category classification result as the final category.

2. The screen mixed content classification method for efficient compression according to claim 1, characterized in that: The S3 specifically includes: S301: performing block processing on each of the target blocks to obtain multiple target sub-blocks; S302: Calculating the characteristic value of each target sub-block; S303: Determine the classification category of the current target sub-block according to the feature values ​​of each target sub-block and in combination with preset conditions; S304: Determine a first category classification result of each target sub-block according to the classification category of each target sub-block.

3. The screen mixed content classification method for efficient compression according to claim 2, characterized in that: The characteristic values ​​specifically include: Number of colors, entropy, horizontal gradient mutation value, vertical gradient mutation value, uniformity, and hash value.

4. The screen mixed content classification method for efficient compression according to claim 3, characterized in that: The uniformity is calculated as follows: ； in, Evenness Indicates uniformity, pix max Indicates the maximum value within the block, pix min represents the minimum value within the block, pix aver represents the mean within the block; The hash value is calculated as follows: ； Among them, HashValue represents the hash value, evenness represents the uniformity within the block, Ttcolor represents the number of colors within the block, gradXsp represents the horizontal gradient mutation value within the block, gradYsp represents the vertical gradient mutation value within the block, and avgDC represents the average number of colors per unit pixel in the block.

5. The screen mixed content classification method for efficient compression according to claim 2, characterized in that: The preset conditions specifically include: The number of hash hits formed by counting the total number of pixels having the same hash value in each target sub-block is used as the feature value; if the number of hash hits is less than a preset number of hash hits, the target sub-block is classified as natural content; otherwise, the target sub-block is classified as computer content; and / or arranging the number of occurrences of multiple pixels in each frame of the screen mixed content image in descending order, and counting the number of occurrences of the top 10 pixels; if the number of occurrences is less than a preset number of occurrences, classifying the target sub-block as natural content; otherwise, classifying the target sub-block as computer content.

6. The screen mixed content classification method for efficient compression according to claim 2, characterized in that: The S304 specifically includes: S3041: Calculating a target ratio of the number of target sub-blocks whose categories are natural content to the total number of sub-blocks; S3042: Determine whether the target ratio is greater than or equal to a preset ratio; if so, classify the target sub-blocks corresponding to the target sub-blocks as natural content; otherwise, classify the sub-blocks as screen content; S3043: Summarize the classification results of each of the target blocks to obtain the first category classification result.

7. The screen mixed content classification method for efficient compression according to claim 1, characterized in that: The improved convolutional neural network model specifically includes: a first feature extraction layer, a batch normalization layer, an activation layer, and a maximum pooling layer connected in sequence; The maximum pooling layer is connected to the second feature extraction layer and the third feature extraction layer respectively; The second feature extraction layer and the third feature extraction layer are both connected to the batch normalization layer; The maximum pooling layer is connected to the output layer through the classification layer and the fully connected layer in sequence.

8. The screen mixed content classification method for efficient compression according to claim 7, characterized in that: The S5 specifically includes: S501: Input each of the target blocks into the first feature extraction layer for feature extraction, and sequentially pass through the batch normalization layer, the activation layer, and the maximum pooling layer to output a first pooling feature map; S502: Input the first pooled feature map into the second feature extraction layer for feature extraction, and sequentially perform a batch normalization layer, an activation layer, and a maximum pooling layer to output a second pooled feature map; S503: Input the second pooled feature map to the third feature extraction layer for feature extraction, and sequentially perform a batch normalization layer, an activation layer, and a maximum pooling layer to output a third pooled feature map; S504: Flatten the third pooling feature map through the flattening operation of the classification layer, and compress the dimension through the fully connected layer; S505: Mapping the feature map after dimension compression to two dimensions through the output layer to obtain the second category classification result.

9. A screen mixed content classification system for efficient compression, characterized in that: include: processor and memory; The memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the screen mixed content classification method for efficient compression according to any one of claims 1 to 8 are implemented.

Images