Method, system and readable storage medium for allocating code rate in image compression

Abstract

The present invention relates to a method, system and readable storage medium for allocating bitrate in image compression, the method comprising: evenly dividing an image to be encoded into a plurality of image blocks; using a plurality of texture complexity statistical algorithms to calculate a plurality of measurement indicators for measuring texture complexity of each image block, and fusing the plurality of measurement indicators of each image block in combination with the optimal human eye subjective evaluation result to obtain the fused texture complexity of each image block; allocating bitrate to each image block based on the encoding order of the image to be encoded, and allocating bitrate to each image block according to the maximum total bitrate under the limitation of storage space or transmission channel bandwidth and the fused texture complexity of each image block. The present invention combines the advantages of different texture complexity statistical algorithms and the optimal human eye subjective evaluation result to fuse a plurality of measurement indicators to allocate corresponding bitrates to different image blocks, and can minimize the bitrate required for compression under the premise of ensuring the subjective effect of reconstructing the image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, system and readable storage medium for allocating bit rates in image compression. Background Art

[0002] Image compression technology is an important technology in the field of image processing. With the continuous growth of image resolution and bit width and the increasing richness of image formats, the amount of data of original images is growing exponentially, and the memory space required to store original images and the bandwidth required to transmit original images are increasing accordingly. In order to alleviate the pressure imposed by the huge amount of data of original images on storage and transmission scenarios, image compression / encoding technology that compresses original images is very necessary.

[0003] In this context, various major standard organizations and video / image processing companies have proposed many image compression technologies and products. The evaluation indicators for various technologies and products depend on the application needs of the downstream industries of the image compression system. Usually, the downstream of the image compression system is the vast consumer group, who need to watch high-definition images, so the subjective evaluation of the vast number of customers has become a major criterion for measuring various technologies / products.

[0004] Based on the above application requirements, the optimization goal of image compression technology can be defined as maximizing the compression rate while optimizing the subjective evaluation result of the human eye. The compression rate can be defined as the ratio of the amount of data in the original image to the amount of data after image compression (i.e., bit rate). Therefore, maximizing the compression rate means minimizing the bit rate after compression. The most direct optimization method is to reduce the bit rate in the area where the human eye is subjectively insensitive until it is close to the distortion that the customer can feel, and allocate more bit rate in the area where the human eye is subjectively interested to obtain the best subjective effect.

[0005] A series of experiments show that distortion in flat areas is easy to detect and has poor subjective effects, while distortion in complex areas is relatively difficult to detect, and it is acceptable to sacrifice a little subjective quality in order to save bitrate. Therefore, how to quantitatively evaluate the complexity of textures and adjust the bitrate accordingly is the key to solving the problem.

[0006] In the research field related to texture complexity, there are many indicators for measuring texture complexity, which can be basically divided into the following four categories:

[0007] 1) Statistical analysis method: Compare the statistical results of pixel grayscale values ​​with the preset threshold to distinguish the complexity of texture, such as first-order moment (mean), second-order moment (variance), gradient histogram (HOG), local binary pattern matching (LBP), etc.

[0008] 2) Structural analysis method: Combine the erosion, expansion, and opening and closing operators in graphics to find areas in the image that match pre-defined texture primitives; this method is often used to find specific texture areas, but can also be used to screen complex texture structures and flat texture structures;

[0009] 3) Spectral analysis method: Spectral analysis based on filters. Common methods include Hilbert, Fourier, Gabor, wavelet functions, etc.

[0010] 4) Model analysis method: Use a pre-defined function model to express texture. Different textures correspond to different function models or different model parameters. Use the function model type to which the texture belongs or the threshold interval to which the parameter belongs to measure the texture complexity. Common methods include Gaussian mixture model (GMM), Laplace model, segmentation model (SAM), etc.

[0011] Among the above four methods, the third type of spectral analysis method performs statistical analysis in the frequency domain, and the other three types of methods perform statistical analysis in the spatial domain; the first type of statistical analysis method usually performs statistics in units of pixels or image blocks, the second type of structural analysis method requires pre-defined texture patterns, usually textures of irregular sizes, and also supports regular texture patterns; the fourth type of model analysis method usually explores the functional relationship corresponding to multiple consecutive pixel values ​​in a single direction.

[0012] However, the above four methods are usually used to screen the texture patterns that need to be explored, and are not integrated into the encoding compression process to guide the bit rate allocation. In addition, the core focus of various texture complexity statistical methods is different, and different texture complexity statistical results will have different effects on the bit rate allocation. Summary of the invention

[0013] The purpose of the present invention is to provide a method, system and readable storage medium for bit rate allocation in image compression, aiming to quickly and efficiently distinguish texture complexity and then guide bit rate allocation in the encoding process.

[0014] To achieve the above object, the present invention provides a method for allocating bit rates in image compression, comprising:

[0015] Evenly divide the image to be encoded into several image blocks;

[0016] A plurality of texture complexity statistical algorithms are used to calculate a plurality of measurement indicators for measuring texture complexity of each image block, and the multiple measurement indicators of each image block are fused in combination with the optimal human eye subjective evaluation result to obtain the fused texture complexity of each image block;

[0017] The code rate is allocated to each image block based on the coding order of the image to be coded, and the code rate is allocated to each image block according to the maximum total code rate under the limitation of storage space or transmission channel bandwidth and the fusion texture complexity of each image block.

[0018] Optionally, the size of the image block is M*N, and M is equal to N and is a power of 2.

[0019] Optionally, for each image block, the measurement index includes the first-order moment, second-order moment and information entropy of all pixels in each image block.

[0020] Optionally, for each image block of size M*N, the calculation formula of the first-order moment μ is as follows:

[0021]

[0022] In the formula, I ij represents the pixel value at the position (i, j) in the image block;

[0023] The calculation formula of the second-order moment σ is as follows:

[0024]

[0025] The calculation formula of the information entropy is as follows:

[0026]

[0027] Wherein, h(p) represents the number of pixels with a pixel value of p in the grayscale histogram corresponding to each image block, and pmax is the maximum pixel value in the grayscale histogram.

[0028] Optionally, when calculating the information entropy of the image block:

[0029] Normalize the pixel values ​​in the image block from [0, 255] to [0, pmax], then:

[0030]

[0031] pmax=255 / 2 s

[0032] I′ ij =I ij / 2 s

[0033]

[0034] Wherein, s is a positive integer greater than 0.

[0035] Optionally, when calculating the information entropy, if h(p)=0 during the accumulation process, no accumulation is performed and the process is skipped.

[0036] Optionally, when fusing multiple measurement indicators of each image block in combination with the optimal human eye subjective evaluation result, the sensitivity of the human eye to the highlights, shadows, flat areas and complex areas of each image block and the amount of information are considered, and the calculation formula for the fused texture complexity f(μ, σ, E) of each image block is obtained as follows:

[0037] f(μ,σ,E)=128-|μ-128|-10σ+16E.

[0038] Optionally, the specific method of allocating code rates to each image block is as follows:

[0039]

[0040] In the formula, R represents the bit rate assigned to the current image block, R left represents the maximum total bit rate minus the bit rate consumed by the encoded image block, f(μ, σ, E) represents the fused texture complexity of the current image block, and ∑f(μ, σ, E) represents the sum of the fused texture complexities of the remaining image blocks to be encoded.

[0041] Based on the same technical concept, the present invention also provides a bit rate allocation system in image compression, which is used to implement the bit rate allocation method in image compression as described above, comprising:

[0042] An image segmentation module is configured to evenly segment the image to be encoded into a plurality of image blocks;

[0043] A calculation module is configured to calculate a plurality of measurement indicators for measuring texture complexity of each image block using a plurality of texture complexity statistical algorithms, and fuse the plurality of measurement indicators of each image block in combination with an optimal human eye subjective evaluation result to obtain a fused texture complexity of each image block;

[0044] The bit rate allocation module is configured to allocate bit rates to each image block based on the encoding order of the image to be encoded, and allocate bit rates to each image block according to the maximum total bit rate under the storage space or transmission channel bandwidth limitation and the fusion texture complexity of each image block.

[0045] Based on the same technical concept, the present invention also provides a readable storage medium on which a computer program is stored. When the computer program is executed, the bit rate allocation method in image compression as described above can be implemented.

[0046] In a method, system and readable storage medium for allocating bit rates in image compression provided by the present invention, a plurality of texture complexity statistical algorithms are pre-adopted in the image compression process to obtain a measurement index for measuring texture complexity of each region (i.e., image block) in the image to be encoded, and then the advantages of different texture complexity statistical algorithms and the optimal human eye subjective evaluation results are combined to fuse the plurality of measurement indexes to allocate corresponding bit rates to different regions, thereby minimizing the bit rate required for compression while ensuring the subjective effect of reconstructing the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Those skilled in the art will appreciate that the accompanying drawings are provided for a better understanding of the present invention and do not constitute any limitation on the scope of the present invention.

[0048] Figure 1 The present invention is a flowchart of a method for allocating bit rates in image compression according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, advantages and features of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings are in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, please refer to the accompanying drawings. It should be noted that the structure, proportion, size, etc. illustrated in the drawings of this specification are only used to match the content disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention. Any modification of the structure, change in the proportional relationship or adjustment of the size, under the same or similar conditions as the effects that can be produced by the present invention and the purposes that can be achieved, should still fall within the scope of the technical content disclosed by the present invention.

[0050] As used in the present invention, the singular forms "a", "an", and "the" include plural referents unless the context clearly indicates otherwise. As used in the present invention, the term "or" is generally used in a sense that includes "and / or", unless the context clearly indicates otherwise. As used in the present invention, the term "several" is generally used in a sense that includes "at least one", unless the context clearly indicates otherwise. As used in the present invention, the term "at least two" is generally used in a sense that includes "two or more", unless the context clearly indicates otherwise.

[0051] Please refer to Figure 1 The embodiment of the present invention provides a method for allocating bit rates in image compression, comprising the following steps:

[0052] S1, evenly divide the image to be encoded into several image blocks;

[0053] S2, using a variety of texture complexity statistical algorithms to calculate multiple measurement indicators for measuring texture complexity of each image block, and combining the multiple measurement indicators of each image block with the optimal human eye subjective evaluation result to obtain the fused texture complexity of each image block;

[0054] S3. Allocate code rates for each image block based on the coding order of the image to be encoded, and allocate code rates for each image block according to the maximum total code rate under the limitation of storage space or transmission channel bandwidth and the fusion texture complexity of each image block.

[0055] By pre-adopting a variety of texture complexity statistical algorithms during the image compression process to obtain measurement indicators for measuring texture complexity in each area (i.e., image block) in the image to be encoded, and then combining the advantages of different texture complexity statistical algorithms and the optimal human eye subjective evaluation results to fuse multiple measurement indicators to allocate corresponding bit rates to different areas, thereby minimizing the bit rate required for compression while ensuring the subjective effect of reconstructing the image.

[0056] First, execute S1 to evenly divide the image to be encoded into a number of image blocks. In this embodiment, the size of the image block is M*N, and M is equal to N and is a power of 2, that is, the image to be encoded is divided into a number of regions of size M*N. Preferably, M=N=2 4 .

[0057] Then, S2 is executed to calculate multiple metrics for measuring texture complexity of each image block using a variety of texture complexity statistical algorithms, and the multiple metrics of each image block are fused in combination with the optimal human eye subjective evaluation result to obtain the fused texture complexity of each image block.

[0058] In this embodiment, for each image block, the measurement index includes the first-order moment, second-order moment and information entropy of all pixels in each image block. Of course, in addition to the three measurement indexes listed above, more measurement indexes can be calculated according to needs, such as Fourier coefficients, wavelet coefficients and elementary functions thereof, etc., and the present invention does not limit this. It should be noted that there are differences in the core concerns of different texture complexity statistical algorithms. Different control strategies and weight parameters are required when adjusting the bit rate using different measurement indexes. Therefore, this embodiment comprehensively considers the advantages of multiple texture complexity statistical algorithms to achieve the optimal subjective effect, so as to quickly and efficiently distinguish texture complexity and guide the bit rate allocation in the encoding process.

[0059] Specifically, for each image block of size M*N, the calculation formula of the first-order moment μ is as follows:

[0060]

[0061] In the formula, I ij represents the pixel value at the position (i, j) in the image block;

[0062] The calculation formula of the second-order moment σ is as follows:

[0063]

[0064] The calculation formula of the information entropy is as follows:

[0065]

[0066] Wherein, h(p) represents the number of pixels with a pixel value of p in the grayscale histogram corresponding to each image block, and pmax is the maximum pixel value in the grayscale histogram.

[0067] Preferably, when calculating the information entropy, if h(p)=0 during the accumulation process, no accumulation is performed and the process is skipped.

[0068] Preferably, when calculating the information entropy of the image block:

[0069] Normalize the pixel values ​​in the image block from [0, 255] to [0, pmax], then:

[0070]

[0071] pmax=255 / 2 s

[0072] I′ ij =I ij / 2 s

[0073]

[0074] Wherein, s is a positive integer greater than 0.

[0075] The purpose of this step is to improve the statistical speed of information entropy, where pmax and I′ ij For example, if s=4, then pmax is 255 which is evenly divisible by 2. 4 is equal to 15, that is, the pixel value in the image block is normalized from [0, 255] to [0, 15], that is, the pixel value in the image block I ij Pixel-by-pixel divisibility of 2 4 Get the normalized pixel value I′ ij . Then we have:

[0076]

[0077] The h(p) in the formula represents the number of pixels with pixel value p in the grayscale histogram of the image block after normalization.

[0078] In this embodiment, when the multiple measurement indicators of each image block are fused in combination with the optimal human eye subjective evaluation result, the sensitivity of the human eye to the highlights, shadows, flat areas and complex areas of each image block and the amount of information are considered, and the calculation formula for the fused texture complexity f(μ, σ, E) of each image block is obtained as follows:

[0079] f(μ,σ,E)=128-|μ-128|-10σ+16E

[0080] Specifically, considering that the human eye is insensitive to highlights (μ approaches 255) and shadows (μ approaches 0), less bit rate can be allocated; the human eye is sensitive to flat areas (σ approaches 0), so more bit rate needs to be allocated, and is insensitive to complex areas (σ is extremely large), so less bit rate can be allocated; and the higher the information entropy (E), the greater the amount of information, and more bit rate needs to be allocated.

[0081] Finally, S3 is executed to allocate code rates to each image block based on the coding order of the image to be coded, and code rates are allocated to each image block according to the maximum total code rate under the storage space or transmission channel bandwidth limitation and the fusion texture complexity of each image block.

[0082] In this embodiment, the specific manner of allocating bit rates to each image block is as follows:

[0083]

[0084] In the formula, R represents the bit rate assigned to the current image block, R left represents the maximum total bit rate minus the bit rate consumed by the encoded image block, f(μ, σ, E) represents the fused texture complexity of the current image block, and ∑f(μ, σ, E) represents the sum of the fused texture complexities of the remaining image blocks to be encoded.

[0085] It should be noted that after allocating bit rates to each image block, the subsequent process of adopting any bit rate control technology to compress the image at a given bit rate is not within the protection scope of the present invention and will not be described in detail in the present invention.

[0086] Based on the same inventive concept, an embodiment of the present invention further proposes a bit rate allocation system in image compression, which is used to implement the bit rate allocation method in image compression as described above, including:

[0087] An image segmentation module is configured to evenly segment the image to be encoded into a plurality of image blocks;

[0088] A calculation module is configured to calculate a plurality of measurement indicators for measuring texture complexity of each image block using a plurality of texture complexity statistical algorithms, and fuse the plurality of measurement indicators of each image block in combination with an optimal human eye subjective evaluation result to obtain a fused texture complexity of each image block;

[0089] The bit rate allocation module is configured to allocate bit rates to each image block based on the encoding order of the image to be encoded, and allocate bit rates to each image block according to the maximum total bit rate under the storage space or transmission channel bandwidth limitation and the fusion texture complexity of each image block.

[0090] Based on the same inventive concept, an embodiment of the present invention further provides a readable storage medium having a computer program stored thereon, and when the computer program is executed, the method for allocating bit rate in image compression as described above can be implemented.

[0091] The readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device, such as but not limited to an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device or any suitable combination thereof. The more specific example (non-exhaustive list) of the readable storage medium includes: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer program described herein can be downloaded from the readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer program from the network and forwards the computer program for storage in the readable storage medium in each computing / processing device. The computer program for performing the operation of the present invention can be a run instruction, an instruction set architecture (ISA) instruction, a machine instruction, a machine-related instruction, a microcode, a firmware instruction, a state setting data, or a source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer program can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet). In some embodiments, by utilizing the state information of a computer program to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute computer-readable program instructions to implement various aspects of the present invention.

[0092] Here, various aspects of the present invention are described with reference to the flowchart and / or block diagram of the method, system and computer program product according to the embodiment of the present invention. It should be understood that each square frame of the flowchart and / or block diagram and the combination of the square frames in the flowchart and / or block diagram can be realized by a computer program. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so as to produce a machine, so that these programs, when executed by the processor of a computer or other programmable data processing device, produce a device for realizing the function / action specified in one or more square frames in the flowchart and / or block diagram. These computer programs can also be stored in a readable storage medium, and these computer programs make the computer, programmable data processing device and / or other equipment work in a specific way, so that the readable storage medium storing the computer program includes a manufactured product, which includes instructions for realizing various aspects of the function / action specified in one or more square frames in the flowchart and / or block diagram.

[0093] The computer program may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are executed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the computer program executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0094] In summary, the embodiments of the present invention provide a method, system and readable storage medium for bit rate allocation in image compression, which obtains a measurement index for measuring texture complexity of each region (i.e., image block) in the image to be encoded by pre-adopting multiple texture complexity statistical algorithms during the image compression process, and then combines the advantages of different texture complexity statistical algorithms and the optimal human eye subjective evaluation results to fuse multiple measurement indexes to allocate corresponding bit rates to different regions, thereby minimizing the bit rate required for compression while ensuring the subjective effect of reconstructing the image.

[0095] The above description is only a description of the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any changes and modifications made by a person skilled in the art in the field of the present invention based on the above disclosure are within the scope of protection of the present invention. Obviously, a person skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for allocating bit rate in image compression, characterized in that: include: Evenly divide the image to be encoded into several image blocks; A plurality of texture complexity statistical algorithms are used to calculate a plurality of measurement indicators for measuring texture complexity of each image block, and the multiple measurement indicators of each image block are fused in combination with the optimal human eye subjective evaluation result to obtain the fused texture complexity of each image block; The code rate is allocated to each image block based on the coding order of the image to be coded, and the code rate is allocated to each image block according to the maximum total code rate under the limitation of storage space or transmission channel bandwidth and the fusion texture complexity of each image block.

2. The method for allocating bit rate in image compression according to claim 1, characterized in that: The size of the image block is M*N, where M is equal to N and is a power of 2.

3. The method for allocating bit rate in image compression according to claim 1, characterized in that: For each image block, the measurement index includes the first-order moment, the second-order moment and the information entropy of all pixels in each image block.

4. The method for allocating bit rate in image compression according to claim 3, characterized in that: For each image block of size M*N, the calculation formula of the first-order moment μ is as follows: In the formula, I ij represents the pixel value at the position (i, j) in the image block; The calculation formula of the second-order moment σ is as follows: The calculation formula of the information entropy is as follows: Wherein, h(p) represents the number of pixels with a pixel value of p in the grayscale histogram corresponding to each image block, and pmax is the maximum pixel value in the grayscale histogram.

5. The method for allocating bit rate in image compression according to claim 4, characterized in that: When calculating the information entropy of the image block: Normalize the pixel values ​​in the image block from [0, 255] to [0, pmax], then: pmax=255 / 2 s I′ ij =I ij / 2 s Wherein, s is a positive integer greater than 0.

6. The method for allocating bit rate in image compression according to claim 4, characterized in that: When calculating the information entropy, if h(p)=0 during the accumulation process, no accumulation is performed and the process is skipped.

7. The method for allocating bit rate in image compression according to claim 4, characterized in that: When the multiple measurement indicators of each image block are fused in combination with the optimal human eye subjective evaluation result, the sensitivity of the human eye to the highlights, shadows, flat areas and complex areas of each image block and the amount of information are considered, and the calculation formula for the fused texture complexity f(μ, σ, E) of each image block is obtained as follows: f(μ,σ,E)=128-|μ-128|-10σ+16E.

8. The method for allocating bit rate in image compression according to claim 1, characterized in that: The specific method of allocating bit rates to each image block is as follows: In the formula, R represents the bit rate assigned to the current image block, R left represents the maximum total bit rate minus the bit rate consumed by the encoded image block, f(μ, σ, E) represents the fused texture complexity of the current image block, and ∑f(μ, σ, E) represents the sum of the fused texture complexities of the remaining image blocks to be encoded.

9. A bit rate allocation system in image compression, used to implement the bit rate allocation method in image compression according to any one of claims 1 to 8, characterized in that: include: An image segmentation module is configured to evenly segment the image to be encoded into a plurality of image blocks; A calculation module is configured to calculate a plurality of measurement indicators for measuring texture complexity of each image block using a plurality of texture complexity statistical algorithms, and fuse the plurality of measurement indicators of each image block in combination with an optimal human eye subjective evaluation result to obtain a fused texture complexity of each image block; The bit rate allocation module is configured to allocate bit rates to each image block based on the encoding order of the image to be encoded, and allocate bit rates to each image block according to the maximum total bit rate under the storage space or transmission channel bandwidth limitation and the fusion texture complexity of each image block.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the method for allocating bit rate in image compression according to any one of claims 1 to 8 can be implemented.

Images