Image quality evaluation method, device, equipment and computer readable storage medium

Abstract

The application discloses an image quality evaluation method, device and equipment and a computer readable storage medium, wherein the method comprises the following steps: acquiring a text region and an image region corresponding to a to-be-processed image, and acquiring a first expected value corresponding to the text region and a second expected value corresponding to the image region; determining a first gradient corresponding to the image region based on a first edge operator and determining a second gradient corresponding to the text region based on a second edge operator; determining an edge similarity between the to-be-processed image and a reference image based on the first gradient, the second gradient, the first expected value and the second expected value; and inputting the edge similarity into a nonlinear mapping function to obtain an image quality evaluation result corresponding to the to-be-processed image. The application fully considers different field sizes of HVS and the significant features of the edges of the image, and improves the accuracy and efficiency of image quality evaluation.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image quality evaluation method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Currently, with the development of internet technology and the widespread use of modern multimedia communication terminals, users are able to perform many complex communication tasks. The visual content presented to consumers by these mobile terminals during use is no longer a single natural image, but a composite image containing text, images, maps, and computer-generated graphics. Images similar to the above visual content are generally called screen images. Screen images inevitably suffer distortion during generation, processing, and transmission, resulting in a decrease in image quality and ultimately affecting the user experience. Over the past few decades, many objective image quality assessment methods have been proposed. These methods perform well in assessing the quality of natural images, but their results in assessing the quality of screen images are poor. This is because screen content images have a more complex composition than natural images.

[0003] However, screen content images not only contain natural images but also include various computer-generated components such as text, graphics, and icons. This results in screen images typically containing text, charts, and natural images, often featuring very sharp edges, relatively simple shapes, thin lines with minimal color, or even single-pixel-wide monochrome lines. With the widespread application of deep learning in computer vision, researchers have proposed many deep learning networks to address image quality assessment problems, achieving performance superior to traditional algorithms. When directly training existing convolutional neural network models on screen images, the network models struggle to simultaneously learn and distinguish features from different regions, such as natural and non-natural images, due to the unique and complex structural characteristics of screen images, thus failing to accurately assess image quality. Summary of the Invention

[0004] The main objective of this invention is to provide an image quality evaluation method, apparatus, device, and computer-readable storage medium, aiming to solve the technical problem of inaccurate image quality evaluation of existing screen images.

[0005] To achieve the above objectives, the present invention provides an image quality evaluation method, which includes the following steps:

[0006] Obtain the text region and image region corresponding to the image to be processed, and obtain the first expected value corresponding to the text region and the second expected value corresponding to the image region;

[0007] A first gradient corresponding to the image region is determined based on a first edge operator, and a second gradient corresponding to the text region is determined based on a second edge operator, wherein the kernel of the first edge operator is greater than the kernel of the second edge operator;

[0008] Based on the first gradient, the second gradient, the first expected value, and the second expected value, the edge similarity between the image to be processed and the reference image is determined.

[0009] The edge similarity is input into a nonlinear mapping function to obtain the image quality evaluation result corresponding to the image to be processed.

[0010] Further, the steps of determining the first gradient corresponding to the image region based on the first edge operator and determining the second gradient corresponding to the text region based on the second edge operator include:

[0011] Based on the first horizontal operator of the first edge operator, the first horizontal edge gradient corresponding to the image region is determined, and based on the first vertical operator of the first edge operator, the first vertical edge gradient corresponding to the image region is determined.

[0012] Based on the second horizontal operator of the second edge operator, the second horizontal edge gradient corresponding to the text region is determined, and based on the second vertical operator of the second edge operator, the second vertical edge gradient corresponding to the text region is determined.

[0013] The first gradient is determined based on the first horizontal edge gradient and the first vertical edge gradient, and the second gradient is determined based on the second horizontal edge gradient and the second vertical edge gradient.

[0014] Further, the step of determining the edge similarity between the image to be processed and the reference image based on the first gradient, the second gradient, the first expected value, and the second expected value includes:

[0015] Based on the first gradient, the second gradient, the first expected value, and the second expected value, the first edge magnitude corresponding to the image to be processed is determined;

[0016] Based on the first edge magnitude, the edge similarity between the image to be processed and the reference image is determined.

[0017] Further, the step of determining the edge similarity between the image to be processed and the reference image based on the first edge magnitude includes:

[0018] Obtain the second edge magnitude of the reference image corresponding to the image to be processed;

[0019] Based on the first edge magnitude and the second edge magnitude, the edge similarity between the image to be processed and the reference image is determined.

[0020] Furthermore, the step of obtaining the text region and image region corresponding to the image to be processed includes:

[0021] The image to be processed is divided into blocks to obtain multiple image blocks, and the activity value of each image block is obtained.

[0022] Obtain a first image block in the image block whose activity value is greater than a preset activity value, and a second image block in the image block whose activity value is less than or equal to the preset activity value;

[0023] The text region is determined based on the first image block, and the image region is determined based on the second image block.

[0024] Further, the step of obtaining the first expected value corresponding to the text region and the second expected value corresponding to the image region includes:

[0025] Based on the activity values ​​of each image block in the first image block, the first activity and value are determined;

[0026] The second activity and value are determined based on the activity values ​​of each image block in the second image block;

[0027] The first expected value is determined based on the first activity and value and the first number of each image block in the first image block, and the second expected value is determined based on the second activity and value and the second number of each image block in the second image block.

[0028] Furthermore, the step of obtaining the activity value of each of the image patches includes:

[0029] For each image block, obtain the pixel value of each pixel in the image block;

[0030] Based on each pixel value, determine the sum of variances along the lower left diagonal and the sum of variances along the lower right diagonal for the image block;

[0031] The activity value corresponding to the image patch is determined based on the sum of variances of the lower left diagonal and the sum of variances of the lower right diagonal.

[0032] Furthermore, to achieve the above objectives, the present invention also provides an image quality evaluation device, the image quality evaluation device comprising:

[0033] The acquisition module is used to acquire the text region and image region corresponding to the image to be processed, and to acquire the first expected value corresponding to the text region and the second expected value corresponding to the image region;

[0034] The first determining module is used to determine a first gradient corresponding to the image region based on a first edge operator, and to determine a second gradient corresponding to the text region based on a second edge operator, wherein the kernel of the first edge operator is greater than the kernel of the second edge operator;

[0035] The second determining module is used to determine the edge similarity between the image to be processed and the reference image based on the first gradient, the second gradient, the first expected value, and the second expected value.

[0036] The evaluation module is used to input the edge similarity into a nonlinear mapping function to obtain the image quality evaluation result corresponding to the image to be processed.

[0037] In addition, to achieve the above objectives, the present invention also provides an image quality evaluation device, the image quality evaluation device comprising: a memory, a processor, and an image quality evaluation program stored in the memory and executable on the processor, wherein the image quality evaluation program, when executed by the processor, implements the steps of the aforementioned image quality evaluation method.

[0038] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing an image quality evaluation program, which, when executed by a processor, implements the steps of the aforementioned image quality evaluation method.

[0039] This invention obtains the text region and image region corresponding to the image to be processed, and obtains a first expected value corresponding to the text region and a second expected value corresponding to the image region. Then, it determines a first gradient corresponding to the image region based on a first edge operator, and a second gradient corresponding to the text region based on a second edge operator, wherein the kernel of the first edge operator is larger than the kernel of the second edge operator. Next, it determines the edge similarity between the image to be processed and a reference image based on the first gradient, the second gradient, the first expected value, and the second expected value. Finally, it inputs the edge similarity into a nonlinear mapping function to obtain the image quality evaluation result corresponding to the image to be processed. By evaluating image quality based on the characteristics of different regions between the image region and the text region, it fully accommodates different field-of-view sizes of the HVS and the significant features of image edges, improving the accuracy and efficiency of image quality evaluation. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of an embodiment of an image quality evaluation device in a hardware operating environment according to an example of the present invention.

[0041] Figure 2This is a flowchart illustrating the first embodiment of the image quality evaluation method of the present invention;

[0042] Figure 3 This is a schematic diagram of the functional modules of an embodiment of the image quality evaluation device of the present invention.

[0043] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0044] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0045] Figure 1 This is a schematic diagram of the structure of an image quality evaluation device in the hardware operating environment involved in the embodiments of the present invention.

[0046] The image quality evaluation device in this embodiment of the invention can be a PC or a terminal device such as a smartphone. Figure 1 As shown, the image quality evaluation device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0047] Optionally, the image quality assessment device may also include a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, and so on. Of course, the image quality assessment device may also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, which will not be elaborated upon here.

[0048] Those skilled in the art will understand that Figure 1 The terminal structure shown does not constitute a limitation on the image quality evaluation device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0049] like Figure 1As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an image quality evaluation program.

[0050] exist Figure 1 In the image quality evaluation device shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the image quality evaluation program stored in the memory 1005.

[0051] In this embodiment, the image quality evaluation device includes: a memory 1005, a processor 1001, and an image quality evaluation program stored in the memory 1005 and executable on the processor 1001. When the processor 1001 calls the image quality evaluation program stored in the memory 1005, it executes the steps of the image quality evaluation methods in the following embodiments.

[0052] This invention also provides an image quality evaluation method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the image quality evaluation method of the present invention.

[0053] Step S101: Obtain the text region and image region corresponding to the image to be processed, and obtain the first expected value corresponding to the text region and the second expected value corresponding to the image region;

[0054] In this embodiment, the image to be processed is a screen image, such as the screen image currently displayed on a mobile terminal. When the image to be processed is obtained, the text region and image region corresponding to the image to be processed are obtained. The text region is the image region of text and / or charts in the image to be processed, and the image region is the image region of natural graphics in the image to be processed. Specifically, existing image recognition algorithms can be used to determine the text region and image region in the image to be processed, or the image to be processed can be divided into blocks, and the image blocks can be determined as text image blocks or natural image blocks based on the activity value of each image block, and then the text region and image region in the image to be processed can be determined.

[0055] When obtaining the text region and the image region, obtain the first expected value corresponding to the text region and the second expected value corresponding to the image region. Specifically, the first expected value and the second expected value can be determined by a preset model. For example, input the text region into the preset model to obtain the first expected value corresponding to the text region, and input the image region into the preset model to obtain the second expected value corresponding to the image region. Alternatively, the first expected value can be determined by the activity value of the image block included in the text region, and the second expected value can be determined by the activity value of the image block included in the image region.

[0056] Step S102: Determine the first gradient corresponding to the image region based on the first edge operator, and determine the second gradient corresponding to the text region based on the second edge operator, wherein the kernel of the first edge operator is greater than the kernel of the second edge operator;

[0057] In this embodiment, since the Human Visual System (HVS) uses a larger field of view when interpreting natural scene image regions and a smaller field of view when interpreting text content, edge operators with different kernel sizes should be used to evaluate image and text regions. Therefore, upon obtaining the text and image regions, a first edge operator and a second edge operator are obtained. Both the first and second edge operators can be Prewitt operators. The kernel of the first edge operator is larger than that of the second edge operator; for example, the kernel of the first edge operator is a 5×5 kernel, and the kernel of the second edge operator is a 3×3 kernel. Thus, for active text regions in the image to be processed, a smaller Prewitt operator kernel is used to calculate the gradient; for stable image regions in the image to be processed, a larger Prewitt operator kernel is used to calculate the gradient.

[0058] In this embodiment, when the first edge operator and the second edge operator are obtained, the first gradient corresponding to the image region is determined based on the first edge operator, and the second gradient corresponding to the text region is determined based on the second edge operator. That is, the first gradient is calculated using the first edge operator and the image region, and the second gradient is calculated using the second edge operator and the text region.

[0059] Step S103: Determine the edge similarity between the image to be processed and the reference image based on the first gradient, the second gradient, the first expected value, and the second expected value;

[0060] In this embodiment, when the first gradient and the second gradient are obtained, the edge similarity between the second edge amplitude of the image to be processed and the reference image is determined based on the first gradient, the second gradient, the first expected value, and the second expected value. Specifically, the first edge amplitude of the image to be processed can be determined first based on the first gradient, the second gradient, the first expected value, and the second expected value. Then, the edge similarity is determined based on the first edge amplitude and the second edge amplitude of the reference image. The reference image is the undistorted image corresponding to the image to be processed, that is, the original image corresponding to the image to be processed. The image to be processed is the image displayed on the screen after the reference image has undergone a series of possible factors. The second edge amplitude of the reference image is calculated in advance.

[0061] Step S104: Input the edge similarity into a nonlinear mapping function to obtain the image quality evaluation result corresponding to the image to be processed.

[0062] In this embodiment, a nonlinear mapping function is pre-set before image quality evaluation. The formula for this nonlinear mapping function is:

[0063]

[0064] Where f(x) is a nonlinear mapping function, τ i (i = 1, 2, ..., 5) are the parameters of the nonlinear mapping function, τ i It can be set appropriately, where x is the edge similarity.

[0065] This embodiment uses a five-parameter fitted nonlinear mapping function to calculate the image quality evaluation result corresponding to the image to be processed, which can accurately obtain the image quality evaluation result.

[0066] The image quality assessment method proposed in this embodiment obtains the text region and image region corresponding to the image to be processed, and obtains the first expected value corresponding to the text region and the second expected value corresponding to the image region. Then, it determines the first gradient corresponding to the image region based on the first edge operator, and determines the second gradient corresponding to the text region based on the second edge operator, wherein the kernel of the first edge operator is larger than the kernel of the second edge operator. Then, it determines the edge similarity between the image to be processed and the reference image based on the first gradient, the second gradient, the first expected value, and the second expected value. Finally, it inputs the edge similarity into a nonlinear mapping function to obtain the image quality assessment result corresponding to the image to be processed. By evaluating the image quality based on the characteristics of different regions between the image region and the text region of the image, it fully accommodates different field-of-view sizes of HVS and the salient features of the image edges, thus improving the accuracy and efficiency of image quality assessment.

[0067] Based on the first embodiment, a second embodiment of the image quality evaluation method of the present invention is proposed. In this embodiment, step S102 includes:

[0068] Step S201: Based on the first horizontal operator of the first edge operator, determine the first horizontal edge gradient corresponding to the image region, and based on the first vertical operator of the first edge operator, determine the first vertical edge gradient corresponding to the image region.

[0069] Step S202: Based on the second horizontal operator of the second edge operator, determine the second horizontal edge gradient corresponding to the text region, and based on the second vertical operator of the second edge operator, determine the second vertical edge gradient corresponding to the text region.

[0070] Step S203: Determine the first gradient based on the first horizontal edge gradient and the first vertical edge gradient, and determine the second gradient based on the second horizontal edge gradient and the second vertical edge gradient.

[0071] In this embodiment, when the first edge operator and the second edge operator are obtained, the first horizontal operator and the first vertical operator of the first edge operator, and the second horizontal operator and the second vertical operator of the second edge operator are obtained.

[0072] Next, the first horizontal edge gradient corresponding to the image region is determined based on the first horizontal operator, the first vertical edge gradient corresponding to the image region is determined based on the first vertical operator, the second horizontal edge gradient corresponding to the text region is determined based on the second horizontal operator, and the second vertical edge gradient corresponding to the text region is determined based on the second vertical operator. Specifically, the first horizontal edge gradient, the first straight edge gradient, the second horizontal edge gradient, and the second straight edge gradient are calculated with reference to the following formulas:

[0073] M HAGx =p x '*I HAG ;

[0074] M HAGy =p y '*I HAG ;

[0075] M LAGx =p x *I LAG ;

[0076] M LAGy =p y *I LAG ;

[0077] Among them, M HAGx M is the first horizontal edge gradient. HAGyM is the gradient of the first straight edge. LAGx M is the second horizontal edge gradient. LAGy For the second straight edge gradient, I HAG For the image region, I LAG For text regions, p x ' is the first-level operator, p y ' is the first vertical operator, p x For the second-level operator, p y This is the second vertical operator, and the asterisk * represents convolution.

[0078] Then, a first gradient is determined based on a first horizontal edge gradient and a first vertical edge gradient, and a second gradient is determined based on a second horizontal edge gradient and a second vertical edge gradient. The first and second gradients are calculated using the following formulas:

[0079]

[0080]

[0081] Among them, EM HAG For the first gradient, EM LAG This is the second gradient.

[0082] The image quality assessment method proposed in this embodiment determines the first horizontal edge gradient corresponding to the image region based on the first horizontal operator of the first edge operator, and determines the first vertical edge gradient corresponding to the image region based on the first vertical operator of the first edge operator; then, it determines the second horizontal edge gradient corresponding to the text region based on the second horizontal operator of the second edge operator, and determines the second vertical edge gradient corresponding to the text region based on the second vertical operator of the second edge operator; then, it determines the first gradient based on the first horizontal edge gradient and the first vertical edge gradient, and determines the second gradient based on the second horizontal edge gradient and the second vertical edge gradient. This method can accurately obtain the first gradient and the second gradient, and then evaluate the image quality based on the characteristics of different regions between the image region and the text region, further improving the accuracy and efficiency of image quality assessment.

[0083] Based on the first embodiment, a third embodiment of the image quality evaluation method of the present invention is proposed. In this embodiment, step S103 includes:

[0084] Step S301: Based on the first gradient, the second gradient, the first expected value, and the second expected value, determine the first edge magnitude corresponding to the image to be processed;

[0085] Step S302: Based on the first edge magnitude, determine the edge similarity between the image to be processed and the reference image.

[0086] In this embodiment, upon obtaining the first gradient and the second gradient, a first edge magnitude corresponding to the image to be processed is determined based on the first gradient, the second gradient, the first expected value, and the second expected value. Specifically, the formula for the first edge magnitude is:

[0087]

[0088] Where α is a constant, in this embodiment α = 0.2, EM HAG For the first gradient, EM LAG For the second gradient, The first expected value, The second expected value is EMI, and the first edge amplitude is EMI.

[0089] Then, based on the first edge magnitude, the edge similarity between the image to be processed and the reference image is determined. Specifically, the edge similarity is calculated based on the first edge magnitude and the second edge magnitude of the reference image.

[0090] The image quality assessment method proposed in this embodiment determines the first edge magnitude corresponding to the image to be processed based on the first gradient, the second gradient, the first expected value, and the second expected value; then, based on the first edge magnitude, it determines the edge similarity between the image to be processed and the reference image. This method can accurately obtain the edge similarity and is compatible with different field-view sizes of HVS and the salient features of the image edges when calculating the edge similarity, further improving the accuracy and efficiency of image quality assessment.

[0091] Based on the third embodiment, a fourth embodiment of the image quality evaluation method of the present invention is proposed. In this embodiment, step S302 includes:

[0092] Step S401: Obtain the second edge magnitude of the reference image corresponding to the image to be processed;

[0093] Step S402: Determine the edge similarity between the image to be processed and the reference image based on the first edge magnitude and the second edge magnitude.

[0094] In this embodiment, when the first edge amplitude is obtained, the second edge amplitude of the reference image corresponding to the image to be processed is obtained. The reference image is the undistorted image corresponding to the image to be processed, that is, the original image corresponding to the image to be processed. The image to be processed is the image displayed on the screen after the reference image has undergone a series of possible factors. The second edge amplitude of the reference image is calculated in advance. For example, the image to be processed is an image sent by the server. The server can send its second edge amplitude when sending the image.

[0095] Next, based on the first edge magnitude and the second edge magnitude, the edge similarity between the image to be processed and the reference image is determined. The formula for this edge similarity is:

[0096]

[0097] Where SEM represents edge similarity, and EMI represents... d For the first edge amplitude, EMI r C1 is the second edge magnitude, and C1 is a constant. Instability can be eliminated by CI.

[0098] The image quality assessment method proposed in this embodiment obtains the second edge amplitude of the reference image corresponding to the image to be processed; then, based on the first edge amplitude and the second edge amplitude, it determines the edge similarity between the image to be processed and the reference image. It can accurately obtain the edge similarity according to the first edge amplitude and the second edge amplitude, and is compatible with different field-view sizes of HVS and the salient features of the image edges when calculating the edge similarity, thereby further improving the accuracy and efficiency of image quality assessment.

[0099] Based on the above embodiments, a fifth embodiment of the image quality evaluation method of the present invention is proposed. In this embodiment, step S101 includes:

[0100] Step S501: The image to be processed is divided into blocks to obtain multiple image blocks, and the activity value of each image block is obtained.

[0101] Step S502: Obtain a first image block in the image block whose activity value is greater than a preset activity value, and a second image block in the image block whose activity value is less than or equal to the preset activity value;

[0102] Step S503: Determine the text region based on the first image block, and determine the image region based on the second image block.

[0103] In this embodiment, when the image to be processed is obtained, the image to be processed is divided into blocks to obtain multiple image blocks. For example, the image to be processed is divided into blocks of 8*8 pixels, so that each image block includes 8*8 pixels and the number of rows and columns of pixels in the image block is 8. Of course, other methods can also be used to divide the image to be processed into blocks.

[0104] When obtaining each image block of the image to be processed, the activity value of each image block is obtained. Specifically, step S501 includes:

[0105] Step S5011: For each image block, obtain the pixel value of each pixel in the image block;

[0106] Step S5012: Based on each pixel value, determine the sum of variances along the lower left diagonal and the sum of variances along the lower right diagonal for the image block;

[0107] Step S5013: Determine the activity value corresponding to the image block based on the sum of variances of the lower left diagonal and the sum of variances of the lower right diagonal.

[0108] In this embodiment, for each image block, the Block Activity Measurement (BAM) method is used to obtain the activity value of the image block. Specifically, the pixel value of each pixel in the image block, as well as the number of rows m and columns n of the pixels in the image block, are obtained. Then, based on each pixel value, the sum of variances of the lower left diagonal and the sum of variances of the lower right diagonal for the image block are determined. The formulas for the sum of variances of the lower left diagonal and the sum of variances of the lower right diagonal are as follows:

[0109]

[0110]

[0111] Where v1 is the sum of variances along the lower left diagonal, v2 is the sum of variances along the lower right diagonal, and b i,j Let m be the pixel value of the pixel in the i-th row and j-th column of the image block, m be the number of rows of pixels in the image block, and n be the number of columns of pixels in the image block.

[0112] When the variance sum of the lower left diagonal and the variance sum of the lower right diagonal are obtained, the activity value corresponding to the image patch is determined based on the variance sum of the lower left diagonal and the variance sum of the lower right diagonal. The formula for the activity value is:

[0113]

[0114] Wherein, BAM is the activity value, v1 is the sum of variances on the lower left diagonal, v2 is the sum of variances on the lower right diagonal, m is the number of rows of pixels in the image block, n is the number of columns of pixels in the image block, and a is the weight of the percentage of variance. In this embodiment, a can be set to 0.5.

[0115] When the activity value of each image block is obtained, the activity value of each image block is compared with the preset activity value to determine the first image block whose activity value is greater than the preset activity value, and the second image block whose activity value is less than or equal to the preset activity value. The preset activity value can be set reasonably. Extensive experiments have shown that text images have a larger BAM, with BAM values ​​concentrated between 600 and 1200, while natural scene images have a smaller BAM. Therefore, the preset activity value can be set to 600.

[0116] Finally, the text region is determined based on the first image block, and the image region is determined based on the second image block. That is, the region formed by each first image block is the text region, and the region formed by each second image block is the image region.

[0117] The image quality assessment method proposed in this embodiment divides the image to be processed into multiple image blocks and obtains the activity value of each image block. Then, it obtains a first image block whose activity value is greater than a preset activity value and a second image block whose activity value is less than or equal to the preset activity value. Then, it determines the text region based on the first image block and the image region based on the second image block. This method can accurately obtain the text region and image region of the image to be processed based on the activity value. By accommodating different field-view sizes of HVS and the salient features of image edges when calculating the edge similarity of text regions and image regions, it further improves the accuracy and efficiency of image quality assessment.

[0118] Based on the fifth embodiment, a sixth embodiment of the image quality evaluation method of the present invention is proposed. In this embodiment, step S101 includes:

[0119] Step S601: Determine the first activity sum value based on the activity values ​​of each image block in the first image block;

[0120] Step S602: Determine the second activity sum value based on the activity values ​​of each image block in the second image block;

[0121] Step S603: Based on the first activity and value and the first number of each image block in the first image block, determine the first expected value, and based on the second activity and value and the second number of each image block in the second image block, determine the second expected value.

[0122] In this embodiment, when the first image block and the second image block are obtained, a second activity sum value is determined based on the activity value of each image block in the first image block and the activity value of each image block in the second image block, and a first activity sum value is determined. The first activity sum value is the sum of the activity values ​​of each image block in the first image block, and the second activity sum value is the sum of the activity values ​​of each image block in the second image block.

[0123] Next, based on the first activity sum and the first number of each image block in the first image block, a first expected value is determined, and based on the second activity sum and the second number of each image block in the second image block, a second expected value is determined. Specifically, the first number of each image block in the first image block and the second number of each image block in the second image block are obtained. A first intermediate expected value is calculated based on the first activity sum and the first number, and a second intermediate expected value is calculated based on the second activity sum and the second number. For example, the first intermediate expected value = first activity sum / first number, and the second intermediate expected value = second activity sum / second number. Then, the first expected value is determined based on the first intermediate expected value and a constant α, and the second expected value is determined based on the second intermediate expected value and the constant α, E(A HAG ) is the first intermediate expected value. As the first expected value, E(A) LAG ) is the second intermediate expected value. This is the second expected value.

[0124] The image quality assessment method proposed in this embodiment determines a first activity sum value based on the activity values ​​of each image block in the first image block, and then determines a second activity sum value based on the activity values ​​of each image block in the second image block. Subsequently, a first expected value is determined based on the first activity sum value and a first number of each image block in the first image block, and a second expected value is determined based on the second activity sum value and a second number of each image block in the second image block. This allows for the accurate acquisition of the first and second expected values ​​through the activity values ​​of each image block, further improving the accuracy and efficiency of image quality assessment.

[0125] The present invention also provides an image quality evaluation device, with reference to Figure 3 The image quality evaluation device includes:

[0126] The acquisition module 10 is used to acquire the text region and image region corresponding to the image to be processed, and to acquire the first expected value corresponding to the text region and the second expected value corresponding to the image region;

[0127] The first determining module 20 is used to determine the first gradient corresponding to the image region based on the first edge operator, and to determine the second gradient corresponding to the text region based on the second edge operator, wherein the kernel of the first edge operator is greater than the kernel of the second edge operator;

[0128] The second determining module 30 is used to determine the edge similarity between the image to be processed and the reference image based on the first gradient, the second gradient, the first expected value and the second expected value.

[0129] Evaluation module 40 is used to input the edge similarity into a nonlinear mapping function to obtain the image quality evaluation result corresponding to the image to be processed.

[0130] Furthermore, the first determining module 20 is also used for:

[0131] Based on the first horizontal operator of the first edge operator, the first horizontal edge gradient corresponding to the image region is determined, and based on the first vertical operator of the first edge operator, the first vertical edge gradient corresponding to the image region is determined.

[0132] Based on the second horizontal operator of the second edge operator, the second horizontal edge gradient corresponding to the text region is determined, and based on the second vertical operator of the second edge operator, the second vertical edge gradient corresponding to the text region is determined.

[0133] The first gradient is determined based on the first horizontal edge gradient and the first vertical edge gradient, and the second gradient is determined based on the second horizontal edge gradient and the second vertical edge gradient.

[0134] Furthermore, the second determining module 30 is also used for:

[0135] Based on the first gradient, the second gradient, the first expected value, and the second expected value, the first edge magnitude corresponding to the image to be processed is determined;

[0136] Based on the first edge magnitude, the edge similarity between the image to be processed and the reference image is determined.

[0137] Furthermore, the second determining module 30 is also used for:

[0138] Obtain the second edge magnitude of the reference image corresponding to the image to be processed;

[0139] Based on the first edge magnitude and the second edge magnitude, the edge similarity between the image to be processed and the reference image is determined.

[0140] Furthermore, module 10 is also used for:

[0141] The image to be processed is divided into blocks to obtain multiple image blocks, and the activity value of each image block is obtained.

[0142] Obtain a first image block in the image block whose activity value is greater than a preset activity value, and a second image block in the image block whose activity value is less than or equal to the preset activity value;

[0143] The text region is determined based on the first image block, and the image region is determined based on the second image block.

[0144] Furthermore, module 10 is also used for:

[0145] Based on the activity values ​​of each image block in the first image block, the first activity and value are determined;

[0146] The second activity and value are determined based on the activity values ​​of each image block in the second image block;

[0147] The first expected value is determined based on the first activity and value and the first number of each image block in the first image block, and the second expected value is determined based on the second activity and value and the second number of each image block in the second image block.

[0148] Furthermore, module 10 is also used for:

[0149] For each image block, obtain the pixel value of each pixel in the image block;

[0150] Based on each pixel value, determine the sum of variances along the lower left diagonal and the sum of variances along the lower right diagonal for the image block;

[0151] The activity value corresponding to the image patch is determined based on the sum of variances of the lower left diagonal and the sum of variances of the lower right diagonal.

[0152] The methods executed by the above-mentioned program units can be referred to in the various embodiments of the image quality evaluation method of the present invention, and will not be repeated here.

[0153] The present invention also provides a computer-readable storage medium.

[0154] The present invention provides a computer-readable storage medium storing an image quality evaluation program, which, when executed by a processor, implements the steps of the image quality evaluation method described above.

[0155] The method implemented when the image quality evaluation program running on the processor is executed can be referred to in various embodiments of the image quality evaluation method of the present invention, and will not be repeated here.

[0156] Furthermore, this invention also proposes a computer program product that includes an image quality evaluation program, which, when executed by a processor, implements the steps of the image quality evaluation method described above.

[0157] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0158] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0159] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0160] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An image quality assessment method, characterized in that, The image quality evaluation method includes the following steps: The text region and image region corresponding to the image to be processed are obtained, and a first expected value corresponding to the text region and a second expected value corresponding to the image region are obtained. The first expected value is determined by the activity value of the image blocks included in the text region, and the second expected value is determined by the activity value of the image blocks included in the image region. For each image block, the pixel value of each pixel in the image block is obtained. Based on each pixel value, the sum of variances of the lower left diagonal and the sum of variances of the lower right diagonal of the image block are determined. Based on the sum of variances of the lower left diagonal and the sum of variances of the lower right diagonal, the activity value corresponding to the image block is determined. A first gradient corresponding to the image region is determined based on a first edge operator, and a second gradient corresponding to the text region is determined based on a second edge operator, wherein the kernel of the first edge operator is larger than the kernel of the second edge operator. A first horizontal edge gradient corresponding to the image region is determined based on a first horizontal operator of the first edge operator, and a first vertical edge gradient corresponding to the image region is determined based on a first vertical operator of the first edge operator. A second horizontal edge gradient corresponding to the text region is determined based on a second horizontal operator of the second edge operator, and a second vertical edge gradient corresponding to the text region is determined based on a second vertical operator of the second edge operator. The first gradient is determined based on the first horizontal edge gradient and the first vertical edge gradient, and the second gradient is determined based on the second horizontal edge gradient and the second vertical edge gradient. Based on the first gradient, the second gradient, the first expected value, and the second expected value, the edge similarity between the image to be processed and the reference image is determined. The edge similarity is input into a nonlinear mapping function to obtain the image quality evaluation result corresponding to the image to be processed.

2. The image quality evaluation method as described in claim 1, characterized in that, The step of determining the edge similarity between the image to be processed and the reference image based on the first gradient, the second gradient, the first expected value, and the second expected value includes: Based on the first gradient, the second gradient, the first expected value, and the second expected value, the first edge magnitude corresponding to the image to be processed is determined; Based on the first edge magnitude, the edge similarity between the image to be processed and the reference image is determined.

3. The image quality evaluation method as described in claim 2, characterized in that, The step of determining the edge similarity between the image to be processed and the reference image based on the first edge magnitude includes: Obtain the second edge magnitude of the reference image corresponding to the image to be processed; Based on the first edge magnitude and the second edge magnitude, the edge similarity between the image to be processed and the reference image is determined.

4. The image quality evaluation method according to any one of claims 1 to 3, characterized in that, The steps for obtaining the text region and image region corresponding to the image to be processed include: The image to be processed is divided into blocks to obtain multiple image blocks, and the activity value of each image block is obtained. Obtain a first image block in the image block whose activity value is greater than a preset activity value, and a second image block in the image block whose activity value is less than or equal to the preset activity value; The text region is determined based on the first image block, and the image region is determined based on the second image block.

5. The image quality evaluation method as described in claim 4, characterized in that, The steps of obtaining the first expected value corresponding to the text region and the second expected value corresponding to the image region include: Based on the activity values ​​of each image block in the first image block, the first activity and value are determined; The second activity and value are determined based on the activity values ​​of each image block in the second image block; The first expected value is determined based on the first activity and value and the first number of each image block in the first image block, and the second expected value is determined based on the second activity and value and the second number of each image block in the second image block.

6. An image quality evaluation device, characterized in that, The image quality evaluation device includes: An acquisition module is used to acquire a text region and an image region corresponding to the image to be processed, and to acquire a first expected value corresponding to the text region and a second expected value corresponding to the image region. The first expected value is determined by the activity values ​​of the image blocks included in the text region, and the second expected value is determined by the activity values ​​of the image blocks included in the image region. For each image block, the pixel value of each pixel in the image block is acquired. Based on each pixel value, the variance sum of the lower left diagonal and the variance sum of the lower right diagonal of the image block are determined. Based on the variance sum of the lower left diagonal and the variance sum of the lower right diagonal, the activity value corresponding to the image block is determined. A first determining module is configured to determine a first gradient corresponding to the image region based on a first edge operator, and to determine a second gradient corresponding to the text region based on a second edge operator, wherein the kernel of the first edge operator is larger than the kernel of the second edge operator; to determine a first horizontal edge gradient corresponding to the image region based on a first horizontal operator of the first edge operator, and to determine a first vertical edge gradient corresponding to the image region based on a first vertical operator of the first edge operator; to determine a second horizontal edge gradient corresponding to the text region based on a second horizontal operator of the second edge operator, and to determine a second vertical edge gradient corresponding to the text region based on a second vertical operator of the second edge operator; and to determine the first gradient based on the first horizontal edge gradient and the first vertical edge gradient, and to determine the second gradient based on the second horizontal edge gradient and the second vertical edge gradient. The second determining module is used to determine the edge similarity between the image to be processed and the reference image based on the first gradient, the second gradient, the first expected value and the second expected value; The evaluation module is used to input the edge similarity into a nonlinear mapping function to obtain the image quality evaluation result corresponding to the image to be processed.

7. An image quality evaluation device, characterized in that, The image quality evaluation device includes: a memory, a processor, and an image quality evaluation program stored in the memory and executable on the processor, wherein the image quality evaluation program, when executed by the processor, implements the steps of the image quality evaluation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image quality evaluation program, which, when executed by a processor, implements the steps of the image quality evaluation method as described in any one of claims 1 to 5.

Images