Method and device for determining JND threshold value of image, electronic equipment and storage medium

Abstract

The invention relates to the field of image processing, and provides a method and device for determining a JND threshold value of an image, electronic equipment and a storage medium. The method comprises the steps that each pixel in an image is traversed, when any pixel is accessed, according to the pixel value of the pixel and the pixel values of other pixels in the neighborhood of the pixel, the brightness self-adaptive threshold value and the texture covering threshold value of the pixel are determined, the brightness self-adaptive threshold value represents the minimum brightness change value perceivable by human eyes, and the texture covering threshold value represents the minimum brightness change value perceivable by human eyes; the texture covering threshold value represents the maximum complexity of the texture which allows the noise to be perceived by human eyes under the condition that the noise and the texture exist at the same time; and determining a brightness JND threshold value of the pixel according to the brightness adaptive threshold value of the pixel, a texture covering threshold value and a preset superposition effect adjustment factor. According to the method, the JND threshold value of the image can be determined only by traversing the image once, the determination efficiency of the JND threshold value of the image is improved, and hardware implementation is facilitated.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a method, device, electronic device and storage medium for determining a JND threshold of an image. Background Art

[0002] Just Noticeable Distortion (JND) is used to indicate the minimum image distortion that the human eye can perceive, reflecting the sensitivity of the human eye to image changes. In the field of image processing, JND can be used to measure the sensitivity of the human eye to changes in different areas of an image. Due to various shielding effects of the human eye, the human eye can only perceive pixel value changes that exceed a certain threshold. This threshold is the just noticeable distortion threshold of the human eye, that is, the JND threshold.

[0003] JND is widely used in various application scenarios of image processing, such as objective image quality evaluation, image super-resolution, image segmentation, etc. The existing JND threshold determination method needs to traverse the image multiple times during the execution process, which is inefficient and difficult to implement in hardware. Summary of the invention

[0004] In view of this, the present disclosure proposes a method, device, electronic device and storage medium for determining a JND threshold of an image. The method for determining a JND threshold of an image in the embodiments of the present disclosure can determine the JND threshold of an image by traversing the image only once, thereby improving the efficiency of determining the JND threshold of the image and facilitating the hardware implementation of the method for determining the JND threshold of an image.

[0005] According to one aspect of the present disclosure, a method for determining a JND threshold of an image is provided, wherein the JND threshold includes a brightness JND threshold, and the method includes: traversing each pixel in the image, wherein when any pixel is accessed, determining a brightness adaptive threshold and a texture masking threshold of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the brightness adaptive threshold represents a minimum brightness change perceptible to the human eye, and the texture masking threshold represents a maximum complexity of the texture that allows noise to be perceived by the human eye in the presence of both noise and texture; wherein, when determining the texture masking threshold, an edge detection operator corresponding to a single traversal of the image is used; and determining the brightness JND threshold of the pixel according to the brightness adaptive threshold, the texture masking threshold, and a preset superposition effect adjustment factor of the pixel.

[0006] In a possible implementation, determining the brightness adaptive threshold and texture masking threshold of the pixel based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: calculating the product of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel and a low-pass filter operator as the background brightness value of the pixel; and finding the brightness adaptive threshold corresponding to the background brightness value from a brightness adaptive threshold lookup table as the brightness adaptive threshold of the pixel.

[0007] In a possible implementation, the method of determining a brightness adaptive threshold and a texture masking threshold of a pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: determining a gradient feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the gradient feature represents the maximum value of the gradient of the pixel in multiple preset directions; determining an edge feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the edge feature indicates whether the pixel is an edge of an image; wherein an edge detection operator corresponding to a single traversal of the image is used when determining the edge feature; and determining the texture masking threshold of the pixel according to the product of the gradient feature of the pixel, the edge feature and a preset control factor.

[0008] In a possible implementation, determining the gradient feature of the pixel based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: for each of the multiple directions, calculating the product of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel and the gradient convolution operator in that direction, averaging the product according to the size of the pixel neighborhood and the gradient convolution operator, and taking the absolute value to obtain the gradient of the pixel in that direction; and taking the maximum value of the gradients of the pixel in multiple directions as the gradient feature of the pixel.

[0009] In a possible implementation, the edge detection operator corresponding to the single-pass image includes a Sober-Federman operator, and determining the edge feature of the pixel based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: calculating the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the Sober-Federman operator in the horizontal direction to obtain a first value; calculating the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the Sober-Federman operator in the vertical direction to obtain a second value; calculating the average value of the first value and the second value, and the difference between a third value and the average value, the third value being the maximum pixel value; and taking the ratio of the difference to the third value as the edge feature of the pixel.

[0010] In a possible implementation, the brightness JND threshold of the pixel is determined based on the brightness adaptation threshold, texture masking threshold and a preset overlay effect adjustment factor of the pixel, including: calculating the smaller of the brightness adaptation threshold and the texture masking threshold of the pixel, and multiplying the result by the overlay effect adjustment factor; summing the brightness adaptation threshold and the texture masking threshold of the pixel, and then subtracting the sum from the product to obtain a fourth value, and determining the fourth value as the brightness JND threshold of the pixel.

[0011] In a possible implementation, the JND threshold also includes a chroma JND threshold, and the method also includes: when accessing any pixel, determining at least one pixel corresponding to the pixel according to the chroma sampling method of the pixel; calculating the average value of the brightness JND threshold of at least one pixel corresponding to the pixel as the chroma JND threshold of the pixel.

[0012] In a possible implementation manner, at least one of the superposition effect adjustment factor and the control factor is an integer power of 2.

[0013] According to another aspect of the present disclosure, a device for determining a JND threshold of an image is provided, wherein the JND threshold includes a brightness JND threshold, and the device includes: a traversal module, used to traverse each pixel in the image, wherein, when any pixel is accessed, a first determination module, used to determine a brightness adaptive threshold and a texture masking threshold of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the brightness adaptive threshold represents a minimum brightness change perceptible to the human eye, and the texture masking threshold represents a maximum complexity of the texture that allows noise to be perceived by the human eye in the presence of both noise and texture; wherein, when determining the texture masking threshold, an edge detection operator corresponding to a single traversal of the image is used; and a second determination module, used to determine the brightness JND threshold of the pixel according to the brightness adaptive threshold, the texture masking threshold and a preset superposition effect adjustment factor of the pixel.

[0014] In a possible implementation, determining the brightness adaptive threshold and texture masking threshold of the pixel based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: calculating the product of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel and a low-pass filter operator as the background brightness value of the pixel; and finding the brightness adaptive threshold corresponding to the background brightness value from a brightness adaptive threshold lookup table as the brightness adaptive threshold of the pixel.

[0015] In a possible implementation, the method of determining a brightness adaptive threshold and a texture masking threshold of a pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: determining a gradient feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the gradient feature represents the maximum value of the gradient of the pixel in multiple preset directions; determining an edge feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the edge feature indicates whether the pixel is an edge of an image; wherein an edge detection operator corresponding to a single traversal of the image is used when determining the edge feature; and determining the texture masking threshold of the pixel according to the product of the gradient feature of the pixel, the edge feature and a preset control factor.

[0016] In a possible implementation, determining the gradient feature of the pixel based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: for each of the multiple directions, calculating the product of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel and the gradient convolution operator in that direction, averaging the product according to the size of the pixel neighborhood and the gradient convolution operator, and taking the absolute value to obtain the gradient of the pixel in that direction; and taking the maximum value of the gradients of the pixel in multiple directions as the gradient feature of the pixel.

[0017] In a possible implementation, the edge detection operator corresponding to the single-pass image includes a Sober-Federman operator, and determining the edge feature of the pixel based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: calculating the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the Sober-Federman operator in the horizontal direction to obtain a first value; calculating the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the Sober-Federman operator in the vertical direction to obtain a second value; calculating the average value of the first value and the second value, and the difference between a third value and the average value, the third value being the maximum pixel value; and taking the ratio of the difference to the third value as the edge feature of the pixel.

[0018] In a possible implementation, the brightness JND threshold of the pixel is determined based on the brightness adaptation threshold, texture masking threshold and a preset overlay effect adjustment factor of the pixel, including: calculating the smaller of the brightness adaptation threshold and the texture masking threshold of the pixel, and multiplying the result by the overlay effect adjustment factor; summing the brightness adaptation threshold and the texture masking threshold of the pixel, and then subtracting the sum from the product to obtain a fourth value, and determining the fourth value as the brightness JND threshold of the pixel.

[0019] In a possible implementation, the JND threshold also includes a chroma JND threshold, and the device also includes: a third determination module, used to determine at least one pixel corresponding to the pixel when any pixel is accessed, according to the chroma sampling method of the pixel; a fourth determination module, used to calculate the average value of the brightness JND threshold of at least one pixel corresponding to the pixel as the chroma JND threshold of the pixel.

[0020] In a possible implementation manner, at least one of the superposition effect adjustment factor and the control factor is an integer power of 2.

[0021] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0022] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0023] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0024] According to the method for determining the JND threshold of an image in the embodiment of the present disclosure, each pixel in the image is traversed, and when any pixel is accessed, the brightness adaptive threshold and texture masking threshold of the pixel are determined according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, the brightness adaptive threshold represents the minimum brightness change that can be perceived by the human eye, and the texture masking threshold represents the maximum complexity of the texture that allows noise to be perceived by the human eye when noise and texture exist at the same time; the brightness JND threshold of the pixel is determined according to the brightness adaptive threshold, texture masking threshold and a preset superposition effect adjustment factor of the pixel. Since the edge detection operator corresponding to a single traversal of the image is used when determining the texture masking threshold, the brightness JND threshold of the image only needs to be traversed once, thereby reducing the number of data readings and improving the efficiency of determining the JND threshold of the image; compared with the solution of traversing the image multiple times in the prior art, a single image traversal is easier to implement in hardware, so that the method for determining the JND threshold of the image in the embodiment of the present disclosure is easy to implement in hardware.

[0025] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0027] Figure 1 An exemplary application scenario of the method for determining the JND threshold of an image according to an embodiment of the present disclosure is shown.

[0028] Figure 2 A schematic diagram showing the flow of a method for determining a JND threshold of an image according to an embodiment of the present disclosure.

[0029] Figure 3 An example of a neighborhood of a pixel according to an embodiment of the present disclosure is shown.

[0030] Figure 4 A schematic diagram showing the corresponding relationship between the background brightness value and the minimum value of the brightness change perceptible to the human eye according to an embodiment of the present disclosure.

[0031] Figure 5 A schematic diagram of a low-pass filter operator according to an embodiment of the present disclosure is shown.

[0032] Figure 6 A schematic diagram of a gradient convolution operator according to an embodiment of the present disclosure is shown.

[0033] Figure 7 A schematic diagram of a Sobel operator according to an embodiment of the present disclosure is shown.

[0034] Figure 8 A schematic diagram is shown of calculating the chroma JND threshold under different chroma sampling modes according to an embodiment of the present disclosure.

[0035] Fig. 9 A schematic diagram showing the structure of a device for determining a JND threshold of an image according to an embodiment of the present disclosure.

[0036] Fig.10 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0037] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0038] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0039] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0040] The method for determining the JND threshold of an image in the embodiment of the present disclosure is a method for determining the JND threshold in the pixel domain. Figure 1 An exemplary application scenario of the method for determining the JND threshold of an image according to an embodiment of the present disclosure is shown.

[0041] like Figure 1 As shown, in an exemplary application scenario, a video on a first device needs to be transmitted to a second device. In order to reduce the transmission cost, the video needs to be compressed.

[0042] The first device may execute the method for determining the JND threshold of an image according to an embodiment of the present disclosure, determine the JND threshold of each pixel of the current video frame, and calculate the pixel value of the pixel after compression under a preset compression method. The change between the original pixel value of the pixel and the compressed pixel value is calculated. If the change is less than the JND threshold of the pixel, it can be considered that the change of the pixel is imperceptible to the human visual system, and therefore the pixel can be ignored or quantized at a coarser grain in the actual process of compressing the current video frame.

[0043] Those skilled in the art should understand that the JND threshold of a pixel may have more uses, such as image quality evaluation, and the embodiments of the present disclosure do not limit the specific uses of the JND threshold of a pixel.

[0044] Figure 2 A schematic diagram showing the flow of a method for determining a JND threshold of an image according to an embodiment of the present disclosure.

[0045] like Figure 2 As shown, in a possible implementation, the JND threshold includes a brightness JND threshold, and the method includes:

[0046] Step S20, traverse each pixel in the image, wherein when any pixel is accessed,

[0047] Step S21, determining a brightness adaptive threshold and a texture masking threshold of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the brightness adaptive threshold represents the minimum brightness change perceptible to the human eye, and the texture masking threshold represents the maximum complexity of the texture that allows the noise to be perceived by the human eye in the presence of both noise and texture; wherein the edge detection operator corresponding to a single traversal of the image is used when determining the texture masking threshold;

[0048] Step S22, determining the brightness JND threshold of the pixel according to the brightness adaptive threshold of the pixel, the texture masking threshold and the preset superposition effect adjustment factor.

[0049] For example, the key parameter of the pixel includes brightness, and thus, the JND threshold of the pixel may include a brightness JND threshold.

[0050] In a frame of image, each pixel has a corresponding neighborhood. The neighborhood refers to the set of other pixels in a quadrilateral area centered on the pixel and including N×N pixels. N can represent the size of the neighborhood and can be set to an odd number greater than 1 according to the application scenario requirements. The embodiment of the present disclosure does not limit the specific value of N. The neighborhood of a pixel does not include the pixel itself. Figure 3 An example of a neighborhood of a pixel according to an embodiment of the present disclosure is shown.

[0051] like Figure 3 As shown, assuming N=5, the neighborhood of a pixel can include all pixels whose horizontal distance / vertical distance from the pixel is less than or equal to 2 ((5-1) / 2=2). Figure 3 In the example, the image includes 8×8 pixels. Since pixel (1,1) is the pixel in the upper left corner of the image, the neighborhood of pixel (1,1) can include a total of 8 pixels to the right and below pixel (1,1). The distance between pixel (6,6) and each boundary of the image is greater than 2, so the neighborhood of pixel (6,6) can include 24 pixels.

[0052] In step S20, each pixel in the image can be traversed, and when any pixel is accessed, steps S21 and S22 are executed to determine the brightness JND threshold of the pixel. After the brightness JND threshold of the pixel is determined, the next pixel is accessed.

[0053] When executing step S21, the brightness adaptive threshold and texture masking threshold of the pixel can be determined according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel. The brightness adaptive threshold represents the minimum brightness change that can be perceived by the human eye, that is, the brightness change of the pixel can only be perceived by the human eye when the brightness change of the pixel is greater than or equal to the brightness adaptive threshold of the pixel. The texture masking threshold represents the maximum complexity of the texture that allows the noise to be perceived by the human eye in the presence of noise and texture at the same time, that is, the human eye can only perceive the noise of the pixel when the complexity of the texture of the pixel is less than or equal to the texture masking threshold.

[0054] The human visual system is more sensitive to brightness contrast than to absolute brightness values. Therefore, the brightness change between a pixel and other pixels in the neighborhood that is perceptible to the human eye (hereinafter referred to as brightness adaptation) is an important factor affecting the brightness JND threshold. The minimum value of the brightness change that is perceptible to the human eye can be determined based on the pixel values ​​of other pixels in the neighborhood of a certain pixel, as the brightness adaptation threshold of the pixel. Since the pixel values ​​of different pixels may be different, the pixel values ​​of pixels in different pixel neighborhoods may be different, and therefore, the brightness adaptation thresholds of different pixels may be different. The embodiments of the present disclosure do not limit the specific calculation method of the brightness adaptation threshold of a pixel, and an exemplary calculation method of the brightness adaptation threshold of a pixel is given later.

[0055] The reduction in human eye visibility is caused by the increase in texture complexity in the neighborhood. Texture can mask noise, and areas with more complex textures can mask more noise. Therefore, the masking of noise by the texture of a pixel (hereinafter also referred to as texture masking) is also an important factor affecting the brightness JND threshold. The maximum complexity of the texture that allows the noise of the pixel to be perceived by the human eye when both noise and texture exist in the pixel can be determined based on the pixel value of the pixel in the neighborhood of a certain pixel, as the texture masking threshold of the pixel. Since the pixel values ​​of different pixels may be different, the pixel values ​​of pixels in different pixel neighborhoods may be different, and therefore, the texture masking thresholds of different pixels may be different. The present disclosure embodiment does not limit the specific calculation method of the texture masking threshold of a pixel, and an exemplary calculation method of the texture masking threshold of a pixel is given later.

[0056] Among them, the edge detection operator corresponding to the single traversal image is used when determining the texture masking threshold, so that the texture masking threshold can be calculated in the process of single traversal image, and thus the brightness JND threshold can also be calculated in the process of single traversal image. An example of the edge detection operator corresponding to the single traversal image is given later.

[0057] For the human visual system, there is a certain overlap between brightness adaptation and texture masking. The superposition effect adjustment factor can be preset according to the overlap between brightness adaptation and texture masking. In step S22, the brightness JND threshold of the pixel can be determined according to the brightness adaptation threshold, texture masking threshold and the preset superposition effect adjustment factor of the pixel, wherein the superposition effect adjustment factor can be used to correct the error caused by the overlap to the brightness JND threshold. An exemplary implementation of step S22 is given later.

[0058] According to the method for determining the JND threshold of an image in the embodiment of the present disclosure, each pixel in the image is traversed, and when any pixel is accessed, the brightness adaptive threshold and texture masking threshold of the pixel are determined according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, the brightness adaptive threshold represents the minimum brightness change that can be perceived by the human eye, and the texture masking threshold represents the maximum complexity of the texture that allows noise to be perceived by the human eye when noise and texture exist at the same time; the brightness JND threshold of the pixel is determined according to the brightness adaptive threshold, texture masking threshold and a preset superposition effect adjustment factor of the pixel. Since the edge detection operator corresponding to a single traversal of the image is used when determining the texture masking threshold, the brightness JND threshold of the image only needs to be traversed once, thereby reducing the number of data readings and improving the efficiency of determining the JND threshold of the image; compared with the solution of traversing the image multiple times in the prior art, a single image traversal is easier to implement in hardware, so that the method for determining the JND threshold of the image in the embodiment of the present disclosure is easy to implement in hardware.

[0059] In a possible implementation, determining the brightness JND threshold of the pixel according to the brightness adaptation threshold of the pixel, the texture masking threshold, and a preset superposition effect adjustment factor includes:

[0060] Calculate the smaller of the brightness adaptive threshold and the texture masking threshold of the pixel and the product of the overlay effect adjustment factor;

[0061] The brightness adaptive threshold and the texture masking threshold of the pixel are summed, and then subtracted from the product to obtain a fourth value, and the fourth value is determined as the brightness JND threshold of the pixel.

[0062] For example, the exemplary calculation method of the brightness JND threshold of a pixel is shown in formula (1):

[0063] (1)

[0064] Where x represents the horizontal coordinate of the pixel (x, y), y represents the vertical coordinate of the pixel (x, y), T l (x,y) represents the brightness adaptive threshold of pixel (x,y), T t(x,y) represents the texture masking threshold of pixel (x,y), C lt Indicates the stacking effect adjustment factor. min{T l (x,y),T t (x,y)} represents the smaller of the brightness adaptive threshold and texture masking threshold of the pixel (x,y). pixel (x,y) represents the brightness JND threshold of pixel (x,y).

[0065] C lt The value is related to the overlap between brightness adaptation and texture masking. The greater the overlap between brightness adaptation and texture masking, the higher the value of C lt The larger the value, the greater the value. lt The specific value of is not limited. In one example, C lt The value of can be 0.3.

[0066] In a possible implementation, the stacking effect adjustment factor is an integer power of 2.

[0067] For example, if the method for determining the JND threshold of an image according to an embodiment of the present disclosure is implemented by hardware, the overlay effect adjustment factor may be set to an integer power of 2, such as 0.25 (2 -2 ). In this case, min{T l (x,y),T t The multiplication operation of (x, y)} and the stacking effect adjustment factor can be achieved through shifting, which reduces the difficulty of hardware implementation.

[0068] An exemplary method for determining a brightness adaptive threshold is described below.

[0069] In a possible implementation, determining a brightness adaptive threshold and a texture masking threshold of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes:

[0070] Calculate the product of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel and the low-pass filter operator as the background brightness value of the pixel;

[0071] From the brightness adaptive threshold lookup table, find the brightness adaptive threshold corresponding to the background brightness value as the brightness adaptive threshold of the pixel.

[0072] For example, when a pixel is regarded as the foreground, the neighborhood of the pixel can be regarded as the background. According to the Weber-Fechner law, when the background brightness value is too small or too large, the minimum value of the brightness change of the foreground that can be perceived by the human eye will increase. Figure 4A schematic diagram showing the corresponding relationship between the background brightness value and the minimum value of the brightness change perceptible to the human eye according to an embodiment of the present disclosure.

[0073] like Figure 4 As shown, the range of background brightness value is 0-255. When the background brightness value is less than or equal to 127, the relationship curve between the minimum value of brightness change perceptible to the human eye and the background brightness value is close to the curve of the root equation; when the background brightness value is greater than 127, the relationship curve between the minimum value of brightness change perceptible to the human eye and the background brightness value is approximately expressed as a linear function curve.

[0074] Since the brightness adaptive threshold of a pixel represents the minimum brightness change that can be perceived by the human eye, based on this, the brightness adaptive threshold of the pixel can be calculated by formula (2):

[0075]

[0076] Where x represents the horizontal coordinate of the pixel (x, y), and y represents the vertical coordinate of the pixel (x, y). Represents the background brightness value of pixel (x, y). l (x,y) represents the brightness adaptive threshold of pixel (x,y).

[0077] For each background brightness value in the value range of 0-255, the brightness adaptive threshold corresponding to the background brightness value is calculated using formula (2) to obtain a brightness adaptive threshold lookup table. The lookup table can record each background brightness value in the value range of 0-255 and its corresponding brightness adaptive threshold.

[0078] When traversing to a certain pixel, the pixel value of the pixel and the product of the pixel values ​​of other pixels in the neighborhood of the pixel and the low-pass filter operator can be calculated first as the background brightness value of the pixel. Figure 5 A schematic diagram of a low-pass filter operator according to an embodiment of the present disclosure is shown.

[0079] Assuming that the size of the neighborhood is N=5, and the pixel currently being accessed is pixel (x, y), then a 5×5 pixel value matrix can be obtained based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of pixel (x, y), with the pixel value of pixel (x, y) as the center of the matrix. Figure 5As shown, a low-pass filter operator with a size of 5×5 can be used. The product of the pixel value of the pixel (x, y) and the pixel values ​​of other pixels in the neighborhood of the pixel (x, y) and the low-pass filter operator can be the cumulative result of the product of each weight in the low-pass filter operator and the pixel value at the corresponding position in the pixel value matrix. It can be understood that the size of the low-pass filter operator can also be larger or smaller, as long as the size of the neighborhood is expanded or reduced accordingly. The embodiments of the present disclosure do not limit the size of the low-pass filter operator, the size of the neighborhood, and the specific values ​​of the weights in the low-pass filter operator.

[0080] According to the background brightness value of the pixel, the brightness adaptive threshold corresponding to the background brightness value is found from the brightness adaptive threshold lookup table, and is used as the brightness adaptive threshold of the pixel. In this way, the efficiency of determining the JND threshold of the image can be improved and the difficulty of hardware implementation can be reduced.

[0081] Those skilled in the art should understand that, when traversing to a certain pixel, the background brightness value of the pixel can be calculated and directly substituted into formula (2) to calculate the brightness adaptive threshold of the pixel. The embodiment of the present disclosure does not limit whether the brightness adaptive threshold lookup table must be used.

[0082] In the prior art, the calculation of the texture masking threshold requires multiple passes through the image, which increases the difficulty of hardware implementation of the JND threshold calculation. The texture masking threshold of the embodiment of the present disclosure can be calculated based on the Sober-Feldman operator, and the texture masking threshold can be determined in a single pass through the image. The following is an exemplary method for determining the texture masking threshold.

[0083] In a possible implementation, determining a brightness adaptive threshold and a texture masking threshold of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes:

[0084] Determine a gradient feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, where the gradient feature represents the maximum value of the gradient of the pixel in multiple preset directions;

[0085] Determine the edge feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, the edge feature indicating whether the pixel is an edge of the image; wherein the edge detection operator corresponding to a single traversal of the image is used when determining the edge feature;

[0086] The texture masking threshold of the pixel is determined according to the product of the gradient feature, the edge feature and the preset control factor of the pixel.

[0087] For example, texture masking can be determined by the local spatial activity of pixels, such as the gradient of pixels. Edges are directly related to image content, which includes visual information such as object boundaries, surface creases, and reflectivity changes. In order to more accurately determine the brightness JND threshold, it is necessary to distinguish between edge and non-edge areas. Due to the characteristics of the human visual system, noise in edge areas is more easily perceived than noise in textured and flat areas. Therefore, the texture masking threshold can be calculated based on the gradient features and edge features of pixels, as shown in formula (3).

[0088] (3)

[0089] Where x represents the horizontal coordinate of the pixel (x, y), and y represents the vertical coordinate of the pixel (x, y). θ (x, y) represents the gradient feature of the pixel (x, y). It represents the maximum value of the gradient of the pixel in multiple preset directions. The multiple directions may include the positive direction in the horizontal direction, the negative direction in the horizontal direction, the positive direction in the vertical direction, and the negative direction in the vertical direction. The embodiment of the present disclosure does not limit the specific number of the multiple preset directions. θ (x,y) represents the edge feature of pixel (x,y), indicating whether the pixel is the edge of the image. t (x,y) represents the texture masking threshold of pixel (x,y). β represents the control factor.

[0090] The value of β can be set according to the application scenario requirements, and the embodiment of the present disclosure does not limit the specific value of β. In one example, the value of β can be 0.117.

[0091] In one possible implementation, the control factor is an integer power of 2.

[0092] For example, if the method for determining the JND threshold of an image according to an embodiment of the present disclosure is implemented by hardware, the control factor may be set to an integer power of 2, such as 0.125 (2 -3 ). In this case, G θ (x,y),W θ The multiplication operation of (x, y) and the control factor can be realized by shifting, which reduces the difficulty of hardware implementation.

[0093] The gradient feature can be calculated in a single pass through the image. An exemplary calculation method of the gradient feature is described below.

[0094] In a possible implementation, determining the gradient feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes:

[0095] For each of the multiple directions, the product of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel and the gradient convolution operator in the direction is calculated, and the absolute value of the product is averaged according to the size of the pixel neighborhood and the gradient convolution operator to obtain the gradient of the pixel in the direction;

[0096] The maximum value of the gradient of the pixel in multiple directions is taken as the gradient feature of the pixel.

[0097] For example, each preset direction has a corresponding gradient convolution operator. Figure 6 A schematic diagram of a gradient convolution operator according to an embodiment of the present disclosure is shown.

[0098] like Figure 6 As shown, assuming that the size of the neighborhood is N=5, and the pixel currently accessed is pixel (x, y), a 5×5 pixel value matrix can be obtained based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of pixel (x, y), and the pixel value of pixel (x, y) is used as the center of the matrix. Accordingly, a gradient convolution operator of size 5×5 can be used. Gradient convolution operators g0-g3 can correspond to the preset four directions respectively.

[0099] In this case, the gradient gradk (k=1, 2, 3, 4) of the pixel (x, y) in direction k is calculated as shown in formula (4).

[0100]

[0101] Where x represents the horizontal coordinate of the pixel (x, y), y represents the vertical coordinate of the pixel (x, y), i represents the i-th row in the gradient convolution operator, and j represents the j-th column in the gradient convolution operator. k (i,j) represents the weight of the i-th row and j-th column in the gradient convolution operator gk. Represents the pixel (x, y) in the neighborhood with g k The pixel value of the pixel at the corresponding position (i, j). abs is a function for finding the absolute value.

[0102] For direction k, calculate the product of the pixel value of pixel (x, y) and the pixel values ​​of the pixels in the neighborhood of pixel (x, y) and the gradient convolution operator in this direction. This product can be the cumulative result of the product of each weight in the gradient convolution operator and the pixel value at the corresponding position in the pixel value matrix. That is, . Since the size of the pixel value matrix and the gradient convolution operator are both 5×5, the weight of the pixel value at the same position as the pixel (x, y) in the gradient convolution operator is 0, so the pixel value of the pixel (x, y) does not need to be involved in the calculation, and the coefficient 1 / 16 can be used to approximate the average of the product. The absolute value of the average result can be taken to ensure that the gradient is positive. In this way, the gradient gradk of the pixel (x, y) in direction k can be obtained.

[0103] It is understandable that the size of the gradient convolution operator can also be larger or smaller, as long as the constant in formula (4) is adjusted accordingly and the size of the neighborhood is expanded or reduced. The embodiment of the present disclosure does not limit the size of the gradient convolution operator and the specific values ​​of the weights in the gradient convolution operator.

[0104] Those skilled in the art should understand that there are more ways to calculate the gradient of a pixel in a certain direction, and the embodiments of the present disclosure do not limit the specific method for calculating the gradient of a pixel in a certain direction.

[0105] The maximum value of the gradient of pixel (x, y) in multiple directions can be used as the gradient feature G of pixel (x, y) θ (x,y), as shown in formula (5).

[0106] (5)

[0107] Among them, grad0 represents the gradient calculated using the gradient convolution operator g0, grad1 represents the gradient calculated using the gradient convolution operator g1, grad2 represents the gradient calculated using the gradient convolution operator g2, and grad3 represents the gradient calculated using the gradient convolution operator g3.

[0108] An exemplary method for calculating edge features is described below.

[0109] In the prior art, edge features are determined using edge detection operators (such as the Canny operator) that correspond to multiple traversals of the image. The principle of the Canny operator is that the first step is to reduce noise and smooth the original image to reduce the impact of noise on the edge detection results. The second step is to calculate the gradient amplitude and direction of each pixel in the image. The third step is to apply the non-maximum suppression technology to screen edge pixels. The fourth step is to further screen edge pixels using a dual threshold detection method, where the pixel value of the edge pixel is 1 and the pixel value of the non-edge pixel is 0. And reset the pixel value of the edge pixel and the pixel value of the non-edge pixel so that the pixel value of the edge pixel is 0.1 and the pixel value of the non-edge pixel is 1. The fifth step is to perform Gaussian filtering on the image with the reset pixel value to smooth the edges in the image and prevent drastic jumps in pixel values. Since each step requires retraversing the image, the image JND threshold determination method of the prior art is difficult to implement in hardware.

[0110] Based on this, the JND threshold determination method of the image in the embodiment of the present disclosure proposes to use the edge detection operator corresponding to the single traversal image to calculate the edge features. For example, the Sobel operator, the Prewitt operator, the Laplacian operator, etc. The embodiment of the present disclosure does not limit the specific type of the edge detection operator corresponding to the single traversal image, as long as the operator can obtain the edge features in the process of traversing the image in a single time.

[0111] The following takes the Sobel operator as an example to introduce an exemplary method for determining the edge feature of the pixel in an embodiment of the present disclosure.

[0112] In a possible implementation, the edge detection operator corresponding to a single traversal of the image includes a Sober-Feldman operator, and the edge feature of the pixel is determined according to the pixel value of the pixel in the neighborhood of the pixel, including:

[0113] Calculate the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the Sober-Feldman operator in the horizontal direction to obtain a first value;

[0114] Calculate the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the Sober-Feldman operator in the vertical direction to obtain a second value;

[0115] Calculate an average value of the first value and the second value, and a difference between the third value and the average value, where the third value is the maximum value of the pixel value;

[0116] The ratio of the difference value to the third value is used as the edge feature of the pixel.

[0117] For example, there are two types of Sobel operators, namely the horizontal Sobel operator and the vertical Sobel operator. Figure 7 A schematic diagram of a Sobel operator according to an embodiment of the present disclosure is shown.

[0118] like Figure 7 As shown, sobel_x represents the sobel operator in the horizontal direction, and sobel_y represents the sobel operator in the vertical direction.

[0119] The vertical Sobel operator is used to detect the horizontal edge in the image. The horizontal Sobel operator is used to detect the vertical edge in the image. Therefore, the edge feature of a pixel can be calculated by convolving the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the horizontal Sobel-Feldman operator to obtain the first value luma. sobelx ; Calculate the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the vertical Sober-Feldman operator to obtain the second value luma sobely .

[0120] Get the first value luma sobelx and the second value luma sobely Afterwards, the edge features of the pixel (x, y) can be calculated using formula (6).

[0121] (6)

[0122] Here, MAX_PIXEL is a preset third value, and its value may be the maximum value of the pixel value, such as 255.

[0123] It should be understood by those skilled in the art that sobelx +luma sobely ) / 2 can represent the edge information detected by the sobel operator, and calculate MAX_PIXEL and (luma sobelx +luma sobely ) / 2, and the ratio of the difference to the third value is similar to the effect achieved by resetting the pixel values ​​of edge pixels and the pixel values ​​of non-edge pixels in the fourth step when the prior art uses the canny operator to calculate edge information. In this way, it can be ensured that the edge features obtained by the embodiment of the present disclosure using the sobel operator are close to the edge features obtained by the prior art using the canny operator, thereby ensuring the calculation accuracy of the edge features.

[0124] The key parameters of a pixel also include chromaticity, so the JND threshold of a pixel may also include a chromaticity JND threshold. In the prior art, the calculation method of the chromaticity JND threshold is the same as the calculation method of the luminance JND threshold, and it also requires multiple image traversals, which is difficult to implement in hardware. Based on this, the present disclosure proposes a calculation method of the chromaticity JND threshold that is easy to implement in hardware.

[0125] In a possible implementation manner, the JND threshold further includes a chrominance JND threshold, and the method further includes:

[0126] When any pixel is accessed, at least one pixel corresponding to the pixel is determined according to the chroma sampling mode of the pixel;

[0127] An average value of the brightness JND threshold of at least one pixel corresponding to the pixel is calculated as the chrominance JND threshold of the pixel.

[0128] For example, since the chromaticity of a pixel is obtained by sampling the brightness of the pixel, the brightness JND threshold of the pixel can be processed accordingly according to the chromaticity sampling method to obtain the chromaticity JND threshold of the pixel.

[0129] The pixel value of each pixel includes the brightness component Y and the chrominance components U and V. There are many chrominance sampling methods, the common ones are 4:4:4 sampling, 4:2:2 sampling, 4:2:0 sampling, etc. Figure 8 A schematic diagram is shown of calculating the chroma JND threshold under different chroma sampling modes according to an embodiment of the present disclosure.

[0130] like Figure 8 As shown in the figure, for 4:4:4 sampling, every 2×2 pixels is used as a sampling unit, and the brightness component Y, chrominance component U, and chrominance component V of each pixel remain unchanged before and after sampling. The pixel corresponding to each pixel in this chrominance sampling mode is the pixel itself. At this time, the JND threshold of the brightness of the pixel can be directly used as the JND threshold of the chrominance. For example Figure 8 In 4:4:4 sampling, the JND threshold of the brightness of pixel 1 is JND_Y_1, and the JND threshold of the chrominance of pixel 1 is JND_UV_1=JND_Y_1. The JND threshold of the brightness of pixel 2 is JND_Y_2, and the JND threshold of the chrominance of pixel 2 is JND_UV_2=JND_Y_2. The JND threshold of the brightness of pixel 3 is JND_Y_3, and the JND threshold of the chrominance of pixel 3 is JND_UV_3=JND_Y3. The JND threshold of the brightness of pixel 4 is JND_Y_4, and the JND threshold of the chrominance of pixel 4 is JND_UV_4=JND_Y_4.

[0131] For 4:2:2 sampling, every 2×2 pixels is used as a sampling unit. The brightness component Y of each pixel remains unchanged before and after sampling, and the pixels in the same row share the same chrominance component U and chrominance component V. Each pixel corresponds to 2 pixels in this chrominance sampling mode, including the pixel itself and the pixels in the same row as the pixel in the sampling unit to which the pixel belongs. At this time, the average value of the brightness JND threshold of the two pixels corresponding to the pixel in this chrominance sampling mode is calculated as the chrominance JND threshold. For example Figure 8 In 4:2:2 sampling, the JND threshold of the brightness of pixel 1 is JND_Y_1, the JND threshold of the brightness of pixel 2 is JND_Y_2, the JND threshold of the chromaticity of pixel 1 JND_UV_1 and the JND threshold of the chromaticity of pixel 2 JND_UV_2 are equal and equal to (JND_Y_1+ JND_Y_2) / 2. The JND threshold of the brightness of pixel 3 is JND_Y_3, the JND threshold of the brightness of pixel 4 is JND_Y_4, the JND threshold of the chromaticity of pixel 3 JND_UV_3 and the JND threshold of the chromaticity of pixel 4 JND_UV_4 are equal and equal to (JND_Y_3+ JND_Y_4) / 2.

[0132] For 4:2:0 sampling, every 2×2 pixels is used as a sampling unit. The brightness component Y of each pixel remains unchanged before and after sampling, and the pixels in the same sampling unit share the same chrominance component U and chrominance component V. Each pixel corresponds to 4 pixels under this chrominance sampling mode, including all pixels in the sampling unit where the pixel is located. At this time, the average value of the brightness JND threshold of the 4 pixels corresponding to the pixel under this chrominance sampling mode is calculated as the chrominance JND threshold of the pixel. For example Figure 8 In 4:2:0 sampling, the JND threshold of the brightness of pixel 1 is JND_Y_1, the JND threshold of the brightness of pixel 2 is JND_Y_2, the JND threshold of the brightness of pixel 3 is JND_Y_3, and the JND threshold of the brightness of pixel 4 is JND_Y_4. The JND threshold of the chromaticity of pixel 1, JND threshold of the chromaticity of pixel 2, JND threshold of the chromaticity of pixel 3, and JND threshold of the chromaticity of pixel 4 are equal and equal to (JND_Y_1+ JND_Y_2+JND_Y_3+ JND_Y_4) / 4.

[0133] In this way, the calculation cost of the JND threshold of the chromaticity can be reduced, and the hardware implementation of the JND threshold calculation of the chromaticity can be facilitated.

[0134] The disclosed embodiment also provides a device for determining a JND threshold of an image. Fig. 9 A schematic diagram showing the structure of a device for determining a JND threshold of an image according to an embodiment of the present disclosure.

[0135] like Fig. 9 As shown, in a possible implementation manner, the JND threshold includes a brightness JND threshold, and the device includes:

[0136] The traversal module 91 is used to traverse each pixel in the image, wherein when any pixel is accessed,

[0137] A first determination module 92 is used to determine a brightness adaptive threshold and a texture masking threshold of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the brightness adaptive threshold represents the minimum brightness change perceptible to the human eye, and the texture masking threshold represents the maximum complexity of the texture that allows the noise to be perceived by the human eye when both noise and texture exist; wherein the edge detection operator corresponding to a single traversal image is used when determining the texture masking threshold;

[0138] The second determination module 93 is used to determine the brightness JND threshold of the pixel according to the brightness adaptive threshold of the pixel, the texture masking threshold and the preset superposition effect adjustment factor.

[0139] In a possible implementation, determining the brightness adaptive threshold and texture masking threshold of the pixel based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: calculating the product of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel and a low-pass filter operator as the background brightness value of the pixel; and finding the brightness adaptive threshold corresponding to the background brightness value from a brightness adaptive threshold lookup table as the brightness adaptive threshold of the pixel.

[0140] In a possible implementation, the method of determining a brightness adaptive threshold and a texture masking threshold of a pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: determining a gradient feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the gradient feature represents the maximum value of the gradient of the pixel in multiple preset directions; determining an edge feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the edge feature indicates whether the pixel is an edge of an image; wherein an edge detection operator corresponding to a single traversal of the image is used when determining the edge feature; and determining the texture masking threshold of the pixel according to the product of the gradient feature of the pixel, the edge feature and a preset control factor.

[0141] In a possible implementation, determining the gradient feature of the pixel based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: for each of the multiple directions, calculating the product of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel and the gradient convolution operator in that direction, averaging the product according to the size of the pixel neighborhood and the gradient convolution operator, and taking the absolute value to obtain the gradient of the pixel in that direction; and taking the maximum value of the gradients of the pixel in multiple directions as the gradient feature of the pixel.

[0142] In a possible implementation, the edge detection operator corresponding to the single-pass image includes a Sober-Federman operator, and determining the edge feature of the pixel based on the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: calculating the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the Sober-Federman operator in the horizontal direction to obtain a first value; calculating the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the Sober-Federman operator in the vertical direction to obtain a second value; calculating the average value of the first value and the second value, and the difference between a third value and the average value, the third value being the maximum pixel value; and taking the ratio of the difference to the third value as the edge feature of the pixel.

[0143] In a possible implementation, the brightness JND threshold of the pixel is determined based on the brightness adaptation threshold, texture masking threshold and a preset overlay effect adjustment factor of the pixel, including: calculating the smaller of the brightness adaptation threshold and the texture masking threshold of the pixel, and multiplying the result by the overlay effect adjustment factor; summing the brightness adaptation threshold and the texture masking threshold of the pixel, and then subtracting the sum from the product to obtain a fourth value, and determining the fourth value as the brightness JND threshold of the pixel.

[0144] In a possible implementation, the JND threshold also includes a chroma JND threshold, and the device also includes: a third determination module, used to determine at least one pixel corresponding to the pixel when any pixel is accessed, according to the chroma sampling method of the pixel; a fourth determination module, used to calculate the average value of the brightness JND threshold of at least one pixel corresponding to the pixel as the chroma JND threshold of the pixel.

[0145] In a possible implementation manner, at least one of the superposition effect adjustment factor and the control factor is an integer power of 2.

[0146] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0147] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0148] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0149] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0150] Fig.10 1 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Fig.10 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0151] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0152] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0153] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0154] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the above. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0155] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0156] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

[0157] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0158] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0159] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0160] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0161] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for determining a JND threshold of an image, characterized in that: The JND threshold comprises a brightness JND threshold, and the method comprises: Traverse each pixel in the image, where when any pixel is accessed, Determine a brightness adaptive threshold and a texture masking threshold of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the brightness adaptive threshold represents the minimum brightness change perceptible to the human eye, and the texture masking threshold represents the maximum complexity of the texture that allows the noise to be perceived by the human eye when both noise and texture exist; wherein an edge detection operator corresponding to a single traversal of the image is used when determining the texture masking threshold; The brightness JND threshold of the pixel is determined according to the brightness adaptive threshold of the pixel, the texture masking threshold and the preset superposition effect adjustment factor.

2. The method according to claim 1, characterized in that: The step of determining the brightness adaptive threshold and the texture masking threshold of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: Calculate the product of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel and the low-pass filter operator as the background brightness value of the pixel; From the brightness adaptive threshold lookup table, find the brightness adaptive threshold corresponding to the background brightness value as the brightness adaptive threshold of the pixel.

3. The method according to claim 1, characterized in that The step of determining the brightness adaptive threshold and the texture masking threshold of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: Determine a gradient feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the gradient feature represents the maximum value of the gradient of the pixel in multiple preset directions; Determine an edge feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the edge feature indicates whether the pixel is an edge of the image; wherein an edge detection operator corresponding to a single traversal of the image is used when determining the edge feature; The texture masking threshold of the pixel is determined according to the product of the gradient feature, the edge feature and the preset control factor of the pixel.

4. The method according to claim 3, characterized in that: Determining the gradient feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: For each of the multiple directions, calculate the product of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel and the gradient convolution operator in the direction, average the products according to the size of the pixel neighborhood and the gradient convolution operator, and take the absolute value to obtain the gradient of the pixel in the direction; The maximum value of the gradient of the pixel in multiple directions is used as the gradient feature of the pixel.

5. The method according to claim 3, characterized in that: The edge detection operator corresponding to the single traversal image includes a Sober-Feldman operator, Determining the edge feature of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel includes: Calculate the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the Sober-Feldman operator in the horizontal direction to obtain a first value; Calculate the convolution result of the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel with the Sober-Feldman operator in the vertical direction to obtain a second value; Calculate an average value of the first value and the second value, and a difference between a third value and the average value, wherein the third value is a maximum pixel value; The ratio of the difference value to the third value is used as the edge feature of the pixel.

6. The method according to claim 1, characterized in that The step of determining the brightness JND threshold of the pixel according to the brightness adaptive threshold of the pixel, the texture masking threshold and the preset superposition effect adjustment factor includes: Calculate the product of the smaller one of the brightness adaptive threshold and the texture masking threshold of the pixel and the superposition effect adjustment factor; The brightness adaptive threshold and the texture masking threshold of the pixel are summed, and then subtracted from the product to obtain a fourth value, and the fourth value is determined as the brightness JND threshold of the pixel.

7. The method according to claim 1, characterized in that The JND threshold also includes a chrominance JND threshold, and the method further includes: When any pixel is accessed, at least one pixel corresponding to the pixel is determined according to the chroma sampling mode of the pixel; An average value of the brightness JND threshold of at least one pixel corresponding to the pixel is calculated as the chrominance JND threshold of the pixel.

8. The method according to claim 3, characterized in that At least one of the stacking effect adjustment factor and the control factor is an integer power of 2.

9. A device for determining a JND threshold of an image, characterized in that: The JND threshold comprises a brightness JND threshold, and the device comprises: The traversal module is used to traverse each pixel in the image. When any pixel is accessed, A first determination module is used to determine a brightness adaptive threshold and a texture masking threshold of the pixel according to the pixel value of the pixel and the pixel values ​​of other pixels in the neighborhood of the pixel, wherein the brightness adaptive threshold represents a minimum brightness change perceptible to the human eye, and the texture masking threshold represents a maximum complexity of the texture that allows the noise to be perceived by the human eye in the presence of both noise and texture; wherein an edge detection operator corresponding to a single traversal image is used when determining the texture masking threshold; The second determination module is used to determine the brightness JND threshold of the pixel according to the brightness adaptive threshold of the pixel, the texture masking threshold and the preset superposition effect adjustment factor.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 8 when executing the instructions stored in the memory.

11. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images