Image processing method and device, electronic device, and storage medium

Abstract

The present disclosure relates to an image processing method and device, an electronic device, and a storage medium. The method includes: obtaining an image to be processed in a luminance-chrominance space domain, and performing a downsampling operation on the image to be processed to obtain a low-frequency image of the image to be processed; determining the chromaticity weights of each first pixel according to the degree of brightness difference between each first pixel in the image to be processed and its respective first neighborhood pixel, wherein the chromaticity weight of each first pixel in the first pixels is positively correlated with the corresponding degree of brightness difference; performing an upsampling operation on the low-frequency image based on the chromaticity weights corresponding to each first pixel, and synthesizing the image obtained by the upsampling operation with the image to be processed to obtain a processed image. Through this method, the chromaticity value of the image can be adjusted during the image processing process to avoid color edge color overflow in the processed image.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method and device, electronic equipment, and storage medium. Background Art

[0002] In the process of collecting, transmitting and processing images, electronic devices are inevitably interfered by various factors, resulting in noise signals in the obtained images. The noise signals accompanying the images mainly include two types of noise signals, namely brightness noise and color noise (also known as chrominance noise).

[0003] In the related art, there are relatively mature noise reduction technologies for the processing of brightness noise. However, it is still difficult to effectively reduce the color noise in the image, especially at the color edge with a large color span, where color overflow or blur often occurs. Summary of the invention

[0004] In view of this, the present disclosure provides an image processing method and device, an electronic device, and a storage medium. In the process of upsampling a low-frequency image, the chromaticity value of the low-frequency image can be adjusted according to the brightness distribution of the high-frequency image to avoid color edge color overflow in the image obtained by upsampling.

[0005] To achieve the above objectives, the present disclosure provides the following technical solutions:

[0006] According to a first aspect of the present disclosure, an image processing method is proposed, comprising:

[0007] Acquire an image to be processed in a luminance-chrominance space domain, and perform a downsampling operation on the image to be processed to obtain a low-frequency image of the image to be processed;

[0008] Determining the chromaticity weights of the first pixels according to the brightness difference between the first pixels and the first neighboring pixels in the image to be processed, wherein the chromaticity weight of each of the first pixels is positively correlated with the corresponding brightness difference;

[0009] An upsampling operation is performed on the low-frequency image based on the chrominance weights corresponding to the first pixels, and an image obtained by the upsampling operation is synthesized with the image to be processed to obtain a processed image.

[0010] According to a second aspect of the present disclosure, an image processing device is provided, comprising:

[0011] An acquisition unit, which acquires an image to be processed in a luminance-chrominance space domain, and performs a downsampling operation on the image to be processed to obtain a low-frequency image of the image to be processed;

[0012] A determination unit, which determines the chromaticity weights of the first pixels in the image to be processed according to the brightness difference between the first pixels and the first neighboring pixels, respectively, wherein the chromaticity weight of each of the first pixels is positively correlated with the corresponding brightness difference;

[0013] A synthesis unit performs an up-sampling operation on the low-frequency image based on the chrominance weights corresponding to the first pixels, and synthesizes the image obtained by the up-sampling operation with the image to be processed to obtain a processed image.

[0014] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0015] processor;

[0016] a memory for storing processor-executable instructions;

[0017] The processor implements the method as described in the first aspect by running the executable instructions.

[0018] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0019] In the technical solution disclosed in the present invention, an image processing method is provided. On the one hand, the method performs a downsampling operation on the image to be processed in the acquired brightness-chrominance space domain to obtain a corresponding low-frequency image; on the other hand, according to the brightness difference between each first pixel in the image to be processed and each first neighboring pixel, a chromaticity weight is assigned to each first pixel included, and the chromaticity weight of each first pixel is positively correlated with the corresponding brightness difference. On this basis, the low-frequency image can be upsampled based on the chromaticity weights corresponding to each first pixel in the image to be processed to obtain an image with the same size as the image to be processed, and the image is merged with the image to be processed to obtain a processed image.

[0020] It is understandable that in the field of images, changes in color in an image will affect changes in brightness, and the larger the color change, the larger the brightness change. In other words, changes in brightness in an image can reflect changes in color. For any pixel, the greater the degree of brightness difference with neighboring pixels, it means that the chromaticity difference between any pixel and its neighboring pixels is usually larger. The chromaticity weight of any pixel in the present disclosure is positively correlated with the corresponding degree of brightness difference, which is equivalent to assigning higher chromaticity weights to pixels in areas with large brightness changes (color edges). On this basis, upsampling the low-frequency image by determining the chromaticity weight is equivalent to increasing the chromaticity value of the pixel at the color edge in the low-frequency image, so that the color change at the color edge is more prominent, thereby avoiding the problem of color overflow at the color edge in the related art.

[0021] In short, the present invention identifies the color edges in the image by analyzing the brightness changes in the image, and then increases the difference between the chromaticity value of the pixels at the color edge and the chromaticity value of the pixels at the color flat area by assigning a higher chromaticity weight to the pixels at the color edge, thereby highlighting the difference between the color edge and the color flat area, and solving the problem of color overflow at the color edge in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] Figure 1 is a flowchart of an image processing method shown in an exemplary embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram of an image to be processed shown in an exemplary embodiment of the present disclosure;

[0025] Figure 3 is a schematic diagram of a low-frequency image shown in an exemplary embodiment of the present disclosure;

[0026] Figure 4A is a schematic diagram of an image obtained by an upsampling operation shown in an exemplary embodiment of the present disclosure;

[0027] Figure 4B is a schematic diagram showing a comparison between an upsampling operation performed by bilinear interpolation and a joint guided upsampling operation according to an exemplary embodiment of the present disclosure;

[0028] Figure 5 is a block diagram of an image processing device shown in an exemplary embodiment of the present disclosure;

[0029] Figure 6is a block diagram of another image processing device shown in an exemplary embodiment of the present disclosure;

[0030] Figure 7 It is a schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0032] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0033] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0034] In the process of collecting, transmitting and processing images, electronic devices are inevitably interfered by various factors, resulting in noise signals in the obtained images. The noise signals accompanying the images mainly include two types of noise signals, namely brightness noise and color noise (also known as chrominance noise).

[0035] In the related art, there are relatively mature noise reduction technologies for the processing of brightness noise. However, it is still difficult to effectively reduce the color noise in the image, especially at the color edge with a large color span, where color overflow or blur often occurs.

[0036] Specifically, since color noise is more obvious in low-frequency images, the related art usually uses a multi-scale noise reduction framework to perform noise reduction on the image. In the process of processing through the framework, it is necessary to perform at least one downsampling operation on the image to be processed to obtain at least one low-frequency image corresponding to the image to be processed. Then, the at least one low-frequency image obtained and the image to be processed can be noise reduced respectively, and then the low-frequency image after noise reduction can be upsampled to obtain the final processed image.

[0037] Taking the example of performing two downsampling operations, in this example, the image to be processed is called a high-frequency image, the image obtained by the first downsampling is called a medium-frequency image, and the image obtained by the second downsampling is called a low-frequency image. Specifically:

[0038] First, the high-frequency image can be downsampled once to obtain a smaller intermediate-frequency image; then the intermediate-frequency image can be downsampled to obtain a smaller low-frequency image. Then, the high-frequency image, the intermediate-frequency image, and the low-frequency image can be denoised respectively to perform relatively detailed denoising. After the denoising is completed, the low-frequency image can be upsampled to obtain an image with the same size as the intermediate-frequency image, which is used to synthesize with the intermediate-frequency image after the denoising; further, the synthesized image can be upsampled to obtain an image with the same size as the high-frequency image, which is used to synthesize with the high-frequency image after the denoising to obtain a processed image.

[0039] In the process of denoising through the multi-scale denoising framework, the low-frequency image (including the upsampling operation on the intermediate frequency image) is upsampled only based on its own chrominance information during the upsampling process, which cannot solve the problem of color overflow or blur at the color edge.

[0040] What is more serious is that since the above-mentioned series of operations of "downsampling operation → noise reduction processing → upsampling operation" involves multiple changes in image scale, the image resolution will be reduced, and may even cause this problem in images that originally did not have this problem (for example, in the upsampling process, the determined interpolation is only determined by its own chromaticity value, which may cause the chromaticity difference between adjacent pixels to decrease, thereby causing color blurring at the color edges).

[0041] To this end, the present disclosure proposes a method that can identify the color edge of an image and adjust the chromaticity value of the pixels at the color edge to remove the color noise at the color edge, thereby avoiding color overflow or blur at the color edge in the related art.

[0042] Figure 1 An image processing method is shown as an exemplary embodiment of the present disclosure. Figure 1 As shown, the method may include the following steps:

[0043] Step 102: Acquire an image to be processed in a luminance-chrominance space domain, and perform a down-sampling operation on the image to be processed to obtain a low-frequency image of the image to be processed.

[0044] The technical solution of the present disclosure can be applied to any type of electronic device, for example, the electronic device can be a mobile terminal such as a smart phone, a tablet computer, or a fixed terminal such as a smart TV, a PC (personal computer). It should be understood that any electronic device that can perform image processing can be used as the electronic device in the present disclosure, and the specific type of electronic device to which the technical solution of the present disclosure is applied can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0045] It is understandable that the reason why color overflow or blurring occurs at the color edge in the image is that the difference in chromaticity values ​​between the color edge and the color flat area is not large enough. For example, assuming that the border between the color flat area A and the color flat area B is the color edge C, if the chromaticity value of the pixel X at the color edge is 5, and the chromaticity value of the pixel Y in the color flat area A that is closer to the color edge C is 4, it is obvious that the difference in the chromaticity values ​​of the two pixels is not large, so the color edge C is not very obvious, that is, from a visual effect, it is easy to have color blurring, or the color of the color edge C spreads to the color flat area A. It can be seen that the reason for the color overflow phenomenon at the color edge in the image is that the difference in pixel chromaticity values ​​between the color edge and the color flat area is small.

[0046] In view of this, the present disclosure takes into account the rule that "areas with large color changes in an image usually have large brightness changes", determines the color edge and color flat area in the image through the brightness value of each pixel in the image, and then assigns chromaticity weights for adjusting the chromaticity value to the pixels in different areas. Among them, the pixels at the color edge are assigned a higher chromaticity weight, while the pixels in the color flat area are assigned a relatively lower chromaticity weight, so that the chromaticity value of the pixels at the color edge of the adjusted image increases the chromaticity difference with the pixels in the color flat area, thereby avoiding the problem of color overflow at the color edge.

[0047] Since the present disclosure needs to adjust the chromaticity value of a pixel based on the brightness value of the pixel, it is obviously necessary that the image to be processed is an image in the brightness-chromaticity space domain. Therefore, when the initial image to be processed is an image in other space domains, it is necessary to preferentially convert the image in other space domains into an image in the brightness-chromaticity space domain. For example, when the initial image to be processed is an image in the RGB space domain, it is necessary to preferentially convert the image in the RGB space domain into an image in the brightness-chromaticity space domain.

[0048] The image to be processed obtained in the present disclosure may be any type of image in the luminance-chrominance space domain. For example, the image to be processed may be an image in the YUV space domain; or, it may be an image in the HIS space domain. The specific type of image in the luminance-chrominance space domain of the image to be processed may be determined by those skilled in the art according to actual conditions, and the present disclosure does not limit this.

[0049] Since the present disclosure aims to remove the color noise in the image to be processed, and the color noise usually appears in the form of low frequency, after obtaining the image to be processed in the luminance-chrominance space domain, the image to be processed can be preferentially downsampled to obtain a low-frequency image of the image to be processed, and then the above-mentioned chromaticity value adjustment operation is performed on the low-frequency image.

[0050] Before introducing the technical solution of the present invention in detail, it should be stated that since the present solution involves many concepts, for the sake of convenience in distinction, a pixel in the image to be processed is referred to as a “first pixel”, the neighborhood pixels of all pixels in the image to be processed are referred to as “first neighborhood pixels”, the window area obtained from the image to be processed is referred to as a “first window area”, a pixel in the low-frequency image is referred to as a “second pixel”, the neighborhood pixels of all pixels in the low-frequency image are referred to as “second neighborhood pixels”, and the window area obtained from the low-frequency image is referred to as a “second window area”.

[0051] In fact, in the related art, in addition to the color overflow or blurring at the color edge of the image, there are also color noise points in the color flat area. Therefore, the present disclosure may also include the step of performing noise reduction processing on the image to be processed and / or the low-frequency image. Among them, when performing noise reduction processing on the image to be processed, the entire image to be processed can be traversed by a sliding window, and the chromaticity values ​​of all first pixels in any first window area obtained by the sliding window are weighted, and the calculated first target value corresponding to any first window area is used as the chromaticity value of the central pixel of any first window area.

[0052] When performing noise reduction processing on a low-frequency image, the process is similar. The entire low-frequency image can be traversed by means of a sliding window, and the chromaticity values ​​of all second pixels in any second window area obtained by the sliding window are weightedly calculated, and the calculated second target value corresponding to any second window area is used as the chromaticity value of the central pixel of any second window area.

[0053] In actual operation, noise reduction processing is mostly performed on both the processed image and the low-frequency image, but noise reduction processing can also be performed on only one of the two. Technical personnel in this field can determine whether to perform noise reduction processing on the processed image and / or the low-frequency image according to actual needs, and the present disclosure does not limit this.

[0054] It should be noted that sliding window is a relatively common way of obtaining values ​​in the field of images. Usually, a window of fixed size is slid with a specific step size, so that the values ​​of pixels in the image that fall within the window are used as the values ​​for calculation (including the values ​​of each channel such as chromaticity value and brightness value). In the present disclosure, the entire image can be traversed with a step size of 1 pixel, so that each pixel in the image is used as the center pixel for chromaticity value adjustment.

[0055] In actual calculation, the chromaticity value of the central pixel of any of the first window areas may be calculated in a variety of ways.

[0056] In one embodiment, after obtaining any first window area from the image to be processed by means of a sliding window, the difference between all first pixels in the any first window area and the central pixel in the any first window area can be calculated, so as to assign noise reduction weights to all first pixels in the any first window area according to the difference, and the noise reduction weight of any first pixel in the any first window area is negatively correlated with the difference corresponding to the any first pixel. On this basis, the chromaticity values ​​of all first pixels in the any first window area can be weighted averaged based on the noise reduction weights of each first pixel in the any first window area, so as to use the calculated value as the chromaticity value of the central pixel of the any first window area.

[0057] After obtaining any second window area from the low-frequency image by means of a sliding window, the chromaticity value of the central pixel of any second window area can be adjusted in a similar manner. For example, the difference between all the second pixels in the any second window area and the central pixel in the any second window area can be calculated, so as to assign noise reduction weights to all the second pixels in the any second window area according to the difference, and the noise reduction weight of any second pixel in the any second window area is negatively correlated with the difference corresponding to the any second pixel. On this basis, the chromaticity values ​​of all the second pixels in the any second window area can be weighted averaged based on the noise reduction weights of each second pixel in the any second window area, so as to use the calculated value as the chromaticity value of the central pixel of the any second window area.

[0058] For example, assuming that the chromaticity value of each first pixel in the image to be processed is Figure 2 As shown, the entire image can be traversed by means of a sliding window to obtain a plurality of first window areas, and the chromaticity value of the central pixel in any first window area can be adjusted by the chromaticity value of each first pixel in the first window area.

[0059] by Figure 2 Taking the first window area A shown in FIG. 1 as an example, how to adjust the chromaticity value of the central pixel of any first window area in this embodiment is described:

[0060] Depend on Figure 2 It can be seen that the first window area A contains 9 first pixels, and the chromaticity value of the central pixel X is 102. Then, the noise reduction weights of the 9 first pixels in the first window area A are determined based on the chromaticity value 102. Among them, the noise reduction weight of any first pixel in the first window area A is negatively correlated with the difference between any first pixel and 102. For example, the chromaticity value of the first pixel M in the upper left corner of the first window area A is 95, and the difference with 102 is 7; the chromaticity value of the first pixel N in the lower left corner is 85, and the difference with 102 is 17. Since 7 is less than 17, the noise reduction weight of the first pixel M is greater than the noise reduction weight of the first pixel N. For example, the noise reduction weight of the first pixel M can be 1.10, and the noise reduction weight of the first pixel N can be 0.60. Assume that the noise reduction weights of all first pixels in the first window area A determined in this way are shown in Table 1 below:

[0061]

[0062] Table 1

[0063] Then, based on Table 1, the chromaticity value of the central pixel after noise reduction processing can be calculated according to the noise reduction weights of all first pixels in the first window area A. For example, the calculation can be performed using a weighted average algorithm, and the calculation process can be:

[0064] (95*1.1+94*1.0+85*0.60+83*0.55+91*0.80+93*0.95+99*1.4+108*1.2+102*1.45) / (1.1+1.0+0.60+0.55+0.80+0.95+1.4+1.2+1.45)=96.40.

[0065] At this time, 96.40 can be used as the chromaticity value of the center pixel of the first window area A, that is, the original chromaticity value 102 of the center pixel of the first window area A is adjusted to 96.40. It should be understood that since the sliding window will traverse the entire image, all first pixels in the image will be used as center pixels and then undergo the above process of adjusting the chromaticity value. When the chromaticity values ​​of all first pixels are adjusted, it can be considered that the noise reduction processing of the image to be processed is completed.

[0066] The process of adjusting the chromaticity of any second pixel in the low-frequency image can refer to the above examples, wherein the processing method for the "low-frequency image", "second pixel" and "second window area" can be adapted to the processing method for the "image to be processed", "first pixel" and "first window area", and the examples will not be repeated here.

[0067] From the above adjustment process, it can be seen that the pixel with a larger difference in chromaticity value with the central pixel in the sliding window has a smaller denoising weight, thereby reducing the difference in chromaticity value between the adjusted central pixel and the neighboring pixels. From a visual perspective, the color noise in the image will be screened out, achieving a denoising effect for color noise.

[0068] However, it can be seen from Table 1 that, since the above embodiment determines the noise reduction weights of each first pixel in the first window area A based on the chromaticity value of the central pixel, the noise reduction weight of the central pixel itself is always the largest, which makes it difficult to achieve effective noise reduction when performing noise reduction on isolated color noise in a color flat area. Figure 2 Take the first window area B in the example: It is not difficult to see that in the first window area B, the chromaticity value of the central pixel is 125, which is much larger than the chromaticity values ​​of other first pixels in the first window area B, and the chromaticity values ​​of several first neighborhood pixels around the central pixel are relatively close. Obviously, from the visual effect, the central pixel belongs to the isolated color noise in the color flat area. Since in the above embodiment, the denoising weight of the central pixel is always the largest, after the first pixel with a chromaticity value of 125 is subjected to denoising, the chromaticity value of the central pixel is still much larger than the chromaticity value of the first neighborhood pixel of the central pixel, that is, the isolated noise in the color flat area cannot be effectively removed.

[0069] In view of this, the present disclosure proposes another noise reduction processing method.

[0070] In another embodiment, the noise reduction weights of all pixels in any window area are no longer determined based on the central pixel, but the noise reduction weights are assigned to the pixels in any window area based on the average of the chromaticity values ​​of all pixels in any window area.

[0071] By way of example, in the process of performing noise reduction processing on the processed image, the following operations may be performed on any first window area: first, the chromaticity mean of all first pixels in the any first window area is calculated, and the first noise reduction weight of each first pixel in the any first window area is determined based on the first difference between the chromaticity value of each first pixel in the any first window area and the chromaticity mean, wherein the first noise reduction weight of any first pixel in the any first window area is negatively correlated with the first difference corresponding to the any first pixel; then, based on the first noise reduction weight of each first pixel in the any first window area, the chromaticity values ​​of all first pixels in the any first window area may be weighted averaged to obtain the first target value corresponding to the any first window area, and the first target value may be used as the chromaticity value of the central pixel of the any first window area.

[0072] In the process of denoising the low-frequency image, it is similar: the following operations can be performed on any second window area: first, the chromaticity mean of all second pixels in the any second window area is calculated, and the second denoising weight of each second pixel in the any second window area is determined according to the second difference between the chromaticity value of each second pixel in the any second window area and the chromaticity mean, wherein the second denoising weight of any second pixel in the any second window area is negatively correlated with the second difference corresponding to the any second pixel; then, based on the second denoising weight of each second pixel in the any second window area, the chromaticity values ​​of all second pixels in the any second window area can be weighted averaged to obtain the second target value corresponding to the any second window area, and the second target value is used as the chromaticity value of the central pixel of the any second window area.

[0073] by Figure 2 Taking the first window area B in the image to be processed as an example, how to adjust the chromaticity value of the central pixel of any first window area in this embodiment is introduced:

[0074] First, the chromaticity mean of all first pixels in the first window area B is calculated: (98+86+99+89+85+91+66+84+125) / 9=91.44 (for the convenience of calculation, 91 is taken as an example in the following text). Then, the first difference between the chromaticity value of each first pixel in the first window area B and the chromaticity mean is calculated, so as to assign a noise reduction weight to the corresponding first pixel according to the first difference corresponding to each first pixel. For example, the assigned noise reduction weight can be shown in the following Table 2:

[0075]

[0076]

[0077] Table 2

[0078] Then, based on Table 2, the chromaticity value of the central pixel after noise reduction processing can be calculated according to the noise reduction weights of all first pixels in the first window area B. For example, the calculation can be performed using a weighted average algorithm, and the calculation process can be:

[0079] (98*1.1+86*1.3+99*1.0+89*1.35+85*1.20+91*1.45+66*0.40+84*1.10+125*0.30) / (1.1+1.3+1.0+1.35+1.20+1.45+0.40+1.10+0.30)=90.11.

[0080] At this time, 90.11 can be used as the chromaticity value of the center pixel, that is, the original chromaticity value 125 of the center pixel of the first window area B is adjusted to 90.11. As in the previous embodiment, since the sliding window traverses the entire image to be processed, all first pixels in the image will be used as center pixels, and then undergo the above-mentioned process of adjusting the chromaticity value. When the chromaticity values ​​of all first pixels are adjusted, it can be considered that the noise reduction process of the image to be processed is completed.

[0081] The process of performing chromaticity adjustment on any second pixel in the low-frequency image in this embodiment can also refer to the above examples, wherein the processing method for the "low-frequency image", "second pixel" and "second window area" can be adapted to the processing method for the "image to be processed", "first pixel" and "first window area", and the examples will not be repeated here.

[0082] It can be seen from Table 2 above that in this embodiment, the chromaticity value of 125 of the central pixel is no longer used as the basis for assigning weights to the first pixels in the first window area B, so that the first pixel with the largest weight is usually not the central pixel. For example, in Table 2 above, since the obtained chromaticity mean value is 91, it is obvious that in the window area B, the first pixel with a pixel value of 91 has the largest denoising weight, thereby avoiding the technical problem in the previous embodiment that the isolated color noise in the color flat area cannot be removed because the weight value of the central pixel is always the largest.

[0083] It is not difficult to understand that in the above embodiment, since the noise reduction weight is assigned to each pixel based on the central pixel, the noise reduction weight of the central pixel is always the largest. Obviously, if the central pixel is isolated color noise in a color flat area (i.e., the chrominance value is greatly different from the chrominance values ​​of the surrounding pixels), after adjusting the chrominance value of the central pixel on this basis, the chrominance value of the central pixel is still greatly different from the chrominance values ​​of the surrounding pixels, so that the color noise cannot be removed.

[0084] In this embodiment, since the weights are allocated based on the chromaticity mean of all pixels in the window area, the denoising weight of the central pixel is usually not the largest. On the contrary, if the central pixel is isolated color noise in the color flat area, since the chromaticity values ​​of the multiple neighboring pixels of the central pixel are not much different, the chromaticity mean obtained will have a larger chromaticity difference with the central pixel, and a smaller chromaticity difference with the neighboring pixels of the central pixel, resulting in a smaller denoising weight for the central pixel. For example, in the first window area B in the above example, the denoising weight of the central pixel is the smallest. Obviously, after the image is subjected to denoising processing by this embodiment, the isolated color noise in the color flat area can be effectively eliminated. For example, after the chromaticity value of the central pixel of the window area B is adjusted, the adjusted chromaticity value 90.11 is obviously closer to the chromaticity value of most of the first pixels in the window area B. From the visual effect, there is no isolated color noise in the color flat area.

[0085] It should be stated that the values ​​of the noise reduction weights of the first pixels in Tables 1 and 2 are only schematic and are only used to indicate that the noise reduction weight of any first pixel is negatively correlated with the difference between the first pixel and the center pixel (or the above-mentioned chromaticity mean). How to determine the noise reduction weights of each first pixel can be determined by those skilled in the art according to actual conditions, and the present disclosure does not limit this. For example, the correspondence between the above-mentioned difference and the noise reduction weight can be set as a basis for determining the noise reduction weight of any first pixel; for example, the first pixels can be sorted according to the difference corresponding to each first pixel, so as to allocate the noise reduction weight according to the order of arrangement. It is similar when performing noise reduction processing on low-frequency images, which will not be repeated here.

[0086] Step 104, determining the chromaticity weights of each first pixel in the image to be processed according to the brightness difference between each first pixel and its respective first neighborhood pixel, wherein the chromaticity weight of each first pixel in the first pixels is positively correlated with the corresponding brightness difference.

[0087] From the above content, it can be known that the reason why the color overflow or blurring occurs at the color edge in the image is that the difference between the chromaticity value of the pixel at the color edge and the chromaticity value of the pixel in the color flat area is too small. Therefore, in the present disclosure, the color edge and the color flat area in the image are determined based on the brightness value of each pixel in the image, and a higher chromaticity weight is assigned to the pixel at the color edge, and a relatively lower chromaticity weight is assigned to the pixel in the color flat area, so that the difference between the chromaticity value of the pixel at the color edge and the chromaticity value of the pixel in the color flat area is increased, thereby solving the problem of color overflow or blurring at the color edge.

[0088] It should be noted that, after performing a downsampling operation on the image to be processed to obtain a low-frequency image, the present disclosure assigns chromaticity weights to each first pixel in the image to be processed according to the brightness value of each first pixel in the image to be processed. Then, by means of the correspondence between each first pixel in the image to be processed and each second pixel in the low-frequency image, the chromaticity value of each second pixel in the low-frequency image is adjusted during the upsampling operation on the low-frequency image. In other words, the chromaticity value of each second pixel in the low-frequency image is adjusted based on the brightness distribution in the image to be processed.

[0089] This is because the low-frequency image is obtained based on the image to be processed, and the brightness distribution of the two is almost the same. The image to be processed can be regarded as: a high-frequency image relative to the low-frequency image, which is larger in size and contains more brightness details. The chromaticity weight of each first pixel is determined more accurately through the image to be processed.

[0090] Step 106: perform an upsampling operation on the low-frequency image based on the chrominance weights corresponding to the pixels, and synthesize the image obtained by the upsampling operation with the image to be processed to obtain a processed image.

[0091] In actual operation, the first neighboring pixels of each first pixel can be determined by a sliding window, so as to determine the chromaticity weight of each first pixel according to the brightness difference between each first pixel and its first neighboring pixels. Figure 2Taking the first window area A shown as an example, other first pixels except the central pixel can be regarded as the first neighborhood pixels of the central pixel (of course, in the calculation process, the central pixel can also be regarded as its own first neighborhood pixel). Of course, the first window area A is only taken as an example that the sliding window includes 9 first pixels. If the sliding window includes 25 first pixels, the number of first neighborhood pixels is also relatively increased. The specific method of determining the first neighborhood pixels can be determined by those skilled in the art according to actual needs, and the present disclosure is not limited to this.

[0092] In one embodiment, when allocating chromaticity weights to any first pixel, the chromaticity weights may be allocated in combination with all first neighboring pixels of the any first pixel. For example, the brightness difference between the any first pixel and any first neighboring pixel may be preferentially calculated, and the weight parameter corresponding to the any first neighboring pixel may be calculated based on the brightness difference. After the weight parameters corresponding to all first neighboring pixels of the any first pixel are calculated by this method, the chromaticity weights may be allocated to the any first pixel based on all the calculated weight parameters.

[0093] Still Figure 2 As an example, assume that Figure 2 The values ​​of the first pixels shown in are the brightness values ​​of the corresponding first pixels. Then, the following calculation can be performed for any first pixel in the first window area A: Taking the first pixel L in the upper right corner with a brightness value of 99 as an example (a first neighboring pixel of the center pixel X in the first window area A, which can be regarded as any of the above-mentioned first neighboring pixels), the brightness difference between the first pixel L and the center pixel X is calculated to be 3, and a weight parameter is assigned to the first pixel L based on the brightness difference 3. After completing the above operation for all first neighboring pixels of the center pixel X in the first window area A and obtaining the weight parameter corresponding to each first neighboring pixel of the center pixel X, the chromaticity weight of the center pixel X in the first window area A can be determined based on all the obtained weight parameters.

[0094] It should be noted that, in this embodiment, although a weight parameter is assigned to each first neighborhood pixel based on the brightness difference between any first pixel and its first neighborhood pixel, in actual operation, any pixel can be regarded as its own neighborhood pixel, that is, any first pixel itself will also be assigned a weight parameter. In other words, when weight parameters are assigned to the first neighborhood pixels of the above-mentioned central pixel X, 9 weight parameters will eventually be assigned.

[0095] After the chrominance weights of all first pixels in the image to be processed are obtained in the above manner, the chrominance values ​​of each second pixel in the low-frequency image can be adjusted during the upsampling operation on the low-frequency image.

[0096] For example, when calculating the chromaticity value of any pixel in the image obtained by the upsampling operation, the brightness value of the first pixel corresponding to the any pixel in the image to be processed and the chromaticity value of the second pixel corresponding to the any pixel in the low-frequency image can be used as the basis for calculation. Since the size of the image obtained by the upsampling operation is consistent with that of the image to be processed, it can also be expressed as: according to the brightness value of any first pixel in the image to be processed and the chromaticity value of the second pixel corresponding to the any pixel in the low-frequency image, the chromaticity value of the target pixel corresponding to the any first pixel in the image obtained by the upsampling operation is calculated.

[0097] In actual operation, after obtaining the weight parameter of any first neighboring pixel of any first pixel in the image to be processed in the above manner, the second pixel corresponding to the any first pixel can be determined in the low-frequency image, and the second neighboring pixel corresponding to the any first neighboring pixel can be further determined from the second neighboring pixels of the second pixel. On this basis, the distance between the second neighboring pixel and the second pixel can be calculated, and the chromaticity reference value corresponding to the any first neighboring pixel can be determined based on the distance and the weight parameter of the any first neighboring pixel; after determining multiple chromaticity reference values ​​of all first neighboring pixels corresponding to the any first pixel, the multiple chromaticity reference values ​​can be weighted averaged to obtain the chromaticity value of the target pixel corresponding to the any first pixel in the image obtained by the upsampling operation.

[0098] For example, the chrominance value of any pixel in the image obtained by the upsampling operation may be calculated by referring to the following formula:

[0099]

[0100] in, is the chromaticity value of any pixel in the image obtained by the upsampling operation; p is the first pixel corresponding to the above-mentioned any pixel in the image to be processed, and q is any first neighboring pixel of the first pixel; I p is the brightness value of the first pixel p, I q is the brightness value of the first neighborhood pixel q; p ↓ is the second pixel corresponding to the first pixel p in the low-frequency image (actually the coordinates of the second pixel. For the convenience of subsequent description, the second pixel p is used ↓ represents the second pixel), q ↓ is the second neighboring pixel corresponding to the first neighboring pixel q in the low-frequency image (actually the coordinates of the second pixel. For the convenience of subsequent description, the second pixel q is used ↓ represents the second pixel); ‖p ↓ -q ↓ ‖ is the second neighboring pixel q ↓ With the second pixel p ↓distance; is the chromaticity value of the second pixel corresponding to the first pixel p in the low-frequency image; Ω is the chromaticity value of the second pixel p ↓ The second window area of ​​the center pixel; k p is a normalization constant; f() is a coefficient function for calculating the chromaticity reference value based on the distance; g(‖I p -I q ‖) is a weight function that calculates the weight parameter according to the brightness difference, which is used to calculate the above weight parameters. is the chromaticity reference value mentioned above.

[0101] For ease of understanding, Figure 2 Assume that the image to be processed is Figure 2 Each value in is the brightness value corresponding to each first pixel. Assume that Figure 2 The low-frequency image obtained by downsampling the corresponding chromaticity map is as follows Figure 3 Obviously, in Figure 2 The first pixel in Figure 3 There is a certain correspondence between the second pixel in . For example, Figure 2 The size of the image to be processed is 8*8, and Figure 3 The size of the low-frequency image shown is 4*4, which means that any second pixel in the low-frequency image corresponds to four first pixels in the image to be processed, and the correspondence is related to the position of the pixel in the image. For example, the second pixel with a brightness value of 56 in the upper left corner of the low-frequency image should correspond to four first pixels with brightness values ​​of 65, 56, 77, and 58 in the upper left corner of the image to be processed.

[0102] What is currently known is: Figure 3 The size of the image obtained by upsampling the low-frequency image shown must be Figure 4A So in the upsampling process, what we need to do is to know how Figure 4A The chromaticity value of each pixel in the image shown. Figure 4A The chromaticity value of pixel X" in the example is used to introduce it. Obviously, the pixel X" is Figure 2 The central pixel of the first window area A in Figure 2 The first pixel X in corresponds to Figure 2 One of the top right 4 pixels in the Figure 3 The second pixel corresponding to the first pixel X is Figure 3The second pixel X' with a color value of 91 in the upper right corner. Similar to the first window area A, in the low-frequency image, a second window area with the second pixel X' as the center pixel can be determined by means of a sliding window. Assuming that a weight parameter needs to be assigned to the first neighborhood pixel Y of the first pixel X, the weight parameter of the first neighborhood pixel Y can be determined based on the brightness difference between the first pixel X and the first neighborhood pixel Y. The weight parameters of other neighborhood pixels of pixel X are determined in a similar manner; after determining the weight parameter of the first neighborhood pixel Y, the second neighborhood pixel Y' corresponding to the first neighborhood pixel Y can be determined from several second neighborhood pixels of the second pixel X', and then the chromaticity reference value of the first neighborhood pixel Y is determined based on the distance between the second neighborhood pixel Y' and the second pixel X' and the weight parameter of the first neighborhood pixel Y (of course, it can also be regarded as the chromaticity reference value of the second neighborhood pixel Y'). The specific formula for calculating the chromaticity reference value can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0103] It should be stated that in Figure 3 The second window region corresponding to the second pixel X' shown includes a region beyond the low-frequency image. In this partial region, the chromaticity value of the second pixel (which can be called a blank pixel) that does not exist in the original image can be supplemented by means of a supplementary value. In the art, the supplementary value technology is relatively mature, and any method can be used to supplement the blank pixels in the second window region, and the present disclosure does not limit this. For example, for any blank pixel, the chromaticity value of the second pixel closest to the blank pixel can be used as its chromaticity value.

[0104] After the chromaticity reference values ​​of all first neighborhood pixels of the first pixel X are obtained in this way, a weighted average calculation can be performed on the multiple chromaticity reference values ​​to obtain the chromaticity value of the pixel X″ corresponding to the first pixel X.

[0105] To reflect the difference between the present disclosure and the related art, please refer to Figure 4B . Figure 4B Figure (a) in the figure can be regarded as the chromaticity map of the image to be processed, Figure (b) is the brightness map of the image to be processed, Figure (c) is the low-frequency image obtained by downsampling Figure (a), Figure (d) is the processed image obtained by upsampling by the bilinear interpolation method in the related art, and Figure (e) is the processed image obtained by upsampling by the method of the present disclosure. Since the present disclosure uses the brightness map of the image to be processed to upsample the low-frequency image, the upsampling operation can also be called joint guided upsampling.

[0106] Compare Figure (d) and Figure (e): In Figure (e) obtained by joint guided upsampling, the pixel with a chroma value of 136 (the dark 136 in Figure (d)) has a chroma value of 132 in the neighboring pixel on the left, and the chroma difference between the two is 4; in Figure (d) obtained by upsampling by bilinear interpolation, the pixel corresponding to the above "pixel with a chroma value of 136" has a chroma value of 135, and the chroma value of the neighboring pixel on the left is 133, and the difference between the two is 2. Obviously, the processed image obtained by joint guided upsampling has a larger chroma difference between pixels at the color edge. Compared with the related art, it can increase the chroma value difference between pixels at the color edge, thereby avoiding color overflow and blurring problems at the color edge.

[0107] As can be seen from the above description, in this embodiment, the chrominance weight is determined, and the chrominance value is adjusted by the chrominance weight during the upsampling operation. From the calculation perspective of the electronic device, the above steps are completed by a formula. Of course, in addition to completing the above steps during the upsampling operation of the low-frequency image, the above steps can also be completed in a step-by-step manner.

[0108] In another embodiment, the brightness mean of any first pixel and all its first neighboring pixels in the image to be processed, as well as the second difference between the brightness value of any first pixel and the brightness mean, can be preferentially calculated; then the chromaticity weight corresponding to any first pixel can be determined based on the second difference, for example, the product of the second difference and a predefined coefficient can be used as the chromaticity weight of any first pixel. Based on the same method, the chromaticity weights of all first pixels in the image to be processed can be obtained.

[0109] Correspondingly, conventional upsampling operations can also be performed on the low-frequency image. For example, the low-frequency image can be calculated by an upsampling interpolation algorithm to obtain a weighted image that is consistent with the size of the image to be processed. Among them, the upsampling interpolation algorithm can be any type of interpolation algorithm, such as a two-dimensional interpolation algorithm and a linear interpolation algorithm, which can be used as the interpolation algorithm adopted in this embodiment. After obtaining the image to be weighted, the corresponding pixels in the image to be weighted can be weighted based on the chromaticity weights of each first pixel in the image to be processed, and the image obtained by the weighted calculation is used as the image obtained through the upsampling operation.

[0110] Still Figure 2Taking the image to be processed as an example, assuming that the chromaticity weight of the first pixel X needs to be determined, the average brightness of all first pixels in the first window area A can be calculated, and the brightness value can be obtained as: (95+94+85+83+91+93+99+108+102) / 9=94.4, then the calculated second difference corresponding to the first pixel X is 7.6, and at this time, the chromaticity weight of the first pixel X can be determined based on the second difference 7.6. The chromaticity weights of other first pixels in the image to be processed can be obtained in the same way.

[0111] On the other hand, assuming Figure 3 The images shown are: Figure 2 The chromaticity map of the image to be processed is downsampled to obtain a low-frequency image, where: Figure 3 The numerical values ​​shown in represent the chromaticity values ​​of each second pixel in the low-frequency image. In this embodiment, a conventional upsampling operation can be performed on the low-frequency image to obtain an image to be weighted whose size is consistent with that of the image to be processed. For example, the image to be weighted can be as follows: Figure 4A As shown, it needs to be stated that although Figure 4A The chromaticity values ​​of each pixel are not shown, but the chromaticity values ​​of each pixel of the weighted image obtained in this embodiment are determined. On this basis, the chromaticity values ​​of the corresponding pixels in the weighted image can be adjusted based on the chromaticity weights of each first pixel in the image to be processed. For example, the chromaticity value of pixel X" can be adjusted based on the chromaticity weight of the first pixel X.

[0112] It can be seen from the above technical solution that, on the one hand, the present disclosure performs a downsampling operation on the acquired image to be processed in the luminance-chrominance space domain to obtain a corresponding low-frequency image; on the other hand, according to the degree of brightness difference between each first pixel in the image to be processed and its respective first neighboring pixel, a chromaticity weight is assigned to each first pixel included, wherein the chromaticity weight of each first pixel is positively correlated with the corresponding degree of brightness difference. On this basis, the low-frequency image can be upsampled based on the chromaticity weight corresponding to each first pixel in the image to be processed to obtain an image with the same size as the image to be processed, and the image is merged with the image to be processed to obtain a processed image.

[0113] It should be understood that in the field of images, changes in color in an image will affect changes in brightness, and areas with larger color changes usually also have larger brightness changes. Therefore, for any pixel, the greater the degree of brightness difference with neighboring pixels, it means that the pixel is in an area with larger color changes, that is, at the color edge. The chromaticity weight of any pixel in the present disclosure is positively correlated with the corresponding chromaticity weight, which is equivalent to assigning a higher chromaticity weight to pixels in areas with large brightness changes (at the color edge). On this basis, upsampling the low-frequency image through the determined chromaticity weight is equivalent to increasing the chromaticity value of the pixel at the color edge in the low-frequency image, so that the color change at the color edge is more prominent, thereby avoiding the problem of color overflow at the color edge in the related art.

[0114] In short, the present invention identifies the color edges in the image by analyzing the brightness changes in the image, and then increases the difference between the chromaticity value of the pixels at the color edge and the chromaticity value of the pixels at the color flat area by assigning a higher chromaticity weight to the pixels at the color edge, thereby highlighting the difference between the color edge and the color flat area, and solving the problem of color overflow at the color edge in the related art.

[0115] Optionally, the present disclosure can also perform noise reduction processing on the image to be processed and / or the low-frequency image. For example, the image to be processed and / or the low-frequency image can be traversed by means of a sliding window, and the chromaticity values ​​of all pixels in any window area obtained by the sliding window are weightedly calculated to use the obtained value as the chromaticity value of the central pixel of any window area. In the process of weighted calculation, a noise reduction weight can be assigned to each pixel according to the mean value of all pixels in any window area, and the noise reduction weight of any pixel is negatively correlated with the difference between the pixel and the above mean value.

[0116] Since the denoising weight is assigned to each pixel based on the mean value of all pixels in any window area, the denoising weight of the central pixel may not be the largest. When denoising isolated color noise in a color-flat area, the chromaticity difference between the central pixel corresponding to the color noise and the neighboring pixels is quite different, resulting in the obtained chromaticity mean having a small difference with the chromaticity value of the neighboring pixels, but a large difference with the chromaticity value of the central pixel, which will minimize the denoising weight of the central pixel. It can be seen that through the above method, the difference between the chromaticity value of the pixel corresponding to the color noise and the chromaticity value of its neighboring pixels can be effectively reduced, thereby effectively removing the isolated color noise in the color-flat area.

[0117] Corresponding to the aforementioned embodiment of the image processing method, the present disclosure also provides an embodiment of an image processing device.

[0118] Figure 5is a block diagram of an image processing device shown in an exemplary embodiment of the present disclosure. Figure 5 The device includes an acquisition unit 501, a determination unit 502 and a synthesis unit 503.

[0119] An acquisition unit 501 acquires an image to be processed in a luminance-chrominance space domain, and performs a downsampling operation on the image to be processed to obtain a low-frequency image of the image to be processed;

[0120] A determining unit 502 determines the chromaticity weights of the first pixels in the image to be processed according to the brightness difference between the first pixels and the first neighboring pixels, wherein the chromaticity weight of each of the first pixels is positively correlated with the corresponding brightness difference.

[0121] The synthesis unit 503 performs an up-sampling operation on the low-frequency image based on the chrominance weights corresponding to the first pixels, and synthesizes the image obtained by the up-sampling operation with the image to be processed to obtain a processed image.

[0122] Optionally, the determining unit 502 is further configured as:

[0123] Calculating a brightness difference between any first pixel and any first neighboring pixel thereof, and calculating a weight parameter corresponding to any first neighboring pixel based on the brightness difference;

[0124] The chrominance weight of any first pixel is determined based on the calculated weight parameters of all first neighborhood pixels of any first pixel.

[0125] Optionally, the synthesis unit 503 is further configured as:

[0126] Performing the following operations on all first pixels in the image to be processed: acquiring a second pixel corresponding to any first pixel in the low-frequency image, and determining a second neighboring pixel corresponding to any first neighboring pixel among all second neighboring pixels of the second pixel;

[0127] Calculating the distance between the second neighborhood pixel and the second pixel, and determining a chromaticity reference value corresponding to any first neighborhood pixel based on the distance and a weight parameter of any first neighborhood pixel;

[0128] Performing weighted average calculation on the determined multiple chromaticity reference values ​​of all first neighborhood pixels corresponding to any first pixel to obtain a chromaticity value of a target pixel corresponding to any first pixel;

[0129] After obtaining the chromaticity values ​​of all target pixels corresponding to all first pixels in the image to be processed, an image obtained through the upsampling operation is generated based on the chromaticity values ​​of all target pixels.

[0130] Optionally, the determining unit 502 is further configured as:

[0131] Calculate a brightness mean value of any first pixel and all first neighboring pixels thereof, and a second difference value between the brightness value of any first pixel and the brightness mean value;

[0132] The product of the second difference and the predefined coefficient is used as the chrominance weight of any one of the first pixels.

[0133] Optionally, the synthesis unit 503 is further configured as:

[0134] Calculating the low-frequency image by an upsampling interpolation algorithm to obtain an image to be weighted that is consistent with the size of the image to be processed;

[0135] Based on the chromaticity weights of the first pixels in the image to be processed, weighted calculation is performed on the chromaticity values ​​of the corresponding pixels in the image to be weighted;

[0136] The image obtained by weighted calculation is used as the image obtained through the upsampling operation.

[0137] like Figure 6 As shown, Figure 6 is a block diagram of another image processing device shown in an exemplary embodiment of the present disclosure. Figure 5 Based on the illustrated embodiment, it further includes: a noise reduction unit 504.

[0138] Optionally, also include:

[0139] The noise reduction unit 504 is configured to traverse the image to be processed by means of a sliding window, and perform weighted calculation on the chromaticity values ​​of all first pixels in any first window area obtained by means of the sliding window to obtain a first target value corresponding to any first window area; use the first target value as the chromaticity value of the center pixel of any first window area; and / or traverse the low-frequency image by means of a sliding window, and perform weighted calculation on the chromaticity values ​​of all second pixels in any second window area obtained by means of the sliding window to obtain a second target value corresponding to any second window area; use the second target value as the chromaticity value of the center pixel of any second window area.

[0140] Optionally, the noise reduction unit 504 is further configured as:

[0141] Calculating the chromaticity mean of all first pixels in any of the first window areas, and determining the first noise reduction weight of each first pixel according to the first difference between the chromaticity value of each first pixel in the any of the window areas and the chromaticity mean; the noise reduction weight of any first pixel in the any of the first window areas is negatively correlated with the first difference corresponding to the any of the first pixels; based on the noise reduction weight of each first pixel in the any of the first window areas, performing weighted average calculation on the chromaticity values ​​of all the first pixels in the any of the first window areas to obtain a first target value corresponding to the any of the first window areas; and / or,

[0142] Calculate the chromaticity mean of all second pixels within any second window area, and determine the second noise reduction weight of each second pixel according to the second difference between the chromaticity value of each second pixel in any second window area and the chromaticity mean; the noise reduction weight of any second pixel in any second window area is negatively correlated with the second difference corresponding to the any second pixel; based on the noise reduction weight of each second pixel in any second window area, perform weighted average calculation on the chromaticity values ​​of all second pixels in any second window area to obtain the second target value corresponding to any second window area.

[0143] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0144] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0145] Accordingly, the present disclosure also provides an image processing device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement an image processing method as described in any of the above embodiments, for example, the method may include: obtaining an image to be processed in a luminance-chrominance space domain, and performing a downsampling operation on the image to be processed to obtain a low-frequency image of the image to be processed; determining the chromaticity weights of each first pixel in the image to be processed according to the degree of brightness difference between each first pixel and each first neighboring pixel, wherein the chromaticity weight of each first pixel in the image to be processed is positively correlated with the corresponding degree of brightness difference; performing an upsampling operation on the low-frequency image based on the chromaticity weights corresponding to the each first pixel, and synthesizing the image obtained by the upsampling operation with the image to be processed to obtain a processed image.

[0146] Accordingly, the present disclosure also provides an electronic device, which includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for implementing the image processing method as described in any of the above embodiments. For example, the method may include: acquiring an image to be processed in a luminance-chrominance space domain, and performing a downsampling operation on the image to be processed to obtain a low-frequency image of the image to be processed; determining the chromaticity weights of each first pixel in the image to be processed according to the degree of brightness difference between each first pixel and its respective first neighboring pixel, wherein the chromaticity weight of each first pixel in the image to be processed is positively correlated with the corresponding degree of brightness difference; performing an upsampling operation on the low-frequency image based on the chromaticity weights corresponding to the each first pixel, and synthesizing the image obtained by the upsampling operation with the image to be processed to obtain a processed image. Figure 7 1 is a block diagram of an apparatus 700 for implementing a process scheduling method according to an exemplary embodiment. For example, the apparatus 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0147] Reference Figure 7 , the device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0148] The processing component 702 generally controls the overall operation of the device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0149] The memory 704 is configured to store various types of data to support operations on the device 700. Examples of such data include instructions for any application or method operating on the device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0150] The power supply component 706 provides power to the various components of the device 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 700.

[0151] The multimedia component 708 includes a screen that provides an output interface between the device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0152] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the device 700 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0153] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0154] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the device 700. For example, the sensor assembly 714 can detect the open / closed state of the device 700, the relative positioning of components, such as the display and keypad of the device 700, the sensor assembly 714 can also detect the position change of the device 700 or a component of the device 700, the presence or absence of user contact with the device 700, the orientation or acceleration / deceleration of the device 700, and the temperature change of the device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0155] The communication component 716 is configured to facilitate wired or wireless communication between the device 700 and other devices. The device 700 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (New Radio) or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0156] In an exemplary embodiment, the apparatus 700 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0157] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the instructions can be executed by the processor 720 of the device 700 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0158] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0159] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0160] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. An image processing method, characterized in that: include: Acquire an image to be processed in a luminance-chrominance space domain, and perform a downsampling operation on the image to be processed to obtain a low-frequency image of the image to be processed; Determining the chromaticity weights of the first pixels according to the brightness difference between the first pixels and the first neighboring pixels in the image to be processed, wherein the chromaticity weight of each of the first pixels is positively correlated with the corresponding brightness difference; An upsampling operation is performed on the low-frequency image based on the chrominance weights corresponding to the first pixels, and an image obtained by the upsampling operation is synthesized with the image to be processed to obtain a processed image.

2. The method according to claim 1, characterized in that Before performing an upsampling operation on the low-frequency image, the method further includes: Traversing the image to be processed in a sliding window manner, and performing weighted calculation on the chromaticity values ​​of all first pixels in any first window area obtained through the sliding window to obtain a first target value corresponding to any first window area; using the first target value as the chromaticity value of the central pixel of any first window area; and / or, The low-frequency image is traversed by means of a sliding window, and the chromaticity values ​​of all second pixels in any second window area obtained by the sliding window are weightedly calculated to obtain a second target value corresponding to any second window area; and the second target value is used as the chromaticity value of the central pixel of any second window area.

3. The method according to claim 2, characterized in that The step of performing weighted calculation on the chromaticity values ​​of all first pixels in any first window area acquired through the sliding window to obtain a first target value corresponding to any first window area includes: Calculate the chromaticity mean of all first pixels in any first window area, and determine the first noise reduction weight of each first pixel according to the first difference between the chromaticity value of each first pixel in any first window area and the chromaticity mean; the noise reduction weight of any first pixel in any first window area is negatively correlated with the first difference corresponding to the any first pixel; based on the noise reduction weight of each first pixel in any first window area, perform weighted average calculation on the chromaticity values ​​of all first pixels in any first window area to obtain a first target value corresponding to any first window area; The step of performing weighted calculation on the chromaticity values ​​of all second pixels in any second window area acquired through the sliding window to obtain a second target value corresponding to any second window area includes: Calculate the chromaticity mean of all second pixels within any second window area, and determine the second noise reduction weight of each second pixel according to the second difference between the chromaticity value of each second pixel in any second window area and the chromaticity mean; the noise reduction weight of any second pixel in any second window area is negatively correlated with the second difference corresponding to the any second pixel; based on the noise reduction weight of each second pixel in any second window area, perform weighted average calculation on the chromaticity values ​​of all second pixels in any second window area to obtain the second target value corresponding to any second window area.

4. The method according to claim 1, characterized in that: Determining the chromaticity weight of each first pixel according to the brightness difference between each first pixel and its first neighboring pixel in the image to be processed, including: Calculating a brightness difference between any first pixel and any first neighboring pixel thereof, and calculating a weight parameter corresponding to any first neighboring pixel based on the brightness difference; The chrominance weight of any first pixel is determined based on the calculated weight parameters of all first neighborhood pixels of any first pixel.

5. The method according to claim 4, characterized in that The up-sampling operation on the low-frequency image based on the chroma weights corresponding to the first pixels includes: Performing the following operations on all first pixels in the image to be processed: acquiring a second pixel corresponding to any first pixel in the low-frequency image, and determining a second neighboring pixel corresponding to any first neighboring pixel among all second neighboring pixels of the second pixel; Calculating the distance between the second neighborhood pixel and the second pixel, and determining a chromaticity reference value corresponding to any first neighborhood pixel based on the distance and a weight parameter of any first neighborhood pixel; Performing weighted average calculation on the determined multiple chromaticity reference values ​​of all first neighborhood pixels corresponding to any first pixel to obtain a chromaticity value of a target pixel corresponding to any first pixel; After obtaining the chromaticity values ​​of all target pixels corresponding to all first pixels in the image to be processed, an image obtained through the upsampling operation is generated based on the chromaticity values ​​of all target pixels.

6. The method according to claim 1, characterized in that Determining the chromaticity weight of each first pixel according to the brightness difference between each first pixel and its first neighboring pixel in the image to be processed, including: Calculate a brightness mean value of any first pixel and all first neighboring pixels thereof, and a second difference value between the brightness value of any first pixel and the brightness mean value; The product of the second difference and the predefined coefficient is used as the chrominance weight of any one of the first pixels.

7. The method according to claim 1, characterized in that The up-sampling operation on the low-frequency image based on the chroma weights corresponding to the first pixels includes: Calculating the low-frequency image by an upsampling interpolation algorithm to obtain an image to be weighted that is consistent with the size of the image to be processed; Based on the chromaticity weights of the first pixels in the image to be processed, weighted calculation is performed on the chromaticity values ​​of the corresponding pixels in the image to be weighted; The image obtained by weighted calculation is used as the image obtained through the upsampling operation.

8. An image processing device, characterized in that: include: An acquisition unit, which acquires an image to be processed in a luminance-chrominance space domain, and performs a downsampling operation on the image to be processed to obtain a low-frequency image of the image to be processed; A determination unit, which determines the chromaticity weights of the first pixels in the image to be processed according to the brightness difference between the first pixels and the first neighboring pixels, respectively, wherein the chromaticity weight of each of the first pixels is positively correlated with the corresponding brightness difference; A synthesis unit performs an up-sampling operation on the low-frequency image based on the chroma weights corresponding to the first pixels, and synthesizes the image obtained by the up-sampling operation with the image to be processed to obtain a processed image.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 7 by running the executable instructions.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images