Image content enhancement method and device based on demosaicing and electronic equipment

Abstract

The invention provides a demosaicing-based image content enhancement method and apparatus, and an electronic device. The method comprises the steps of calculating a weight factor variance of each pixel point in a RAW image; calculating spatial similarity vectors of a cache window where the current pixel point is located and a plurality of adjacent windows, and further adjusting a weight factor of the current pixel point; and carrying out demosaicing processing by using the adjusted weight factor. Whether the region where the current pixel point is located has the texture or not is judged through the weight factor variance and the spatial similarity vector, the weight factor of the texture region is enhanced, and the weight factor of the flat region is weakened, so that enhancement of texture lines and slight denoising of the flat region can be realized in the demosaicing process through less calculation, and the demosaicing accuracy is improved. The demosaiced image is clear in line, less in noise and better in image quality, and the problem of how to improve the demosaiced image processing effect while saving hardware computing resources is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device and electronic equipment for enhancing image content based on de-mosaicing. Background Art

[0002] Current imaging devices, such as cameras, usually use a single image sensor with a filter for their imaging system due to cost considerations. After the image sensor captures the image, a RAW image arranged in a Bayer format is usually obtained. If you want to obtain a clear and good-looking RGB color image, you must use the ISP system to process the captured image. ISP (Image Signal Processing) usually includes linear error correction, denoising, bad pixel removal, white balance, color correction, tone mapping, demosaicing, etc. Among them, demosaicing is the key step in converting RAW images to RGB images.

[0003] However, existing demosaicing algorithms usually ignore the edge structure of the image and the correlation of multiple channels, resulting in loss of image details and color confusion; and complex demosaicing algorithms, although they can incorporate more factors to obtain better image processing results, are computationally complex, resulting in a computational load that exceeds the hardware's carrying capacity. Summary of the invention

[0004] The object of the present invention is to provide a method, device and electronic device for enhancing image content based on demosaicing, so as to solve the problem of how to improve the image processing effect of demosaicing while saving hardware computing resources.

[0005] In order to solve the above technical problems, the present invention provides an image content enhancement method based on demosaicing, comprising: Get RAW images; Calculate the variance of the weight factor of each pixel in the RAW image; Calculate the spatial similarity vectors of the cache window where the current pixel is located and multiple adjacent windows; According to the spatial similarity vector and the variance of the weight factor, the weight factor of the current pixel is adjusted; De-mosaicing is performed using the adjusted weight factors to output an RGB image.

[0006] Optionally, in the image content enhancement method based on demosaicing, the method for calculating the variance of the weight factor of each pixel in the RAW image includes: Calculate the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of each pixel in the RAW image; Calculate the horizontal weight factor, vertical weight factor, diagonal weight factor and anti-diagonal weight factor of the pixel point using the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of the pixel point; The variances of the horizontal weight factor, the vertical weight factor, the diagonal weight factor, and the anti-diagonal weight factor of the pixel are calculated to obtain the weight factor variance.

[0007] Optionally, in the image content enhancement method based on demosaicing, the method of calculating the spatial similarity vectors of the cache window where the current pixel point is located and multiple adjacent windows includes: Extracting a cache window and eight neighboring windows adjacent to the cache window; Calculate the brightness difference between each adjacent window and the cache window; Using the brightness difference, calculate the spatial similarity between each neighboring window and the cache window; 8 spatial similarities are counted to obtain a spatial similarity vector.

[0008] Optionally, in the de-mosaicing based image content enhancement method, in the extracted cache window, the current pixel is located in the middle row and middle column of the cache window; each of the extracted neighboring windows overlaps with the cache window in at least one pixel; and the size of the neighboring window is consistent with the size of the cache window.

[0009] Optionally, in the demosaicing-based image content enhancement method, the method of calculating the brightness difference between each adjacent window and the cache window includes: For each adjacent window, calculate the absolute value of the pixel value difference between each pixel point therein and the pixel point at the same position in the cache window; The absolute values ​​of all calculated pixel value differences are summed to obtain the brightness difference between the neighboring window and the cache window.

[0010] Optionally, in the image content enhancement method based on demosaicing, the spatial similarity between each adjacent window and the cache window is calculated by using the brightness difference, which is expressed as:

[0011] in, represents the spatial similarity between the i-th neighboring window and the cache window, represents the brightness difference between the i-th neighboring window and the cache window, i=0,1,2,…,7; and This is an adjustable parameter with a value greater than 0.

[0012] Optionally, in the image content enhancement method based on demosaicing, the method for adjusting the weight factor of the current pixel point according to the spatial similarity vector and the weight factor variance includes: According to the spatial similarity vector and the variance of the weight factor, it is determined whether there is texture in the area where the current pixel is located; If it is determined that there is texture in the area where the current pixel is located, the weight factor of the current pixel is directionally enhanced; If it is determined that there is no texture in the area where the current pixel is located, the weight factor of the current pixel is weakened in a certain direction.

[0013] Optionally, in the image content enhancement method based on demosaicing, the method of judging whether there is texture in the area where the current pixel point is located according to the spatial similarity vector and the weight factor variance includes: If the variance of the weight factor is greater than the first variance threshold, and at least one spatial similarity in the spatial similarity vector is greater than the similarity threshold, it is determined that there is texture in the area where the current pixel is located; If the variance of the weight factor is less than the second variance threshold, it is determined that there is no texture in the area where the current pixel point is located.

[0014] Optionally, in the image content enhancement method based on de-mosaicing, the method of directional strengthening the weight factor of the current pixel point includes: The maximum weight factor of the current pixel is enlarged by a first multiple, and the minimum weight factor is reduced by a second multiple; The method for directional weakening of the weight factor of the current pixel point comprises: The maximum weight factor of the current pixel is reduced by a third multiple, and the minimum weight factor is increased by a fourth multiple.

[0015] Optionally, in the image content enhancement method based on demosaicing, the method for adjusting the weight factor of the current pixel point according to the spatial similarity vector and the weight factor variance further includes: If the weight factor variance is less than or equal to the first variance threshold and greater than or equal to the second variance threshold, the weight factor of the current pixel point is kept unchanged.

[0016] In order to solve the above technical problems, the present invention further provides an image content enhancement device based on demosaicing, which is used to implement the image content enhancement method based on demosaicing as described in any one of the above items, and the image content enhancement device based on demosaicing includes: An image acquisition module, used for acquiring RAW images; An image processing module, used to calculate the variance of the weight factor of each pixel in the RAW image acquired by the image acquisition module; also used to calculate the spatial similarity vector of the cache window where the current pixel is located and multiple adjacent windows; also used to adjust the weight factor of the current pixel according to the spatial similarity vector and the variance of the weight factor; and also used to perform de-mosaic processing using the adjusted weight factor to obtain an RGB image; The image output module is used to output the RGB image obtained after being processed by the image processing module.

[0017] In order to solve the above technical problems, the present invention also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the de-mosaicing-based image content enhancement method as described in any one of the above items is executed.

[0018] The image content enhancement method, device and electronic device based on demosaicing provided by the present invention include: obtaining a RAW image; calculating the variance of the weight factor of each pixel in the RAW image; calculating the spatial similarity vector of the cache window where the current pixel is located and multiple adjacent windows; adjusting the weight factor of the current pixel according to the spatial similarity vector and the variance of the weight factor; performing demosaicing processing using the adjusted weight factor to output an RGB image. The weight factor variance and the spatial similarity vector are used to determine whether there is a texture in the area where the current pixel is located, and the weight factor of the current pixel in the texture area is strengthened, and the weight factor of the current pixel in the flat area is weakened, so that the texture lines can be enhanced and the flat area can be slightly denoised in the demosaicing process through less calculation, so that the image output after demosaicing has clear lines, less noise, and better image quality, solving the problem of how to improve the image processing effect of demosaicing while saving hardware computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of the method for enhancing image content based on demosaicing provided in this embodiment; Figure 2 A schematic diagram of the image pixel distribution structure of the Bayer arrangement provided in this embodiment; Figure 3 A schematic diagram of a method for extracting a cache window and an adjacent window provided in this embodiment; Figure 4 This is a structural block diagram of the image content enhancement device based on demosaicing provided in this embodiment. DETAILED DESCRIPTION

[0020] The following is a further detailed description of the image content enhancement method, device and electronic device based on de-mosaicing proposed in the present invention in conjunction with the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the accompanying drawings is often a part of the actual structure. In particular, the emphasis that each drawing needs to show is different, and sometimes different proportions are used.

[0021] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present invention are used to distinguish similar objects in order to describe the embodiments of the present invention, rather than to describe a specific order or sequence. It should be understood that the structures used in this way can be interchanged under appropriate circumstances. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] Among the existing common de-mosaicing algorithms, one of the important steps is to calculate the weight factor of the current pixel in different directions (generally including horizontal direction, vertical direction, diagonal direction, and anti-diagonal direction), and use the weight factor to guide the interpolation calculation to improve the accuracy of the interpolation calculation, thereby improving the image de-mosaicing effect.

[0023] Based on this, this embodiment provides an image content enhancement method based on demosaicing, such as Figure 1 As shown, including: S1, obtaining a RAW image; S2, calculate the variance of the weight factor of each pixel in the RAW image; S3, calculating the spatial similarity vectors of the cache window where the current pixel is located and multiple neighboring windows; S4, adjusting the weight factor of the current pixel according to the spatial similarity vector and the weight factor variance; S5, performing a demosaicing process using the adjusted weight factor to output an RGB image.

[0024] The demosaicing-based image content enhancement method provided in this embodiment calculates the weight factor variance and the spatial similarity vector, and uses the weight factor variance and the spatial similarity vector to determine whether there is a texture in the area where the current pixel is located, and strengthens the weight factor of the current pixel in the texture area, and weakens the weight factor of the current pixel in the flat area. Therefore, with less calculation, the texture lines can be enhanced and the flat area can be slightly denoised during the demosaicing process, so that the output image after demosaicing has clear lines, less noise, and better image quality, which solves the problem of how to improve the image processing effect of demosaicing while saving hardware computing resources.

[0025] Specifically, in this embodiment, in step S1, the method for obtaining the Raw image may be to obtain the original image from various photographic and video equipment.

[0026] Raw images are the original image files obtained from the image sensors of devices such as digital cameras, scanners, or film scanners. Raw images are not processed, so they have internal colors with a wide color gamut and can be adjusted accurately. In addition, in Raw images, the arrangement of RGB pixels (Bayer colors) has certain regularity, such as Figure 2 As shown, the Gs in each row / column are arranged at intervals, and the Gs in adjacent rows / columns are arranged in an alternating manner; adjacent Gs in the same row / column are all separated by R or B, and the R or B in adjacent rows / columns is different. For example, when all the Gs in the i-th row are separated by R, all the Gs in the i-1th row and the i+1th row are separated by B.

[0027] Further, in this embodiment, in step S2, the method of calculating the variance of the weight factor of each pixel in the RAW image includes: S21, calculating the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of each pixel in the RAW image.

[0028] Specifically, in this embodiment, a data window can be extracted, and then the pixels in the data window are used to calculate the four directional gradients of the current pixel. For example, a 5×5 data window is extracted, where the current pixel is located in the 3rd row and the 3rd column, and the pixel value of each pixel in the data window is recorded as D(i,j), where i represents the row number of the pixel in the data window, and j represents the column number of the pixel in the data window, i=0,1,2,3,4, j=0,1,2,3,4, then the pixel value of the current pixel can be recorded as D(2,2); then, the horizontal gradient gradh of the current pixel is calculated using the pixel values ​​of the pixels in the same row as the current pixel in the data window and the pixel value of the current pixel, which can be expressed as:

[0029] Similarly, the vertical gradient gradv of the current pixel can be obtained, which can be expressed as:

[0030] And the diagonal gradient gradd of the current pixel can be expressed as:

[0031] And the anti-diagonal gradient gradad of the current pixel can be expressed as:

[0032] Of course, in other embodiments, other parameters may be used to calculate the gradients of the pixels in four directions, such as using the G channel value for calculation, or using the color difference value for calculation; other calculation formulas may also be used to calculate the gradients of the pixels in four directions, such as changing the size of the cache window, using more pixels or fewer pixels for gradient calculation, or changing the interval between the two pixels for difference calculation, using pixels with a wider interval or closer pixels for gradient calculation, or changing the weight parameters assigned during gradient calculation, etc. This application does not impose any restrictions on this.

[0033] It should be noted that this embodiment only takes the above-mentioned gradient calculation method as an example to illustrate a possible gradient calculation method. Those skilled in the art can know other gradient calculation methods based on the prior art. Other gradient calculation methods without violating the main purpose of this application should also belong to the protection scope of this application.

[0034] Taking into account that during the ISP image processing, some image processing algorithms will calculate the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of each pixel in the RAW image, therefore, in practical applications, in step S1, when acquiring the RAW image, the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of each pixel in the RAW image can be obtained at the same time, thereby reducing repeated calculations and improving image processing efficiency.

[0035] S22, using the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of the pixel point, calculate the horizontal weight factor, vertical weight factor, diagonal weight factor and anti-diagonal weight factor of the pixel point.

[0036] Specifically, in this embodiment, the inverse of the gradient is used as the weight factor in the corresponding direction, that is, the horizontal direction weight factor Wh can be expressed as: Wh=1 / gradh; the vertical direction weight factor Wv can be expressed as: Wv=1 / gradv; the diagonal direction weight factor Wd can be expressed as: Wd=1 / gradd; the anti-diagonal reverse weight factor Wa can be expressed as: Wa=1 / gradad.

[0037] Of course, in other embodiments, the weight factor may also be calculated in other ways according to the gradient, and this application does not limit this. Other weight factor calculation methods that do not violate the purpose of this application should also fall within the scope of protection of this application.

[0038] S23, calculating the variance of the horizontal weight factor, the vertical weight factor, the diagonal weight factor and the anti-diagonal weight factor of the pixel point to obtain the weight factor variance.

[0039] Specifically, in this embodiment, first, the horizontal weight factor, vertical weight factor, diagonal weight factor and anti-diagonal weight factor of the pixel point are used to calculate the weight factor mean E, which can be expressed as:

[0040] Then, the weight factor variance Var is calculated, which can be expressed as:

[0041] The size of the variance of the weight factor can reflect whether there are obvious texture lines in the local area where the pixel is located. When the area is flat and has no texture, the gradient values ​​in the four directions of the pixel are very small, the gradients are relatively close in size, and the values ​​of the four weight factors are also relatively close. At this time, the variance of the weight factor is very small; when there are obvious texture lines in the area, the gradient values ​​of the pixel in some directions are large, while the gradient values ​​in other directions are very small. The difference between the gradient values ​​is large, and the values ​​of the four weight factors are also large. At this time, the variance of the weight factor is large. In other words, if the variance of the weight factor is small, it means that the current pixel is likely to be in a flat area; if the variance of the weight factor is large, it means that the current pixel is likely to be in a textured area.

[0042] Furthermore, in this embodiment, in step S3, the method of calculating the spatial similarity vectors of the cache window where the current pixel point is located and the plurality of adjacent windows includes: S31, extracting a cache window and eight neighboring windows adjacent to the cache window.

[0043] Specifically, in this embodiment, in the extracted cache window, the current pixel point is located in the middle row and middle column of the cache window to ensure that the current pixel point has equal pixels around it in the cache window, thereby improving the accuracy of the calculation. Also, each of the extracted neighboring windows and the cache window has at least one pixel point overlapped to ensure that the extracted neighboring window and the cache window are continuous on the image, thereby improving the matching degree between the calculation result and the actual situation. Also, the size of the neighboring window is consistent with the size of the cache window to ensure that the pixel points in the neighboring window can correspond to the pixel points in the cache window, so that the brightness difference between the two can be correctly calculated.

[0044] In a specific embodiment, if Figure 3 As shown in the figure, the size of the extracted cache window and neighboring window are both 3×3, and the center pixel of the cache window (solid box in the figure) is the current pixel (coordinate position is (3,3)). Taking the neighboring window in the upper left corner (dashed box in the figure) as an example, the coordinate position of its center pixel is (1,1), and the number of pixels between the center pixel of the neighboring window and the current pixel (window sliding step) is 2. The pixel point that overlaps with the cache window is the pixel point with coordinate position (2,2). With the same window sliding step and window size, the other 7 neighboring windows can be obtained (located directly above, upper right, left, right, lower left, directly below, and lower right of the cache window, respectively). At this time, all cache windows and neighboring windows constitute a large cache window of size 7×7.

[0045] Of course, in other embodiments, the sizes of the cache window and the neighboring window can also be other sizes, such as 5×5, 7×7, etc.; the number of pixels between the center pixel of the neighboring window and the current pixel (window sliding step) can also be other numbers, such as 1, 3, etc.; this application does not limit this. In practical applications, the window sliding step should match the size of the window to ensure that at least one pixel in the neighboring window and the cache window overlaps.

[0046] S32, calculating the brightness difference between each adjacent window and the cache window.

[0047] Specifically, in this embodiment, for each adjacent window, first, the absolute value of the pixel value difference between each pixel point therein and the pixel point at the same position in the cache window is calculated. Figure 3 Taking the window structure shown in the figure as an example, in the neighboring window in the upper left corner, the pixel with coordinates (0,0) corresponds to the pixel with coordinates (2,2) in the cache window. The absolute value of the difference between the two pixel values ​​can be expressed as , where Dw(i,j) represents the pixel value of the pixel point with coordinate position (i,j) in the large cache window; similarly, the absolute value of the difference between the pixel values ​​of the 9 pixels in the neighboring window and the corresponding pixel points in the cache window can be obtained.

[0048] Then, the absolute values ​​of all calculated pixel value differences are summed to obtain the brightness difference between the neighboring window and the cache window. Figure 3 Taking the window structure shown in the figure as an example, the brightness difference between the neighboring window in the upper left corner and the cache window can be expressed as:

[0049] Similarly, the brightness differences between the other 7 neighboring windows and the cache window can be calculated and recorded as , ,……, .

[0050] S33, using the brightness difference, calculate the spatial similarity between each adjacent window and the cache window.

[0051] Specifically, in this embodiment, the spatial similarity between each neighboring window and the cache window is calculated, which can be expressed as:

[0052] in, represents the spatial similarity between the i-th neighboring window and the cache window, represents the brightness difference between the i-th neighboring window and the cache window, i=0,1,2,…,7; and It is an adjustable parameter with a value greater than 0, which is used to control the shape of the curve mapping from brightness difference to spatial similarity. Its specific value needs to be determined according to actual needs.

[0053] S34, counting the spatial similarities corresponding to the eight adjacent windows to obtain a spatial similarity vector.

[0054] When the brightness difference is small and the calculated spatial similarity is large, it indicates that the textures of the image in the neighboring window and the image in the cache window are close, or continuous on the image, indicating that the current pixel may be in an area with obvious textures; and when the brightness difference is small and the calculated spatial similarity is large, it may also indicate that the neighboring window and the cache window are both in a flat area of ​​the image. Since the same spatial similarity result reflects two different image textures, it is necessary to combine the weight factor variance to determine whether there is an accurate result of texture in the area where the current pixel is located.

[0055] Furthermore, in this embodiment, S4, the method of adjusting the weight factor of the current pixel point according to the spatial similarity vector and the weight factor variance includes: According to the spatial similarity vector and the variance of the weight factor, it is determined whether there is texture in the area where the current pixel is located. Specifically, in this embodiment, if the variance of the weight factor is greater than the first variance threshold, and at least one spatial similarity in the spatial similarity vector is greater than the similarity threshold, it is determined that there is texture in the area where the current pixel is located; if the variance of the weight factor is less than the second variance threshold, it is determined that there is no texture in the area where the current pixel is located.

[0056] If it is determined that there is texture in the area where the current pixel is located, the weight factor of the current pixel is directionally enhanced. Specifically, in this embodiment, the maximum weight factor of the current pixel is magnified by a first multiple, and the minimum weight factor is reduced by a second multiple.

[0057] If it is determined that there is no texture in the area where the current pixel is located, the weight factor of the current pixel is directionally weakened. Specifically, in this embodiment, the maximum weight factor of the current pixel is reduced by a third multiple, and the minimum weight factor is increased by a fourth multiple.

[0058] And, in this embodiment, if the variance of the weight factor is less than or equal to the first variance threshold and greater than or equal to the second variance threshold, it is determined that the area where the current pixel is located may have cluttered or weak texture. At this time, the weight factor of the current pixel is kept unchanged.

[0059] In practical applications, the first variance threshold, the second variance threshold and the similarity threshold need to be reasonably set according to actual needs. In addition, the first multiple, the second multiple, the third multiple and the fourth multiple are used to control the degree of directional strengthening and weakening, and also need to be reasonably set according to actual needs. Usually, the values ​​of the first multiple, the second multiple, the third multiple and the fourth multiple are greater than 1, and the typical value of the first multiple is 4, the typical value of the second multiple is 2, the typical value of the third multiple is 2, and the typical value of the fourth multiple is 2.

[0060] Furthermore, in this embodiment, in step S5, de-mosaicing is performed using the adjusted weight factors to output an RGB image.

[0061] The method of performing de-mosaic processing using weight factors to output an RGB image is well known to those skilled in the art and will not be described in detail in this application.

[0062] The image content enhancement method based on demosaicing provided in this embodiment can accurately determine whether there is texture in the area where the current pixel is located through the weight factor variance and the spatial similarity vector, and then directionally strengthen the weight factor of the texture area, so that more texture directions are referenced during weighted fusion, and interference in other directions is removed, so that the texture area of ​​the demosaiced image is clearer and obvious, and more texture details are retained. At the same time, the weight factor of the flat area is directionally weakened, so that the image after demosaicing has less noise and better image quality, which solves the problem of how to improve the image processing effect of demosaicing while saving hardware computing resources.

[0063] This embodiment also provides a de-mosaicing-based image content enhancement device, which is used to implement the de-mosaicing-based image content enhancement method as described above. Figure 4 As shown, the image content enhancement device based on demosaicing includes: An image acquisition module, used for acquiring RAW images; An image processing module, used to calculate the variance of the weight factor of each pixel in the RAW image acquired by the image acquisition module; also used to calculate the spatial similarity vector of the cache window where the current pixel is located and multiple adjacent windows; also used to adjust the weight factor of the current pixel according to the spatial similarity vector and the variance of the weight factor; and also used to perform de-mosaic processing using the adjusted weight factor to obtain an RGB image; The image output module is used to output the RGB image obtained after being processed by the image processing module.

[0064] Taking into account that during the ISP image processing, some image processing algorithms will calculate the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of each pixel in the RAW image, therefore, in practical applications, when the image acquisition module acquires the RAW image, the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of each pixel in the RAW image can also be obtained at the same time, thereby reducing repeated calculations and improving image processing efficiency.

[0065] The image content enhancement device based on demosaicing provided in this embodiment calculates the weight factor variance and the spatial similarity vector through the image processing module, and then determines whether there is a texture in the area where the current pixel point is located according to the weight factor variance and the spatial similarity vector, and strengthens the weight factor of the current pixel point in the texture area, and weakens the weight factor of the current pixel point in the flat area, so that the texture lines can be enhanced and the flat area can be slightly denoised in the demosaicing process with less calculation, so that the image output after demosaicing has clear lines, less noise, and better image quality, which solves the problem of how to improve the image processing effect of demosaicing while saving hardware computing resources.

[0066] Furthermore, this embodiment further provides an electronic device, comprising a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the image content enhancement method based on de-mosaicing as described above is executed.

[0067] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. In addition, the different parts between the various embodiments can also be used in combination with each other, and the present invention is not limited to this.

[0068] The image content enhancement method, device and electronic device based on de-mosaicing provided in this embodiment include: obtaining a RAW image; calculating the variance of the weight factor of each pixel in the RAW image; calculating the spatial similarity vector of the cache window where the current pixel is located and multiple adjacent windows; adjusting the weight factor of the current pixel according to the spatial similarity vector and the variance of the weight factor; performing de-mosaicing using the adjusted weight factor to output an RGB image. The weight factor variance and the spatial similarity vector are used to determine whether there is a texture in the area where the current pixel is located, and the weight factor of the current pixel in the texture area is strengthened, and the weight factor of the current pixel in the flat area is weakened, so that the texture lines can be enhanced and the flat area can be slightly denoised in the de-mosaicing process through less calculation, so that the image output after de-mosaicing has clear lines, less noise, and better image quality, which solves the problem of how to improve the image processing effect of de-mosaicing while saving hardware computing resources.

[0069] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A method for enhancing image content based on demosaicing, characterized in that: include: Get RAW images; Calculate the variance of the weight factor of each pixel in the RAW image; Calculate the spatial similarity vectors of the cache window where the current pixel is located and multiple adjacent windows; According to the spatial similarity vector and the variance of the weight factor, the weight factor of the current pixel is adjusted; De-mosaicing is performed using the adjusted weight factors to output an RGB image.

2. The method for enhancing image content based on demosaicing according to claim 1, characterized in that: The method for calculating the variance of the weight factor of each pixel in the RAW image comprises: Calculate the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of each pixel in the RAW image; Calculate the horizontal weight factor, vertical weight factor, diagonal weight factor and anti-diagonal weight factor of the pixel point using the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of the pixel point; The variances of the horizontal weight factor, the vertical weight factor, the diagonal weight factor, and the anti-diagonal weight factor of the pixel are calculated to obtain the weight factor variance.

3. The method for enhancing image content based on demosaicing according to claim 1, characterized in that: The method for calculating the spatial similarity vectors of the cache window where the current pixel point is located and multiple adjacent windows includes: Extracting a cache window and eight neighboring windows adjacent to the cache window; Calculate the brightness difference between each adjacent window and the cache window; Using the brightness difference, calculate the spatial similarity between each neighboring window and the cache window; 8 spatial similarities are counted to obtain a spatial similarity vector.

4. The method for enhancing image content based on demosaicing according to claim 3, characterized in that: In the extracted cache window, the current pixel is located in the middle row and middle column of the cache window; each of the extracted neighboring windows overlaps with the cache window in at least one pixel; and the size of the neighboring window is consistent with the size of the cache window.

5. The method for enhancing image content based on demosaicing according to claim 4, characterized in that: The method for calculating the brightness difference between each adjacent window and the cache window includes: For each adjacent window, calculate the absolute value of the pixel value difference between each pixel point therein and the pixel point at the same position in the cache window; The absolute values ​​of all calculated pixel value differences are summed to obtain the brightness difference between the neighboring window and the cache window.

6. The method for enhancing image content based on demosaicing according to claim 3, characterized in that: The brightness difference is used to calculate the spatial similarity between each adjacent window and the cache window, which is expressed as: in, represents the spatial similarity between the i-th neighboring window and the cache window, represents the brightness difference between the i-th neighboring window and the cache window, i=0,1,2,…,7; and This is an adjustable parameter with a value greater than 0.

7. The method for enhancing image content based on demosaicing according to claim 1 or 3, characterized in that: The method for adjusting the weight factor of the current pixel point according to the spatial similarity vector and the weight factor variance includes: According to the spatial similarity vector and the variance of the weight factor, it is determined whether there is texture in the area where the current pixel is located; If it is determined that there is texture in the area where the current pixel is located, the weight factor of the current pixel is directionally enhanced; If it is determined that there is no texture in the area where the current pixel is located, the weight factor of the current pixel is weakened in a certain direction.

8. The method for enhancing image content based on demosaicing according to claim 7, characterized in that: The method for determining whether there is texture in the area where the current pixel point is located according to the spatial similarity vector and the weight factor variance includes: If the variance of the weight factor is greater than the first variance threshold, and at least one spatial similarity in the spatial similarity vector is greater than the similarity threshold, it is determined that there is texture in the area where the current pixel is located; If the variance of the weight factor is less than the second variance threshold, it is determined that there is no texture in the area where the current pixel point is located.

9. The method for enhancing image content based on demosaicing according to claim 8, characterized in that: The method for directional strengthening of the weight factor of the current pixel point comprises: The maximum weight factor of the current pixel is enlarged by a first multiple, and the minimum weight factor is reduced by a second multiple; The method for directional weakening of the weight factor of the current pixel point comprises: The maximum weight factor of the current pixel is reduced by a third multiple, and the minimum weight factor is increased by a fourth multiple.

10. The method for enhancing image content based on demosaicing according to claim 8, characterized in that: The method of adjusting the weight factor of the current pixel point according to the spatial similarity vector and the weight factor variance also includes: If the weight factor variance is less than or equal to the first variance threshold and greater than or equal to the second variance threshold, the weight factor of the current pixel point is kept unchanged.

11. A device for enhancing image content based on de-mosaicing, used to implement the method for enhancing image content based on de-mosaicing as claimed in any one of claims 1 to 10, characterized in that: The image content enhancement device based on demosaicing comprises: An image acquisition module, used for acquiring RAW images; An image processing module, used to calculate the variance of the weight factor of each pixel in the RAW image acquired by the image acquisition module; also used to calculate the spatial similarity vector of the cache window where the current pixel is located and multiple adjacent windows; also used to adjust the weight factor of the current pixel according to the spatial similarity vector and the variance of the weight factor; and also used to perform de-mosaic processing using the adjusted weight factor to obtain an RGB image; The image output module is used to output the RGB image obtained after being processed by the image processing module.

12. An electronic device, characterized in that: The method comprises a memory, a processor and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the method for enhancing image content based on de-mosaicing as described in any one of claims 1 to 10 is executed.

Images