Image processing method, device and equipment

Abstract

The present application provides an image processing method, device and equipment, which are applied to the field of image processing technology. The method includes determining the estimated values ​​of each channel of the current pixel point based on the original pixel value of the current pixel point and the original pixel values ​​of the pixels close to the current pixel point in a preset direction for any current pixel point in the original image. According to the first estimated value, the second estimated value and the third estimated value, a first difference set of the current pixel point is determined. According to the second difference set of pixels within a preset range centered on the current pixel point in the original image, the green pixel value of the green channel, the red pixel value of the red channel and the blue pixel value of the blue channel are determined. Based on the green pixel values, red pixel values ​​and blue pixel values ​​of all pixels in the original image, a target image is generated, which can solve the process of demosaicing the image in complex scenes, which may cause image quality problems such as color artifacts and jagged edges.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device and equipment. Background Art

[0002] In modern digital imaging systems, image sensors are the core components for capturing visual information. Most image sensors use the Bayer array to decompose incident light into three basic colors: red (R), green (G), and blue (B).

[0003] However, for each pixel of the original image captured by the image sensor, the Bayer array can only capture information of one color. To restore the full-color image, additional processing steps are required, which is the demosaicing process.

[0004] Although traditional demosaicing methods can restore the color information of images to a certain extent, the demosaicing process of images in complex scenes may cause image quality problems such as color artifacts and jagged edges if it is not handled properly. Summary of the invention

[0005] The purpose of this application is to provide an image processing method, device and equipment to solve the process of demosaicing images in complex scenes, which may cause image quality problems such as color artifacts and jagged edges.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to an image sensor, wherein the image sensor is configured to collect an original image in a Bayer array format, wherein each pixel in the original image records an acquisition channel, and the acquisition channel is any one of a red channel, a green channel, or a blue channel. The image processing method comprises: for any current pixel in the original image, based on the original pixel value of the current pixel and the original pixel values ​​of the pixels close to the current pixel in a preset direction, determining the estimated values ​​of each channel of the current pixel. The close pixels are used to represent a preset number of pixels in the original image with the smallest distance from the current pixel. The estimated values ​​include a first estimated value, a second estimated value, and a third estimated value. The first estimated value is a pixel estimated value of the acquisition channel. According to the first estimated value, the second estimated value, and the third estimated value, a first difference set of the current pixel is determined. According to a second difference set of pixels within a preset range centered on the current pixel in the original image, a green pixel value of the green channel, a red pixel value of the red channel, and a blue pixel value of the blue channel are determined. The second difference set includes a first difference set of each pixel within the preset range. Based on the green pixel values, red pixel values, and blue pixel values ​​of all pixels in the original image, a target image is generated. The target image is a full-color image that is a demosaiced version of the original image.

[0007] The image processing method provided in the embodiment of the present application can more accurately estimate the missing color channel values ​​and reduce artifacts and noise by comprehensively considering the color information of adjacent pixels. The difference set is used to analyze local color changes, effectively retain the edge details of the image, and avoid over-smoothing. Ensure smooth transitions between different color channels and provide natural and realistic color performance. Support the selection of multiple interpolation algorithms and filters, and adjust the de-mosaic strategy according to specific application scenarios. The selection mechanism of preset directions and similar pixels is adopted to reduce unnecessary calculations and improve processing speed. Even in complex lighting conditions or in the presence of slight noise, it can work stably to ensure the consistency of image quality.

[0008] A possible implementation method is to determine the estimated values ​​of each channel of the current pixel based on the original pixel value of the current pixel and the original pixel values ​​of the pixels close to the current pixel in a preset direction, including: obtaining the original pixel value of the current pixel as the first estimated value of the acquisition channel. According to the channel type of the acquisition channel, obtaining the original pixel values ​​of the pixels close to the current pixel in a preset direction. The preset direction includes one or more of the following vertical directions, horizontal directions, and diagonal directions. Based on the original pixel values ​​of the close pixels, determine the average pixel difference of the original pixel values ​​of the close pixels. Determine the second estimated value and the third estimated value based on the average pixel difference and the first estimated value.

[0009] In a possible implementation, the first difference set includes a first difference and a second difference. The first difference is used to represent the difference between the estimated value corresponding to the red channel and the estimated value corresponding to the green channel in the current pixel. The second difference is used to represent the difference between the estimated value corresponding to the blue channel and the estimated value corresponding to the green channel in the current pixel.

[0010] A possible implementation method, when the acquisition channel is a red channel or a blue channel, the green pixel value of the green channel, the red pixel value of the red channel, and the blue pixel value of the blue channel are determined according to the second difference set of pixels within a preset range centered on the current pixel in the original image, including: determining the first estimated value as a target pixel value. The target pixel value is a pixel value of the color corresponding to the acquisition channel. According to the second difference set, a first difference gradient value of the current pixel is determined. A difference filter is constructed based on the first difference gradient value. The second difference set is filtered using the difference filter to obtain a green pixel value. According to the second difference set and the green pixel value, the pixel value of another channel of the current pixel is determined. The other channel is a channel other than the green channel and the acquisition channel in the current pixel. When the acquisition channel is a red channel, the pixel value of the other channel is a blue pixel value, and when the acquisition channel is a blue channel, the pixel value of the other channel is a red pixel value.

[0011] A possible implementation manner is to construct a first difference filter based on the first difference gradient value, including:

[0012]

[0013] in, is the difference filter, is the first difference gradient value, is the sum of the filters.

[0014] In a possible implementation, filtering the second difference set using a difference filter to obtain a green pixel value includes: filtering a target difference in the second difference set using a difference filter to obtain a standard difference value. The target difference value is a difference between a collection channel and a green channel in the second difference set. The green pixel value is determined according to the target pixel value and the standard difference value.

[0015] A possible implementation, when the acquisition channel is a green channel, a green pixel value of the green channel, a red pixel value of the red channel, and a blue pixel value of the blue channel are determined according to a second difference set of pixels within a preset range centered on a current pixel in the original image, including: determining the first estimated value as a green pixel value. Determining the red pixel value according to the difference between the red channel and the acquisition channel in the green pixel value and the second difference set. Determining the blue pixel value according to the difference between the blue channel and the acquisition channel in the green pixel value and the second difference set.

[0016] In a second aspect, an embodiment of the present application provides an image processing device, which is applied to an image sensor, wherein the image sensor is configured to collect an original image in a Bayer array format, wherein each pixel in the original image records an acquisition channel, and the acquisition channel is any one of a red channel, a green channel, or a blue channel. The device includes: a processing module and a generation module.

[0017] Wherein, the processing module is used to determine the estimated values ​​of each channel of the current pixel point for any current pixel point in the original image based on the original pixel value of the current pixel point and the original pixel values ​​of the pixels close to the current pixel point in a preset direction. The close pixels are used to represent the preset number of pixels in the original image with the smallest distance from the current pixel point. The estimated values ​​include a first estimated value, a second estimated value and a third estimated value. The first estimated value is the pixel estimated value of the acquisition channel. According to the first estimated value, the second estimated value and the third estimated value, a first difference set of the current pixel point is determined. According to the second difference set of pixels within a preset range centered on the current pixel point in the original image, the green pixel value of the green channel, the red pixel value of the red channel and the blue pixel value of the blue channel are determined. The second difference set includes the first difference set of each pixel point within the preset range.

[0018] The generation module is used to generate a target image based on the green pixel value, the red pixel value and the blue pixel value of all the pixels in the original image. The target image is a full-color image obtained by demosaicing the original image.

[0019] In a third aspect, an embodiment of the present application provides an image processing device, which has the function of implementing the control method of the image processing device of the first aspect or any possible implementation method. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer can execute the control method of the image processing device of the first aspect or any possible implementation method mentioned above.

[0021] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the control method of an image processing device according to the first aspect or any possible implementation method.

[0022] Among them, the technical effects brought about by any design method in the second to fifth aspects can refer to the technical effects brought about by different possible implementation methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A diagram showing an arrangement of a Bayer array provided in an embodiment of the present application;

[0025] Figure 2 A structural schematic diagram of an image processing system provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0027] Figure 4 A specific example diagram of an image processing method provided in an embodiment of the present application;

[0028] Figure 5A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0029] Figure 6 Another structural schematic diagram of an image processing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the drawings in the embodiments of the present application are collected below to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0031] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0032] First, the terms in the embodiments of the present application are explained.

[0033] The Bayer array, or Bayer color filter array, is a color filter arrangement that is primarily used in digital image sensors. The main purpose of the Bayer array is to enable a single image sensor to capture color images. The Bayer array typically contains three channels: red (R), green (G), and blue (B), such as Figure 1 As shown in the figure, there are four arrangement forms of the Bayer array. According to the different positions of the current pixel in the image, the pixel types can include green-blue (GB), green-red (GR), blue (B) and red (R). Among them, green-blue (GB) means that the current pixel and the blue (B) pixel are arranged alternately in the horizontal direction, and green-red (GR) means that the current pixel and the red (R) pixel are arranged alternately in the horizontal direction.

[0034] In this application, for the original image, taking the current pixel point as Gb as an example, the coordinates of the current pixel point are (i, j), the pixel value of the current pixel point is G(i, j), and the pixel points close to the current pixel point in the vertical direction are R(i-1, j) and R(i+1, j).

[0035] The characteristic of the Bayer array is that each pixel can only capture information of one color channel, so the recorded original image data is actually an incomplete monochrome image. Specifically, the Bayer array is usually arranged repeatedly in an n*n pattern, where green pixels occupy half of the position, and red and blue pixels each occupy a quarter. This design is based on the fact that the human eye is more sensitive to green light, and aims to improve the overall brightness perception quality of the image. However, since each pixel only records information of one color, additional processing steps are required to restore the full-color image, which is the demosaicing process.

[0036] The role of the demosaicing algorithm is to reconstruct a complete RGB color image from the raw data obtained from the monochrome image sensor. Its performance directly determines the quality of the final output image, including color accuracy, detail retention, and the presence of artifacts. An efficient demosaicing algorithm can significantly improve the visual effect of the image, making the colors more vivid and realistic, and the details more delicate; on the contrary, if the demosaicing process is not handled properly, it may cause image quality problems such as color artifacts and jagged edges.

[0037] For example, near high-contrast edges, unnatural color transitions are prone to occur, resulting in color artifacts. The restoration of fine structures or textures is not ideal, causing the image to appear blurry and loss of details. Although some advanced algorithms improve image quality, they also increase the processor burden, which is not conducive to real-time applications and leads to high computational complexity. When faced with different lighting conditions or noise interference, the performance of existing algorithms may be unstable, affecting overall reliability.

[0038] Based on this, an embodiment of the present application provides an image processing method, device and equipment, which are applied to an image sensor, and the image sensor is configured to capture an original image in a Bayer array format, and each pixel in the original image records an acquisition channel, which is any one of a red channel, a green channel or a blue channel.

[0039] The method includes determining the estimated values ​​of each channel of the current pixel point for any current pixel point in the original image based on the original pixel value of the current pixel point and the original pixel values ​​of the pixels close to the current pixel point in a preset direction. The close pixels are used to represent the preset number of pixels with the smallest distance from the current pixel point in the original image. The estimated values ​​include a first estimated value, a second estimated value and a third estimated value. The first estimated value is the pixel estimated value of the acquisition channel. According to the first estimated value, the second estimated value and the third estimated value, a first difference set of the current pixel point is determined. According to the second difference set of pixels within a preset range centered on the current pixel point in the original image, a green pixel value of the green channel, a red pixel value of the red channel and a blue pixel value of the blue channel are determined. The second difference set includes the first difference set of each pixel point within the preset range. Based on the green pixel values, red pixel values ​​and blue pixel values ​​of all pixels in the original image, a target image is generated. The target image is a full-color image obtained by demosaicing the original image.

[0040] The image processing method provided in the embodiment of the present application can more accurately estimate the missing color channel values ​​and reduce artifacts and noise by comprehensively considering the color information of adjacent pixels. The difference set is used to analyze local color changes, effectively retain the edge details of the image, and avoid over-smoothing. Ensure smooth transitions between different color channels and provide natural and realistic color performance. Support the selection of multiple interpolation algorithms and filters, and adjust the de-mosaic strategy according to specific application scenarios. The selection mechanism of preset directions and similar pixels is adopted to reduce unnecessary calculations and improve processing speed. Even in complex lighting conditions or in the presence of slight noise, it can work stably to ensure the consistency of image quality.

[0041] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0042] On the one hand, the solution shown in the embodiment of the present application can be executed by an image processing device. Figure 2 As shown, the image processing device 200 may include: an image sensor 201 , a processor 202 and a display module 203 .

[0043] The image sensor 201 may be an image sensor configured with a Bayer array, and the image sensor 201 is used to convert an optical signal into an electrical signal, and output an original image based on the electrical signal to the processor 202. The original image is a monochrome image arranged in a Bayer array format, in which green pixels occupy half of the position, and red and blue pixels each occupy a quarter.

[0044] The processor 202 can receive the original image input by the image sensor 201, and process the original image through the image processing method provided in the embodiment of the present application to obtain a demosaiced target image.

[0045] The display module 203 is used to interact with the user. The display module 203 allows the graphical interface to adjust the parameters such as the mosaic algorithm, filter type and strength, etc. The display module 203 can also be used to provide a target image preview function, allowing the user to view the processing effect in real time.

[0046] It should be noted that the function of the above-mentioned image processing device 200 is only an example, and the present application does not limit the function of each component in the image processing device 200.

[0047] On the one hand, the embodiment of the present application provides an image processing method, which can be Figure 2 The image processing device 200 shown in FIG. Figure 3 As shown, the method may include the following steps.

[0048] S301, for any current pixel in the original image, based on the original pixel value of the current pixel and the original pixel values ​​of pixels close to the current pixel in a preset direction, determine the estimated values ​​of each channel of the current pixel.

[0049] The close pixel points are used to represent a preset number of pixel points in the original image that are the shortest distance from the current pixel point. The estimated value includes a first estimated value, a second estimated value and a third estimated value. The first estimated value is a pixel estimated value of the acquisition channel.

[0050] A possible implementation method is to obtain the original pixel value of the current pixel as the first estimated value of the acquisition channel. According to the channel type of the acquisition channel, the original pixel value of the pixel close to the current pixel in the preset direction is obtained. The preset direction includes one or more of the following vertical directions, horizontal directions, and diagonal directions. Based on the original pixel values ​​of the similar pixel points, the average pixel difference of the original pixel values ​​of the similar pixel points is determined. According to the average pixel difference and the first estimated value, the second estimated value and the third estimated value are determined.

[0051] Exemplarily, when the acquisition channel is the GB channel, for the vertical direction of the current pixel, the first red pixel value and the second red pixel value of the two red pixels close to the current pixel and the first green pixel value and the second green pixel value of the two green pixels close to the current pixel are obtained. According to the first estimated value, the first red pixel value, the second red pixel value, the first green pixel value and the second green pixel value, the second estimated value of the red channel of the current pixel is determined. For the horizontal direction of the current pixel, the first blue pixel value and the second blue pixel value of the two blue pixels close to the current pixel and the third green pixel value and the fourth green pixel value of the two green pixels close to the current pixel are obtained. According to the first estimated value, the first blue pixel value, the second blue pixel value, the third green pixel value and the fourth green pixel value, the third estimated value of the blue channel of the current pixel is determined.

[0052] For example, the second estimated value is determined by the following equation.

[0053] .

[0054] in, is the first estimate, is the second estimated value, is the first red pixel value, is the second red pixel value, is the first green pixel value, is the second green pixel value.

[0055] The third estimated value is determined by the following equation.

[0056] .

[0057] in, is the third estimated value, is the first blue pixel value, is the second blue pixel value, is the third green pixel value, is the fourth green pixel value.

[0058] When the acquisition channel is the GR channel, for the horizontal direction of the current pixel, the third red pixel value and the fourth red pixel value of the two red pixels close to the current pixel, and the fifth green pixel value and the sixth green pixel value of the two green pixels close to the green pixel are obtained. According to the first estimated value, the third red pixel value, the fourth red pixel value, the fifth green pixel value and the sixth green pixel value, the second estimated value of the red channel of the green pixel is determined. For the vertical direction of the green pixel, the third blue pixel value and the fourth blue pixel value of the two blue pixels close to the green pixel, and the seventh green pixel value and the eighth green pixel value of the two green pixels close to the green pixel are obtained. According to the first estimated value, the third blue pixel value, the fourth blue pixel value, the seventh green pixel value and the eighth green pixel value, the third estimated value of the blue channel of the green pixel is determined.

[0059] For example, the second estimated value is determined by the following equation.

[0060] .

[0061] in, is the second estimated value, is the first estimate, is the third red pixel value, is the fourth red pixel value, is the fifth green pixel value, is the sixth green pixel value.

[0062] The third estimated value is determined by the following equation.

[0063] .

[0064] in, is the third estimated value, is the first estimate, is the third blue pixel value, is the fourth blue pixel value, is the seventh green pixel value, is the eighth green pixel value.

[0065] When the acquisition channel is a blue channel, the original pixel value of the current pixel is obtained as the first target estimated value of the first target channel. When the current pixel is a blue pixel, the first target channel is a blue channel, and the first target estimated value is a third estimated value. When the current pixel is a red pixel, the first target channel is a red channel, and the first target estimated value is a second estimated value. For the first direction of the current pixel, a first set of similar pixel values ​​of a preset number of pixels close to the current pixel is obtained. The first direction includes a horizontal direction and a vertical direction. According to the third estimated value and the first set of similar pixel values, a first estimated value of the green channel is determined. For the second direction of the current pixel, a second set of similar pixel values ​​of a preset number of pixels close to the current pixel is obtained. The second direction includes a diagonal direction. According to the first estimated value, the first set of similar pixel values, and the second set of similar pixel values, a second target estimated value of the second target channel is determined. When the current pixel is a blue pixel, the second target channel is a red channel, and the second target estimated value is a second estimated value. When the current pixel is a red pixel, the second target channel is a blue channel, and the second target estimated value is a third estimated value.

[0066] For example, the second estimated value is determined by the following equation.

[0067] .

[0068] in, is the first estimate, is the second estimated value, is the first set of similar pixels.

[0069] For example, the third estimated value is determined by the following equation.

[0070] .

[0071] in, is the second estimated value, is the third estimated value, is the second closest pixel set.

[0072] When the acquisition channel is a red channel, the original pixel value of the current pixel is obtained as the first target estimated value of the first target channel. When the current pixel is a blue pixel, the first target channel is a blue channel, and the first target estimated value is a third estimated value. When the current pixel is a red pixel, the first target channel is a red channel, and the first target estimated value is a second estimated value. For the first direction of the current pixel, a first set of similar pixel values ​​of a preset number of pixels close to the current pixel is obtained. The first direction includes a horizontal direction and a vertical direction. According to the third estimated value and the first set of similar pixel values, a first estimated value of the green channel is determined. For the second direction of the current pixel, a second set of similar pixel values ​​of a preset number of pixels close to the current pixel is obtained. The second direction includes a diagonal direction. According to the first estimated value, the first set of similar pixel values, and the second set of similar pixel values, a second target estimated value of the second target channel is determined. When the current pixel is a blue pixel, the second target channel is a red channel, and the second target estimated value is a second estimated value. When the current pixel is a red pixel, the second target channel is a blue channel, and the second target estimated value is a third estimated value.

[0073] For example, the second estimated value is determined by the following equation.

[0074] .

[0075] in, is the first estimate, is the second estimated value, is the first set of similar pixels.

[0076] For example, the third estimated value is determined by the following equation.

[0077] .

[0078] in, is the second estimated value, is the third estimated value, is the second closest pixel set.

[0079] In this process, by comprehensively considering the color information of adjacent pixels, the missing color channel values ​​can be estimated more accurately, reducing artifacts and noise. At the same time, the selection mechanism of preset directions and similar pixels is adopted to reduce unnecessary calculations and improve processing speed.

[0080] S302: Determine a first difference value set of the current pixel according to the first estimated value, the second estimated value, and the third estimated value.

[0081] The first difference value set includes a first difference value and a second difference value. The first difference value is used to represent the difference between the estimated value corresponding to the red channel and the estimated value corresponding to the green channel in the current pixel point. The second difference value is used to represent the difference between the estimated value corresponding to the blue channel and the estimated value corresponding to the green channel in the current pixel point.

[0082] S303, determining a green pixel value of a green channel, a red pixel value of a red channel, and a blue pixel value of a blue channel according to a second difference set of pixels within a preset range centered on the current pixel in the original image.

[0083] The second difference value set includes the first difference value set of each pixel point within a preset range.

[0084] In a possible implementation, when the acquisition channel is a red channel, the first estimated value is determined to be a target pixel value, and the target pixel value is a pixel value of a color corresponding to the acquisition channel.

[0085] For example, .

[0086] in, is the red pixel value of the red channel.

[0087] According to the second difference value set, a first difference gradient value of the current pixel is determined.

[0088] For example, .

[0089] .

[0090] in, is the difference between the first estimated value and the second estimated value in the second difference value set. is the first difference gradient value.

[0091] A difference filter is constructed based on the first difference gradient value.

[0092] Specifically, the first difference filter can be constructed based on the first difference gradient value through the following equation.

[0093]

[0094] in, is the difference filter, is the first difference gradient value, is the sum of the filters, that is is the sum of all weight coefficients in the filter.

[0095] The second difference value set is filtered using a difference filter to obtain a green pixel value.

[0096] Specifically, the target difference in the second difference set is filtered using a difference filter to obtain a standard difference. The target difference is the difference between the acquisition channel and the green channel in the second difference set.

[0097] Determine the green pixel value based on the target pixel value and the standard deviation value.

[0098] For example, .

[0099] .

[0100] in, is the green pixel value of the green channel, diff Indicates the second difference set of the current pixel point within the preset range, represents matrix multiplication and sums the elements, where kd is the standard deviation.

[0101] According to the second difference set and the green pixel value, the pixel value of the other channel of the current pixel is determined. The other channel is the channel of the current pixel except the green channel and the acquisition channel. When the acquisition channel is the red channel, the pixel value of the other channel is the blue pixel value, and when the acquisition channel is the blue channel, the pixel value of the other channel is the red pixel value.

[0102] For example, the blue pixel value of a blue pixel is determined by the following equation.

[0103] .

[0104] in, is the blue pixel value of the blue pixel, It is determined based on the 9 pixels in the diagonal direction of the current pixel, and the (x, y) are .

[0105] In a possible implementation, when the acquisition channel is a blue channel, the first estimated value is determined to be a target pixel value, and the target pixel value is a pixel value of a color corresponding to the acquisition channel.

[0106] For example, .

[0107] According to the second difference value set, a first difference gradient value of the current pixel is determined.

[0108] A difference filter is constructed based on the first difference gradient value.

[0109] Specifically, the first difference filter can be constructed based on the first difference gradient value through the following equation.

[0110]

[0111] The second difference value set is filtered using a difference filter to obtain a green pixel value.

[0112] Specifically, the target difference in the second difference set is filtered using a difference filter to obtain a standard difference. The target difference is the difference between the acquisition channel and the green channel in the second difference set.

[0113] For example, .

[0114] .

[0115] According to the second difference set and the green pixel value, the pixel value of the other channel of the current pixel is determined. The other channel is the channel of the current pixel except the green channel and the acquisition channel. When the acquisition channel is the red channel, the pixel value of the other channel is the blue pixel value, and when the acquisition channel is the blue channel, the pixel value of the other channel is the red pixel value.

[0116] For example, the red pixel value of a red pixel is determined by the following equation.

[0117] .

[0118] in, is the red pixel value, It is determined based on the 9 pixels in the diagonal direction of the current pixel, and the (x, y) are .

[0119] In another possible implementation, when the acquisition channel is a green channel, the first estimated value is determined to be a green pixel value.

[0120] For example, .

[0121] The red pixel value is determined according to the green pixel value and the difference between the red channel and the acquisition channel in the second difference value set.

[0122] For example, .

[0123] in, is the green pixel value, is the red pixel value, It is determined based on the 9 pixels in the horizontal and vertical directions of the current pixel, and the (x, y) are .

[0124] The blue pixel value is determined according to the green pixel value and the difference between the blue channel and the acquisition channel in the second difference value set.

[0125] For example, .

[0126] in, is the green pixel value, is the blue pixel value, It is determined based on the 9 pixels in the horizontal and vertical directions of the current pixel, and the (x, y) are .

[0127] In this process, the difference set is used to analyze local color changes, effectively retaining image edge details and avoiding over-smoothing, ensuring smooth transitions between different color channels and providing natural and realistic color performance.

[0128] S304: Generate a target image based on the green pixel values, red pixel values, and blue pixel values ​​of all pixels in the original image.

[0129] The target image is a full-color image obtained by demosaicing the original image.

[0130] Exemplarily, for the entire original image, all pixels in all original images are traversed, and each pixel is processed one by one through the above-mentioned process S301-S303 to obtain the green pixel value, red pixel value and blue pixel value of each pixel. The green pixel value, red pixel value and blue pixel value of each pixel are used to generate a full-color image of the original image after demosaicing. The full-color image can be a full-color three-channel image.

[0131] Furthermore, in order to better illustrate the effect of this application, different methods were evaluated by objective indicators, as shown in Table 1 and Figure 4 The MAE results of different methods are compared. Figure 4 Compared with the results of , the present invention has better reconstruction of R, G, and B channels, and there is no obvious aliasing and false color in dense texture areas.

[0132] Table 1 Comparison of MAE results of different methods

[0133]

[0134] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the working principle of the device. It is understandable that in order to realize the above functions, the image processing device includes a hardware structure and / or software module corresponding to each function. Those skilled in the art should easily realize that the algorithm steps of each example described in the embodiment disclosed in this article can be implemented in the form of hardware or a collection of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0135] The embodiment of the present application can divide the functional modules of the image processing device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or software functional module.

[0136] It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. Figure 5 FIG. 2 shows a possible schematic diagram of the composition of the image processing device involved in the above-mentioned embodiments. Figure 5 As shown, the image processing device 500 may include: a processing module 501 and a generating module 502 .

[0137] The processing module 501 is used to support the image processing device 500 to execute Figure 3 S301, S302 and S303 in the illustrated image processing method.

[0138] Generating module 502, used to support image processing apparatus 500 to execute Figure 3 S304 in the illustrated image processing method.

[0139] In a possible implementation, the processing module is specifically used to obtain the original pixel value of the current pixel as the first estimated value of the acquisition channel. According to the channel type of the acquisition channel, the original pixel value of the pixel close to the current pixel in a preset direction is obtained. The preset direction includes one or more of the following vertical directions, horizontal directions, and diagonal directions. Based on the original pixel values ​​of the similar pixel points, the average pixel difference of the original pixel values ​​of the similar pixel points is determined. According to the average pixel difference and the first estimated value, the second estimated value and the third estimated value are determined.

[0140] In a possible implementation, the first difference set includes a first difference and a second difference. The first difference is used to represent the difference between the estimated value corresponding to the red channel and the estimated value corresponding to the green channel in the current pixel. The second difference is used to represent the difference between the estimated value corresponding to the blue channel and the estimated value corresponding to the green channel in the current pixel.

[0141] In a possible implementation, when the acquisition channel is a red channel or a blue channel, the processing module is specifically used to determine that the first estimated value is a target pixel value. The target pixel value is a pixel value of a color corresponding to the acquisition channel. According to the second difference set, a first difference gradient value of the current pixel is determined. A difference filter is constructed based on the first difference gradient value. The second difference set is filtered using the difference filter to obtain a green pixel value. According to the second difference set and the green pixel value, a pixel value of another channel of the current pixel is determined. The other channel is a channel other than the green channel and the acquisition channel in the current pixel. When the acquisition channel is a red channel, the pixel value of the other channel is a blue pixel value, and when the acquisition channel is a blue channel, the pixel value of the other channel is a red pixel value.

[0142] In a possible implementation manner, the processing module is specifically configured to determine the difference filter through the following equation.

[0143]

[0144] in, is the difference filter, is the first difference gradient value, is the sum of the filters.

[0145] In a possible implementation, the processing module is specifically used to filter the target difference in the second difference set using a difference filter to obtain a standard difference value. The target difference value is the difference between the acquisition channel and the green channel in the second difference set. The green pixel value is determined according to the target pixel value and the standard difference value.

[0146] In a possible implementation, when the acquisition channel is a green channel, the processing module is specifically configured to determine that the first estimated value is a green pixel value. The red pixel value is determined based on the difference between the red channel and the acquisition channel in the green pixel value and the second difference value set. The blue pixel value is determined based on the difference between the blue channel and the acquisition channel in the green pixel value and the second difference value set.

[0147] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0148] The image processing device 500 provided in the embodiment of the present application is used to perform the above Figure 3The image processing method shown can therefore achieve the same effect as the above-mentioned image processing method.

[0149] The present application also provides an image processing device, which can be the image processing device described in the above embodiment. Figure 2 The image processing device 200 shown can execute the image processing method and related steps in the above method embodiment.

[0150] The embodiment of the present application also provides a computer-readable storage medium on which instructions are stored. When the instructions are executed, the image processing method and related steps in the above method embodiment are executed.

[0151] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes the image processing method and related steps in the above method embodiment.

[0152] In some embodiments, the methods described herein may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or encoded on other non-transitory media or articles of manufacture.

[0153] The present application also provides an image processing system 600. Figure 6 As shown, the image processing system 600 includes at least one processor 601 and at least one interface circuit 602 .

[0154] As an example, when the image processing system 600 includes a processor and an interface circuit, the processor may be Figure 6 The processor 601 shown in the solid line frame (or the processor 601 shown in the dotted line frame) may be Figure 6 The interface circuit 602 shown in the solid line frame (or the interface circuit 602 shown in the dotted line frame) is shown in the solid line frame. When the image processing system 600 includes two processors and two interface circuits, the two processors include Figure 6 The processor 601 shown in the solid line frame and the processor 601 shown in the dotted line frame, the two interface circuits include Figure 6 The interface circuit 602 shown in the solid line frame and the interface circuit 602 shown in the dotted line frame are not limited to this.

[0155] The processor 601 and the interface circuit 602 may be interconnected via a line. For example, the interface circuit 602 may be used to receive a signal. In another example, the interface circuit 602 may be used to send a signal to another device (e.g., the processor 601). For example, the interface circuit 602 may read a computer instruction stored in a memory and send the computer instruction to the processor 601. The processor 601 executes the instruction and combines the input and output devices to implement the various steps in the above embodiments, such as implementing Figure 2-Figure 4 Of course, the image processing system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0156] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0157] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0158] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0159] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0160] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0161] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that: Applied to an image sensor, the image sensor is configured to collect an original image in a Bayer array format, each pixel in the original image records an acquisition channel, and the acquisition channel is any one of a red channel, a green channel or a blue channel; the method comprises: For any current pixel in the original image, based on the original pixel value of the current pixel and the original pixel values ​​of the adjacent pixels in a preset direction, the estimated values ​​of the various channels of the current pixel are determined; the adjacent pixels are used to represent a preset number of pixels in the original image that are at the shortest distance from the current pixel; the estimated values ​​include a first estimated value, a second estimated value and a third estimated value; the first estimated value is a pixel estimated value of the acquisition channel; the second estimated value and the third estimated value are determined based on an average pixel difference of the original pixel values ​​of the adjacent pixels and the first estimated value; Determine a first difference value set of the current pixel according to the first estimated value, the second estimated value and the third estimated value; determining a green pixel value of the green channel, a red pixel value of the red channel, and a blue pixel value of the blue channel according to a second difference set of pixels within a preset range centered on the current pixel in the original image, wherein the second difference set includes the first difference set of each pixel within the preset range; A target image is generated based on the green pixel values, red pixel values ​​and blue pixel values ​​of all pixels in the original image; the target image is a full-color image obtained by demosaicing the original image.

2. The method according to claim 1, characterized in that The determining, based on the original pixel value of the current pixel and the original pixel values ​​of pixels close to the current pixel in a preset direction, estimated values ​​of each channel of the current pixel includes: Obtaining an original pixel value of the current pixel as a first estimated value of the acquisition channel; According to the channel type of the acquisition channel, the original pixel value of the pixel point close to the current pixel point in the preset direction is obtained; the preset direction includes one or more of the following: vertical direction, horizontal direction and diagonal direction; Based on the original pixel values ​​of the similar pixels, determining an average pixel difference of the original pixel values ​​of the similar pixels; The second estimated value and the third estimated value are determined according to the average pixel difference and the first estimated value.

3. The method according to claim 1, characterized in that: The first difference value set includes a first difference value and a second difference value; the first difference value is used to represent the difference between the estimated value corresponding to the red channel and the estimated value corresponding to the green channel in the current pixel point; The second difference is used to represent the difference between the estimated value corresponding to the blue channel and the estimated value corresponding to the green channel in the current pixel.

4. The method according to claim 1, characterized in that: When the acquisition channel is a red channel or a blue channel, determining the green pixel value of the green channel, the red pixel value of the red channel, and the blue pixel value of the blue channel according to a second difference set of pixels within a preset range centered on the current pixel in the original image, comprises: Determine the first estimated value as a target pixel value; the target pixel value is a pixel value of a color corresponding to the acquisition channel; Determine a first difference gradient value of the current pixel according to the second difference value set; constructing a difference filter based on the first difference gradient value; Filtering the second difference value set by using the difference filter to obtain the green pixel value; According to the second difference set and the green pixel value, determine the pixel value of another channel of the current pixel point; the other channel is a channel of the current pixel point except the green channel and the acquisition channel; when the acquisition channel is the red channel, the pixel value of the other channel is the blue pixel value, and when the acquisition channel is the blue channel, the pixel value of the other channel is the red pixel value.

5. The method according to claim 4, characterized in that The step of constructing a first difference filter based on the first difference gradient value comprises: Among them, the is the difference filter, the is the first difference gradient value, is the sum of the filters.

6. The method according to claim 4, characterized in that The filtering the second difference value set by using the difference filter to obtain the green pixel value includes: Filtering the target difference in the second difference set by using a difference filter to obtain a standard difference; the target difference is the difference between the acquisition channel and the green channel in the second difference set; The green pixel value is determined according to the target pixel value and the standard deviation value.

7. The method according to claim 1, characterized in that When the acquisition channel is a green channel, determining the green pixel value of the green channel, the red pixel value of the red channel, and the blue pixel value of the blue channel according to a second difference set of pixels within a preset range centered on the current pixel in the original image, comprises: Determine the first estimated value as the green pixel value; Determine the red pixel value according to the green pixel value and the difference between the red channel and the acquisition channel in the second difference value set; The blue pixel value is determined according to the green pixel value and the difference between the blue channel and the acquisition channel in the second difference value set.

8. An image processing device, characterized in that: Applied to an image sensor, the image sensor is configured to collect an original image in a Bayer array format, each pixel in the original image records an acquisition channel, and the acquisition channel is any one of a red channel, a green channel or a blue channel; the device comprises: A processing module, for determining, for any current pixel in the original image, estimated values ​​of each channel of the current pixel based on the original pixel value of the current pixel and the original pixel values ​​of the adjacent pixels in a preset direction; the adjacent pixels are used to represent a preset number of pixels in the original image with the smallest distance from the current pixel; the estimated values ​​include a first estimated value, a second estimated value and a third estimated value; the first estimated value is a pixel estimated value of the acquisition channel; the second estimated value and the third estimated value are determined based on the average pixel difference of the original pixel values ​​of the adjacent pixels and the first estimated value; a first difference set of the current pixel is determined according to the first estimated value, the second estimated value and the third estimated value; a green pixel value of the green channel, a red pixel value of the red channel and a blue pixel value of the blue channel are determined according to a second difference set of pixels within a preset range centered on the current pixel in the original image; the second difference set includes the first difference set of each pixel within the preset range; A generation module is used to generate a target image based on the green pixel values, red pixel values ​​and blue pixel values ​​of all pixels in the original image; the target image is a full-color image obtained by demosaicing the original image.

9. An image processing device, characterized in that: The image processing device comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

Images