Raw domain image processing method, image denoising method and electronic device

Abstract

The application discloses a RAW domain image processing method, an image noise reduction method and an electronic device. The RAW domain image processing method comprises the following steps: acquiring a pixel matrix of an image to be processed, and acquiring an initial target confidence of each pixel in the pixel matrix and an initial target direction corresponding to the initial target confidence; performing double-threshold segmentation according to the initial target confidence of each pixel to obtain coarse segmentation edge information of the pixel array; and acquiring fine segmentation edge information of the pixel array according to the obtained coarse segmentation edge information and based on the initial target direction. The application can improve the brightness of a picture and optimize the quality of the picture.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a RAW domain image processing method, an image noise reduction method based on the above-mentioned RAW domain image processing method, and an electronic device. Background Technology

[0002] Image sensors are a crucial component of digital cameras, converting optical images into electrical signals. They are widely used in digital cameras, mobile terminals, portable electronic devices, and other electronic equipment. Image sensors are broadly classified into two categories: CCD (Charge Coupled Device) image sensors and CMOS (Complementary Metal Oxide Semiconductor) image sensors. CMOS image sensors, with their advantages of high integration, low power consumption, high speed, and low cost, are widely used in many products, including mobile phones, tablets, automobiles, and security monitoring systems.

[0003] To ensure the performance of CMOS image sensors in low-light conditions, image sensors typically increase exposure time or gain to enhance image brightness. Increasing exposure time can cause motion blur when shooting moving objects, while increasing gain can introduce significant noise. Summary of the Invention

[0004] The purpose of this application is to provide a RAW domain image processing method, an image noise reduction method based on the above-mentioned RAW domain image processing method, and an electronic device, which can improve image brightness and optimize image quality.

[0005] To achieve the above objectives, the technical solution of this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide an image processing method, comprising: acquiring a pixel matrix of an image to be processed, and acquiring an initial target confidence score and an initial target direction corresponding to the initial target confidence score for each pixel in the pixel matrix; performing dual threshold segmentation based on the initial target confidence score of each pixel to obtain coarse edge information of the pixel array; and acquiring subdivided edge information of the pixel array based on the obtained coarse edge information and the initial target direction.

[0007] Optionally, obtaining the initial target confidence and the initial target direction specifically includes:

[0008] Edge detection is performed on the pixel matrix using an edge detection operator to obtain the initial target confidence and the initial target direction. The gradient of a preset direction for each pixel is calculated by the edge detection operator as a confidence, and the maximum value of each confidence is output as the initial target confidence for the corresponding pixel. The preset direction corresponding to the initial target confidence is used as the initial target direction.

[0009] Optionally, the edge detection operator includes the Sobel operator; and / or, the preset direction includes 0 degrees, 45 degrees, 90 degrees, and 135 degrees.

[0010] Optionally, obtaining the coarse edge information of the pixel array specifically includes: defining a first preset threshold and a second preset threshold greater than the first preset threshold;

[0011] Pixels with an initial target confidence level lower than the first preset threshold are identified as initial flat pixels and are located in a flat region; pixels with an initial target confidence level higher than the second preset threshold are identified as initial edge pixels and are located in an edge region; the remaining pixels are identified as initial undetermined pixels and are located in an undetermined region.

[0012] Optionally, obtaining the subdivision edge information of the pixel array includes: performing isolated point detection on the initial edge pixels to obtain the target edge pixels and supplementary undetermined pixels.

[0013] Optionally, isolated point detection is performed on the initial edge pixels to obtain the target edge pixels and supplementary undetermined pixels, specifically including:

[0014] Define the initial edge pixels to be detected, and define an isolated point detection area centered on the initial edge pixels to be detected;

[0015] If the number of initial edge pixels in the isolated point detection area is greater than a first detection threshold, the initial edge pixel to be detected is determined to be the target edge pixel; otherwise, the initial edge pixel to be detected is determined to be the supplementary undetermined pixel. Alternatively, if the number of initial edge pixels in the isolated point detection area that have the same initial target direction as the initial edge pixel to be detected is greater than a second detection threshold, the initial edge pixel to be detected is determined to be the target edge pixel; otherwise, the initial edge pixel to be detected is determined to be the supplementary undetermined pixel.

[0016] Optionally, based on the obtained coarse edge information and the initial target direction, the subdivided edge information of the pixel array is obtained, including:

[0017] Neighborhood main direction voting is performed on the supplementary undetermined pixels, the initial undetermined pixels, and the initial flat pixels to obtain the true target direction.

[0018] Optionally, neighborhood main direction voting is performed on the supplementary undetermined pixels, the initial undetermined pixels, and the initial flat pixels to obtain the true target direction, specifically including:

[0019] Define the pixel to be voted on, and define the target voting area centered on the pixel to be voted on;

[0020] The initial target directions of the initial edge pixels and initial undetermined pixels in the target voting area are counted, and the initial target direction with the most occurrences is defined as the voting direction;

[0021] The voting direction is compared with the initial target direction of the corresponding pixel to be voted. If they match, the voting direction is defined as the true target direction of the pixel to be voted. If they do not match, the voting direction is defined as the pixel to be assigned.

[0022] Optionally, when the pixel to be assigned is not defined, for the initial flat pixel, define the pixel to be voted and the target voting area centered on it, and obtain the pixel to be voted whose initial target direction is inconsistent with the initial target direction of the initial edge pixel and the initial undetermined pixel in the target voting area. Define the pixel to be voted as the inconsistent voting pixel, and the rest as consistent voting pixels. Count the number of consistent voting pixels in the target voting area corresponding to the inconsistent voting pixels. If it is less than a preset threshold, the pixel to be voted is a truly flat pixel; the pixels to be voted other than the truly flat pixels are pixels with undetermined direction.

[0023] Alternatively, when the pixel to be assigned is defined, for the initial flat pixel among the pixels to be assigned, count the number of pixels in the target voting area centered on it whose initial target direction is the voting direction. If the number is less than a preset threshold, then the pixel to be assigned is a truly flat pixel; the pixels to be assigned other than the truly flat pixels are pixels whose direction is yet to be determined.

[0024] Optionally, based on the obtained coarse edge information and the initial target direction, the subdivided edge information of the pixel array is obtained, including:

[0025] The orientation matching degree of the pixel to be determined is detected to obtain the true target orientation of the pixel.

[0026] Optionally, orientation matching degree detection is performed on the pixels whose orientation is to be determined to obtain the true target orientation of the pixels whose orientation is to be determined, specifically including:

[0027] Define a pixel to be detected in a certain direction, define a matching degree detection area centered on the pixel to be detected in a certain direction, define several score reference matrices, and the size of the score reference matrices is the same as the size of the matching degree detection area;

[0028] A score reference matrix corresponding to the initial target direction of the pixel to be detected is selected, and the sum of the scores of the initial edge pixels in the matching degree detection area at the corresponding positions of the score reference matrix is ​​calculated as the first score sum, and the sum of the scores of the initial pixel to be detected at the corresponding positions of the score reference matrix is ​​calculated as the second score sum. The first score sum is multiplied by a weight coefficient greater than or equal to 1 to obtain the first weight score, and the second score sum is multiplied by a weight coefficient less than or equal to 1 to obtain the second weight score. If the sum of the first weight score and the second weight score is greater than the matching degree threshold, the initial target direction of the pixel to be detected is defined as its true target direction; if it is less than the matching degree threshold, the voting direction of the pixel to be detected is defined as its true target direction.

[0029] Alternatively, a score reference matrix corresponding to the initial target direction of the pixel to be detected can be selected, and the sum of the scores of the initial edge pixels and the initial undetermined pixels in the matching degree detection area at the corresponding positions in the score reference matrix can be calculated as a first calculated score. The score reference matrix corresponding to the voting direction of the pixel to be detected can be selected, and the sum of the scores of the initial edge pixels and the initial undetermined pixels in the matching degree detection area at the corresponding positions in the score reference matrix can be calculated as a second calculated score. The initial target direction or voting direction corresponding to the larger of the first calculated score and the second calculated score can be taken as the true target direction of the pixel to be detected.

[0030] Secondly, embodiments of this application provide an image denoising method, the image denoising method comprising:

[0031] Obtain a pixel array with defined truly flat regions and non-truly flat regions; wherein, the truly flat regions include the truly flat pixels, and the non-truly flat regions include other types of pixels besides the truly flat pixels;

[0032] Noise reduction is performed on the pixels in the truly flat region and the pixels in the non-truly flat region respectively to obtain the noise-reduced pixel values.

[0033] Thirdly, embodiments of this application provide an electronic device, such as an image sensor, including a memory and a processor; the processor is used to execute a computer program stored in the memory to implement the steps of the image processing method.

[0034] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, wherein the steps of the image processing method and / or the image noise reduction method are executed by a processor.

[0035] The beneficial effects of the technical solutions provided in this application are:

[0036] The RAW domain image processing method, image denoising method, and electronic device provided in this application obtain a pixel matrix of an image to be processed, and obtain the initial target confidence and the initial target direction corresponding to the initial target confidence of each pixel in the pixel matrix; perform dual threshold segmentation based on the initial target confidence of each pixel to obtain coarse edge information of the pixel array; and obtain subdivided edge information of the pixel array based on the obtained coarse edge information and the initial target direction, and denoise the image pixels, thereby greatly improving the image brightness and optimizing the image quality. Attached Figure Description

[0037] Figure 1 This is a schematic flowchart of the RAW domain image processing method provided in the embodiments of this application;

[0038] Figure 2a yes Figure 1 A schematic diagram illustrating the confidence and orientation obtained in the RAW domain image processing method;

[0039] Figure 2b yes Figure 1 A schematic diagram illustrating the coarse edge information obtained in the RAW domain image processing method;

[0040] Figure 2c yes Figure 1 A schematic diagram illustrating the acquisition of subdivision edge information in RAW domain image processing methods;

[0041] Figure 3 This is a schematic flowchart of the image noise reduction method provided in the embodiments of this application. Detailed Implementation

[0042] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this application's specification is for the purpose of describing particular embodiments only and is not intended to limit this application.

[0043] Figure 1This is a schematic flowchart of the RAW domain (raw domain) image processing method provided in this application embodiment. The image processing method can improve image brightness and optimize image quality. Please refer to... Figure 1 The RAW domain image processing method in this embodiment may include the following steps:

[0044] Step S101: Obtain the pixel matrix of the image to be processed, and obtain the initial target confidence and the initial target direction corresponding to the initial target confidence for each pixel in the pixel matrix;

[0045] In step S101, obtaining the initial target confidence and the initial target direction corresponding to the initial target confidence of each pixel in the pixel matrix may include: using an edge detection operator to perform edge detection on the pixel matrix to obtain the initial target confidence and the initial target direction. Specifically, the gradient of the preset direction of each pixel is calculated by the edge detection operator as the confidence, and the maximum value among the confidences is output as the initial target confidence of the corresponding pixel. The preset direction corresponding to the initial target confidence is used as the initial target direction.

[0046] Edge detection operators can include the Sobel operator, which can calculate the gradient of each pixel in a preset direction (e.g., 0 degrees, 45 degrees, 90 degrees, and 135 degrees) as the confidence level. Figure 2a As shown, the edge detection operator also outputs the maximum confidence value in four directions and the direction in which the maximum confidence value is output, to obtain the initial target confidence value (i.e., the maximum confidence value) and the initial target direction for each pixel. The Sobel edge detection operator can employ existing image edge detection methods, which will not be elaborated upon here. Of course, in this invention, other first-order differential operators can also be used to calculate the gradient of each pixel in a preset direction as the confidence value, such as the Roberts operator, the Prewitt operator, etc., as needed; this is not limited here.

[0047] Step S103: Perform double threshold segmentation based on the initial target confidence of each pixel to obtain the coarse edge information of the pixel array.

[0048] Here, after the calculation in step S101, each pixel will obtain an initial target confidence score and its initial target direction information. In this step, the initial target confidence score can be segmented using a double threshold to obtain the coarse edge information of the pixel array.

[0049] Step S103 may specifically include: defining a first preset threshold and a second preset threshold greater than the first preset threshold; wherein, pixels with an initial target confidence level lower than the first preset threshold are determined to be initial flat pixels and are located in a flat region; pixels with an initial target confidence level higher than the second preset threshold are determined to be initial edge pixels and are located in an edge region; the remaining pixels are determined to be initial undetermined pixels and are located in an undetermined region. Figure 2b As shown.

[0050] The method for defining the first preset threshold and the second preset threshold, which is greater than the first preset threshold, can be as follows: During the image processing of the original image to be processed, based on the target confidence of each pixel, the confidence values ​​of flat pixels and edge pixels can be roughly distinguished based on experience. The first preset threshold and the second preset threshold can be defined based on the confidence values ​​of different types of pixels. For example, it is generally considered that the target confidence of flat pixels is 10%-30% of the theoretical maximum value of the selected edge detection operator, and the target confidence of edge pixels is 40%-60% of the theoretical maximum value of the selected edge detection operator. Therefore, the target confidence of the pixel to be determined is 30%-40% of the theoretical maximum value of the selected edge detection operator. Thus, the first preset threshold can be defined as 30% of the theoretical maximum value of the selected edge detection operator, and the second preset threshold as 40% of the theoretical maximum value of the selected edge detection operator. It should be noted that the first preset threshold and the second preset threshold can be defined according to the selection of the edge detection operator and the actual needs of the scheme.

[0051] Step S105: Based on the obtained coarse edge information, obtain the refined edge information of the pixel array based on the initial target direction.

[0052] Here, after the calculation in step S103, taking a certain pixel as the center, the coarse edge information of all pixels within an N*M row and column range around that pixel can be obtained. N and M can be the same or different integers, for example, N*M is 5*5. In this step, all pixels that have obtained the coarse edge information can be further subdivided, for example, the N*M pixel matrix can be further subdivided.

[0053] In step S105, obtaining the subdivided edge information can be divided into three steps: performing isolated point detection, performing neighborhood main direction voting, and performing direction matching degree detection, such as... Figure 2c As shown.

[0054] In step S105, obtaining the subdivision edge information of the pixel array includes: performing isolated point detection on the initial edge pixels to obtain the target edge pixels and supplementary undetermined pixels.

[0055] Isolated point detection is performed on the initial edge pixels to obtain the target edge pixels and supplementary undetermined pixels. Specifically, this may include:

[0056] Define the initial edge pixels to be detected, and define the isolated point detection area with the initial edge pixels to be detected as the center;

[0057] If the number of initial edge pixels in the isolated point detection area is greater than the first detection threshold, the initial edge pixel to be detected is determined to be the target edge pixel; otherwise, the initial edge pixel to be detected is determined to be a supplementary undetermined pixel. Alternatively, if the number of initial edge pixels in the isolated point detection area that have the same initial target direction as the initial edge pixel to be detected is greater than the second detection threshold, the initial edge pixel to be detected is determined to be the target edge pixel; otherwise, the initial edge pixel to be detected is determined to be a supplementary undetermined pixel.

[0058] Specifically, a pixel matrix of, for example, size 5*5 is defined as the isolated point detection area, centered on the initial edge pixel to be detected, and isolated point detection is performed on the initial edge pixel to be detected. Of course, in other embodiments, a pixel matrix of, for example, size 3*3 or size 7*7 can also be defined, and this is not limited here.

[0059] Optionally, the number of initial edge pixels in the isolated point detection area is counted. If the number is greater than the first detection threshold, it indicates that the initial edge pixel to be detected is the target edge pixel; if the number is less than the first detection threshold, it indicates that the initial edge pixel to be detected is a supplementary undetermined pixel.

[0060] The method for defining the first detection threshold can be as follows: for an image to be processed that is divided into multiple A*A pixel matrices, the first detection threshold of each pixel matrix is ​​greater than or equal to A. It should be noted that the size of the pixel matrix in the isolated point detection area should be the same as the size of the pixel matrix when defining the first detection threshold.

[0061] For example, the first detection threshold for the isolated point detection area of ​​a 5*5 pixel matrix is ​​8. That is, in the 5*5 pixel matrix, if the number of initial edge pixels is greater than 8, the initial edge pixel to be detected at the center of the isolated point detection area is the target edge pixel. If the number of initial edge pixels is less than 8, the initial edge pixel to be detected at the center of the isolated point detection area is a supplementary undetermined pixel.

[0062] Optionally, the number of initial edge pixels in the isolated point detection region that share the same initial target direction as the initial edge pixel to be detected is counted. If the number exceeds the second detection threshold, the initial edge pixel to be detected is determined to be a target edge pixel; otherwise, it is determined to be a supplementary undetermined pixel. Here, the initial target direction of the target edge pixel is defined as the true target direction of the target edge pixel. The method for defining the second detection threshold is similar to the method for defining the first detection threshold and will not be elaborated here.

[0063] In step S105, obtaining the subdivision edge information of the pixel array may further include:

[0064] Neighborhood main direction voting is performed on supplementary undetermined pixels, initial undetermined pixels, and initial flat pixels to obtain the true target direction.

[0065] Specifically, neighboring main direction voting is performed on supplementary undetermined pixels, initial undetermined pixels, and initial flat pixels to obtain the true target direction. This can include: defining pixels to be voted on, defining a target voting area centered on the undetermined pixels; counting the initial target directions of the initial edge pixels and initial undetermined pixels in the target voting area, and defining the initial target direction with the most occurrences as the voting direction; comparing the voting direction with the initial target direction of the corresponding undetermined pixel, if they match, defining the voting direction as the true target direction of the undetermined pixel, and if they do not match, defining it as a pixel with undetermined direction.

[0066] Within the target voting area centered on the voting pixel, the initial target direction of all pixels except those in the flat region is counted. The initial target direction with the most occurrences is defined as the voting direction. Each voting pixel also has its own initial target direction. If the voting direction and the initial target direction of the voting pixel are the same, then this direction is defined as the true target direction of that pixel after voting by the main direction of its neighborhood. If they are different, it is defined as a pixel with a direction to be assigned. It can be understood that the pixels to be voted here include supplementary undetermined pixels, initial undetermined pixels, and initial flat pixels, while the pixels participating in the counting of voting directions are initial undetermined pixels and initial edge pixels.

[0067] In step S105, obtaining the subdivision edge information of the pixel array may further include:

[0068] When no direction pixel to be assigned is defined, for an initial flat pixel, define the pixel to be voted and the target voting area centered on it. In the target voting area, identify the pixel whose initial target direction is most frequently inconsistent with the initial target direction of the pixel to be voted. Define this as an inconsistent voting pixel, and the rest as consistent voting pixels. Count the number of consistent voting pixels in the target voting area corresponding to the inconsistent voting pixels. If the number is less than a preset threshold, the pixel to be voted is a truly flat pixel; pixels to be voted other than truly flat pixels are pixels with undetermined direction.

[0069] Alternatively, when defining the direction pixels to be assigned, for the initial flat pixels among the direction pixels to be assigned, count the number of pixels in the target voting area centered on it whose initial target direction is the voting direction. If it is less than a preset threshold, then the direction pixels to be assigned are truly flat pixels; the direction pixels to be assigned other than truly flat pixels are pixels with undetermined directions.

[0070] The preset threshold can be defined as follows: obtain the theoretical maximum value under the selected algorithm. For example, if all pixels in a 5*5 pixel matrix are edge pixels, then the theoretical maximum value under the selected algorithm is 25, and the preset threshold can be 10%-30% of the above theoretical maximum value. It should be noted that the size of the pixel matrix of the target voting area should be the same as the size of the pixel matrix divided when defining the preset threshold.

[0071] In this implementation step, truly flat pixels are filtered out using a preset threshold, preventing them from proceeding to the next direction matching degree detection. All pixels in the unassigned direction pixels, excluding truly flat pixels, are defined as direction-undetermined pixels and participate in the next direction matching degree detection step.

[0072] In step S105, obtaining the subdivision edge information of the pixel array may further include:

[0073] The orientation matching degree of the pixels with undetermined orientation obtained in the above steps is detected to obtain the true target orientation of the pixels with undetermined orientation.

[0074] Specifically, a pixel to be detected in a certain direction is defined, a matching degree detection area is defined with the pixel to be detected in a certain direction as the center, and several score reference matrices are defined, with the size of the score reference matrices being consistent with the size of the matching degree detection area.

[0075] When performing orientation matching degree detection, first define the pixels to be detected in the desired orientation. Then, define a matching degree detection area (e.g., a 5x5 pixel array) centered on these pixels. Finally, define score reference tables for four orientations (e.g., 0 degrees, 45 degrees, 90 degrees, and 135 degrees). The pixel array size of these score reference tables is the same as that of the matching degree detection area. Optionally, for the 0-degree score reference table, the middle row has the highest values, gradually decreasing towards the sides; for the 90-degree score reference table, the middle column has the highest values, gradually decreasing towards the sides. Similarly, for the 45-degree and 135-degree score reference tables, the scores are highest on the main diagonal and secondary diagonal, respectively, gradually decreasing towards the sides. Numbers from the top left to the bottom right are assigned to the main diagonal, and numbers from the bottom left to the top right are assigned to the secondary diagonal.

[0076] Further optionally, the direction matching detection method may include: selecting a score reference matrix corresponding to the initial target direction of the pixel to be detected, and calculating the sum of the scores of the initial edge pixels in the matching detection area at the corresponding positions of the score reference matrix as the first score sum and the sum of the scores of the initial pixel to be detected at the corresponding positions of the score reference matrix as the second score sum. The first score sum is multiplied by a weight coefficient greater than or equal to 1 to obtain the first weight score, and the second score sum is multiplied by a weight coefficient less than or equal to 1 to obtain the second weight score. If the sum of the first weight score and the second weight score is greater than the matching degree threshold, the initial target direction of the pixel to be detected is defined as its true target direction; if it is less than the matching degree threshold, the voting direction of the pixel to be detected is defined as its true target direction.

[0077] Specifically, in the matching degree detection centered on the matching degree detection pixel, the center pixel has an initial target direction, such as 0 degrees. For edge pixels in the matching degree detection, the score value in the score reference table corresponding to the initial target direction, such as 0 degrees, is assigned, and then the scores are added together to calculate the score. If the score is greater than the threshold, the angle is determined to be the true target direction of the center pixel; if it is less than the threshold, the voting direction, such as 45 degrees, is taken as the true target direction.

[0078] Alternatively, a score reference matrix corresponding to the initial target direction of the pixel to be detected can be selected, and the sum of the scores of the initial edge pixels and the initial pixel to be detected in the matching detection area at the corresponding positions in the score reference matrix can be calculated as the first calculated score. A score reference matrix corresponding to the voting direction of the pixel to be detected can be selected, and the sum of the scores of the initial edge pixels and the initial pixel to be detected in the matching detection area at the corresponding positions in the score reference matrix can be calculated as the second calculated score. The initial target direction or voting direction corresponding to the larger of the first calculated score and the second calculated score can be taken as the true target direction of the pixel to be detected.

[0079] The direction matching detection method can also include: calculating a first score for the initial target direction, such as 0 degrees; calculating a second score for the voting direction, such as 45 degrees, according to the 45-degree reference table; comparing the first score and the second score; and taking the angle corresponding to the larger score as the recalculated true target direction for the pixel.

[0080] Through the above steps, each pixel has a recalculated true target direction, and the edge pixels in the detection are also found, realizing the detection of edge regions and directions, which can then be used for subsequent noise reduction, interpolation and other processing.

[0081] In this way, when detecting edges, this application first uses an optimized Sobel operator to extract edge direction and confidence level, and then uses dual thresholds to coarsely classify edge cases. For pixel values ​​with confidence levels between the dual thresholds, its neighborhood information is further analyzed to refine the edge cases, thereby greatly improving image quality.

[0082] like Figure 3 As shown in the embodiments of this application, an image denoising method for the RAW domain image processing method described above is also provided, the image denoising method comprising:

[0083] S301, Obtain a pixel array with defined truly flat regions and non-truly flat regions; wherein, the truly flat regions include the truly flat pixels, and the non-truly flat regions include other types of pixels besides the truly flat pixels;

[0084] S303 performs noise reduction on pixels in truly flat areas and pixels in non-truly flat areas separately to obtain noise-reduced pixel values;

[0085] Optionally, in step S303, the pixel matrix of the truly flat region is denoised to obtain denoised pixel values, specifically including:

[0086] For pixels in truly flat areas, obtain the corresponding initial pixel values;

[0087] Define a first pixel to be denoised, and define a first denoising region centered on the first pixel to be denoised;

[0088] The first reference mean value of the initial pixels in the first noise reduction area is calculated based on the initial pixel value, and used as the noise reduction pixel value of the first pixel to be noise-reduced.

[0089] Optionally, calculating the first reference mean includes:

[0090] Calculate the average of the initial pixel values ​​of all pixels in the first noise reduction region, and use this as the first reference average; or,

[0091] Define a first processing threshold and a second processing threshold that is greater than the first processing threshold;

[0092] All initial pixel values ​​in the first noise reduction region are compared with the first processing threshold and the second processing threshold. Initial pixel values ​​that are lower than the first processing threshold and higher than the second processing threshold are slowed down to obtain the corresponding updated pixel values.

[0093] Calculate the average pixel value of the updated pixel value of the first noise reduction region and the remaining initial pixel value, and use it as the first reference average.

[0094] Optionally, the absolute value of the first difference between the first processing threshold and the mean of the initial pixel value, |a|, is equal to the absolute value of the second difference between the second processing threshold and the mean of the initial pixel value, |b|. For example, if the absolute value of the first difference, |a|, and the absolute value of the second difference, |b| are both 50% of the first reference mean, then when the first reference mean is 30, the corresponding first processing threshold is 15 and the second processing threshold is 45.

[0095] Specifically, pixels below the first processing threshold and above the second processing threshold undergo a slowdown process, meaning pixels between the two thresholds retain their original pixel values. For example, if the pixel values ​​are 20, 30, 80, 90, 100, 150, and 180, where 120 and 80 are the first and second processing thresholds respectively, pixels above the threshold of 120 undergo a slowdown process, subtracting 50 from 150 to obtain 100, and subtracting 100 from 180 to obtain 80.

[0096] Optionally, in step S303, noise reduction is performed on the pixels in the non-truly flat region to obtain noise-reduced pixel values, specifically including:

[0097] For pixels in non-truly flat regions, obtain the corresponding initial pixel values;

[0098] Define a second pixel to be denoised, and define a second denoising region centered on the second pixel to be denoised;

[0099] Based on the initial pixel value, the second reference mean value of the pixels in the second noise reduction area is calculated as the noise reduction pixel value of the second pixel to be noise-reduced.

[0100] Among them, the selection of pixels to be denoised for the second pixel with different true target directions is different.

[0101] The difference lies in the different methods of selecting the pixels that yield the second reference mean.

[0102] Optionally, the method for selecting the pixels from which the second reference mean is obtained includes:

[0103] For the second pixel to be denoised with a true target direction of 0 degrees, select the pixels in the second denoising area that are in the same row as the second pixel to be denoised and calculate to obtain the second reference value.

[0104] For the second pixel to be denoised with a true target direction of 90 degrees, select the pixels in the second denoising area that are in the same column as the second pixel to be denoised and calculate to obtain the second reference value;

[0105] For the second pixel to be denoised with a true target direction of 45 degrees, the second reference value is obtained by selecting the pixel on the diagonal of the second pixel to be denoised with a 45-degree angle in the second denoising area.

[0106] For the second pixel to be denoised with a true target direction of 135 degrees, the second reference value is obtained by calculating the pixel on the diagonal of the second denoising region that is 135 degrees away from the second pixel to be denoised.

[0107] Optionally, the method for selecting the pixels from which the second reference mean is obtained also includes:

[0108] The second reference mean pixel is obtained by selecting from the extended row groups, wherein the extended row groups include several pixel rows extending to both sides from the selected row corresponding to the preset direction.

[0109] The extended row group includes several pixel rows extending to both sides from the selected row corresponding to the preset direction. These can be: for a 5*5 pixel matrix with the preset direction of the middle pixel being 0 degrees, it can be extended to a 7*11 pixel matrix; for a 5*5 pixel matrix with the preset direction of the middle pixel being 90 degrees, it can be extended to a 7*7 pixel matrix; for a 5*5 pixel matrix with the preset directions of the middle pixels being 45 degrees and 135 degrees, it can be extended to a 7*7 pixel matrix.

[0110] Based on the same concept as the foregoing embodiments, this application provides an electronic device, such as an image sensor. The image sensor includes a memory and a processor; the processor executes a computer program stored in the memory to implement the steps of the image processing method described above. The image sensor can be applied to electronic devices such as cameras, mobile phones, personal digital assistants, computers, monitoring equipment, and machine vision equipment.

[0111] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described image processing method and / or the above-described image noise reduction method.

[0112] In summary, the RAW domain image processing method, image denoising method, and electronic device provided in this application obtain the pixel matrix of the image to be processed, and obtain the initial target confidence and the initial target direction corresponding to the initial target confidence of each pixel in the pixel matrix; perform double threshold segmentation based on the initial target confidence of each pixel to obtain coarse edge information of the pixel array; and obtain subdivided edge information of the pixel array based on the obtained coarse edge information and the initial target direction, and denoise the image pixels, thereby greatly improving the brightness of the image and optimizing the image quality.

[0113] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0115] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of a feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" as used herein are interpreted as inclusive, or mean any one or any combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0116] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0117] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A RAW domain image processing method, characterized in that, It includes: Obtain the pixel matrix of the image to be processed, and obtain the initial target confidence score and the initial target direction corresponding to the initial target confidence score for each pixel in the pixel matrix; Double threshold segmentation is performed based on the initial target confidence of each pixel to obtain coarse edge information of the pixel matrix; as well as Based on the obtained coarse edge information, and based on the initial target direction, the subdivided edge information of the pixel matrix is ​​obtained; The process includes obtaining the pixel matrix of the image to be processed, and obtaining the initial target confidence score and the initial target direction corresponding to the initial target confidence score for each pixel in the pixel matrix, including: The gradient of each pixel in a preset direction is calculated by the edge detection operator as a confidence level, and the maximum value of each confidence level is output as the initial target confidence level of the corresponding pixel. The preset direction corresponding to the initial target confidence level is used as the initial target direction. The process of obtaining the coarse edge information of the pixel matrix includes: Define a first preset threshold and a second preset threshold that is greater than the first preset threshold; Pixels with an initial target confidence level lower than the first preset threshold are identified as initial flat pixels and are located in a flat region; pixels with an initial target confidence level higher than the second preset threshold are identified as initial edge pixels and are located in an edge region; the remaining pixels are identified as initial undetermined pixels and are located in an undetermined region. The process of obtaining the subdivision edge information of the pixel matrix includes: Isolated point detection is performed on the initial edge pixels to obtain target edge pixels and supplementary undetermined pixels, wherein the initial target direction of the target edge pixels is defined as the true target direction of the target edge pixels; The process of performing isolated point detection on the initial edge pixels to obtain target edge pixels and supplementary undetermined pixels includes: Define the initial edge pixels to be detected, and define an isolated point detection area centered on the initial edge pixels to be detected; If the number of initial edge pixels in the isolated point detection area is greater than a first detection threshold, the initial edge pixel to be detected is determined to be the target edge pixel; otherwise, the initial edge pixel to be detected is determined to be the supplementary undetermined pixel. Alternatively, if the number of initial edge pixels in the isolated point detection area that have the same initial target direction as the initial edge pixel to be detected is greater than a second detection threshold, the initial edge pixel to be detected is determined to be the target edge pixel; otherwise, the initial edge pixel to be detected is determined to be the supplementary undetermined pixel.

2. The RAW domain image processing method according to claim 1, characterized in that, Obtaining the initial target confidence and the initial target direction specifically includes: Edge detection is performed on the pixel matrix using an edge detection operator to obtain the initial target confidence score and the initial target direction.

3. The RAW domain image processing method according to claim 2, characterized in that, The edge detection operator includes the Sobel operator; and / or, the preset direction includes 0 degrees, 45 degrees, 90 degrees and 135 degrees.

4. The RAW domain image processing method according to claim 1, characterized in that, Obtaining the subdivision edge information of the pixel matrix further includes: The supplementary undetermined pixels, the initial undetermined pixels, and the initial flat pixels are subjected to neighborhood main direction voting based on the initial target direction to obtain the true target direction.

5. The RAW domain image processing method according to claim 4, characterized in that, The neighborhood main direction voting is performed on the supplementary undetermined pixels, the initial undetermined pixels, and the initial flat pixels to obtain the true target direction, specifically including: Define the pixel to be voted on, and define the target voting area centered on the pixel to be voted on; The initial target direction of the initial edge pixels and the initial undetermined pixels in the target voting area is counted, and the initial target direction with the most occurrences is defined as the voting direction; The voting direction is compared with the initial target direction of the corresponding pixel to be voted. If they match, the voting direction is defined as the true target direction of the pixel to be voted. If they do not match, the voting direction is defined as the pixel to be assigned.

6. The RAW domain image processing method according to claim 5, characterized in that, The image processing method further includes: When the pixel to be assigned is not defined, for the initial flat pixel, define the pixel to be voted and the target voting area centered on it. In the target voting area, identify the pixel whose initial target direction is most frequently inconsistent with the initial target direction of the pixel to be voted. Define this as an inconsistent voting pixel, and the rest as consistent voting pixels. Count the number of consistent voting pixels in the target voting area corresponding to the inconsistent voting pixels. If the number is less than a preset threshold, the pixel to be voted is a truly flat pixel; the pixels to be voted other than the truly flat pixels are pixels with undetermined direction. Alternatively, when the pixel to be assigned is defined, for the initial flat pixel among the pixels to be assigned, count the number of pixels in the target voting area centered on it whose initial target direction is the voting direction. If the number is less than a preset threshold, then the pixel to be assigned is a truly flat pixel; the pixels to be assigned other than the truly flat pixels are pixels whose direction is yet to be determined.

7. The RAW domain image processing method according to claim 6, characterized in that, Obtaining the subdivision edge information of the pixel matrix further includes: The orientation matching degree of the pixel to be determined is detected based on the initial target orientation to obtain the true target orientation of the pixel to be determined.

8. The RAW domain image processing method according to claim 7, characterized in that, Perform orientation matching degree detection on the pixels whose orientation is to be determined, so as to obtain the true target orientation of the pixels whose orientation is to be determined, specifically including: Define a pixel to be detected in a certain direction, define a matching degree detection area centered on the pixel to be detected in a certain direction, define several score reference matrices, and the size of the score reference matrices is the same as the size of the matching degree detection area; A score reference matrix corresponding to the initial target direction of the pixel to be detected is selected, and the sum of the scores of the initial edge pixels in the matching degree detection area at the corresponding positions of the score reference matrix is ​​calculated as the first score sum, and the sum of the scores of the initial pixel to be detected at the corresponding positions of the score reference matrix is ​​calculated as the second score sum. The first score sum is multiplied by a weight coefficient greater than or equal to 1 to obtain the first weight score, and the second score sum is multiplied by a weight coefficient less than or equal to 1 to obtain the second weight score. If the sum of the first weight score and the second weight score is greater than the matching degree threshold, the initial target direction of the pixel to be detected is defined as its true target direction; if it is less than the matching degree threshold, the voting direction of the pixel to be detected is defined as its true target direction. Alternatively, a score reference matrix corresponding to the initial target direction of the pixel to be detected can be selected, and the sum of the scores of the initial edge pixels and the initial undetermined pixels in the matching degree detection area at the corresponding positions in the score reference matrix can be calculated as a first calculated score. The score reference matrix corresponding to the voting direction of the pixel to be detected can be selected, and the sum of the scores of the initial edge pixels and the initial undetermined pixels in the matching degree detection area at the corresponding positions in the score reference matrix can be calculated as a second calculated score. The initial target direction or voting direction corresponding to the larger of the first calculated score and the second calculated score can be taken as the true target direction of the pixel to be detected.

9. An image denoising method based on the RAW domain image processing method as described in any one of claims 1-8, characterized in that, The image noise reduction method includes: Obtain a pixel matrix with defined truly flat regions and non-truly flat regions; wherein, the truly flat regions include truly flat pixels, and the non-truly flat regions include pixels of other types besides the truly flat pixels; Noise reduction is performed on the pixels in the truly flat region and the pixels in the non-truly flat region respectively to obtain the noise-reduced pixel values.

10. The image denoising method according to claim 9, characterized in that, The pixel matrix of the truly flat region is denoised to obtain the denoised pixel values, specifically including: For the pixels in the truly flat region, obtain the corresponding initial pixel value; Define a first pixel to be denoised, and define a first denoising region centered on the first pixel to be denoised; The first reference mean value of the initial pixels in the first noise reduction region is calculated based on the initial pixel value, and is used as the noise reduction pixel value of the first pixel to be noise-reduced.

11. The image denoising method according to claim 10, characterized in that, Calculating the first reference mean includes: Calculate the average of the initial pixel values ​​of all pixels in the first noise reduction region as the first reference average; or, Define a first processing threshold and a second processing threshold that is greater than the first processing threshold; All the initial pixel values ​​in the first noise reduction region are compared with the first processing threshold and the second processing threshold. The initial pixel values ​​that are lower than the first processing threshold and higher than the second processing threshold are slowed down to obtain the corresponding updated pixel values. The average pixel value of the updated pixel value and the remaining initial pixel value in the first noise reduction region is calculated as the first reference average.

12. The image denoising method according to claim 11, characterized in that, Denoising is performed on the pixels in the non-truly flat region to obtain the denoised pixel values, specifically including: For the pixels in the non-truly flat region, obtain the corresponding initial pixel value; Define a second pixel to be denoised, and define a second denoising region centered on the second pixel to be denoised; Based on the initial pixel value, a second reference mean value of the pixels in the second noise reduction region is calculated, which is used as the noise reduction pixel value of the second pixel to be noise-reduced. Specifically, for the second pixel to be denoised in different true target directions, the selection of the pixels participating in the calculation to obtain the second reference mean may differ.

13. The image denoising method according to claim 12, characterized in that, The method for selecting the pixel that yields the second reference mean includes: For the second pixel to be denoised with a true target direction of 0 degrees, the pixels in the second denoising area that are in the same row as the second pixel to be denoised are selected for calculation to obtain the second reference mean. For the second pixel to be denoised with a true target direction of 90 degrees, the pixels in the second denoising area that are in the same column as the second pixel to be denoised are selected for calculation to obtain the second reference mean. For the second pixel to be denoised with a true target direction of 45 degrees, the pixels on the diagonal of the second pixel to be denoised with a 45-degree angle to the second pixel to be denoised in the second denoising area are selected for calculation to obtain the second reference mean. For the second pixel to be denoised with a true target direction of 135 degrees, the second reference mean is obtained by calculating the pixel on the diagonal of the second pixel to be denoised with a 135-degree angle to the second pixel in the second denoising region.

14. The image denoising method according to claim 12, characterized in that, The method of selecting the pixel from which the second reference mean is obtained also includes: The pixels for which the second reference mean is obtained are selected based on the extended row group, wherein the extended row group includes a number of pixel rows extending to both sides from the selected row corresponding to the preset direction.

15. An electronic device, characterized in that, Including memory and processor; The processor is used to execute the computer program stored in the memory to implement the steps of the image processing method as described in any one of claims 1-8 and / or the image noise reduction method as described in any one of claims 9-14.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the image processing method as described in any one of claims 1-8 and / or the image noise reduction method as described in any one of claims 9-14.

Images